Do Plugins Make Permanent Changes to Code?

I’ve been trying to find the best plugins for various functions and I’ve been going through many different plugins for a specific function (i.e. caching plugins, image optimizer plugins, etc) to find the one that performs the best for my site.

Is there any potential danger in installing and then deactivating/deleting multiple plugins like this? (I’m worried that by installing one plugin it makes changes to the code permanently and then mucks up the site from here on out and I’ll never even know about it)

Or is it more like when you deactivate/delete a plugin that it is as if it was never there and you start over from where you started before the install?

Thanks!