Disabling database caching worked for me (I’m with Gandi Simple Hosting). The others seemed to not have any adverse effect one way or the other.
I believe that page caching will also cause a page to always show the same sequence of quotes, but the sequence will change from page to page. Page caching does have a big impact on browser reloading, though, so I recommend leaving it on.
To test your changes, don’t forget to flush all your caches, or at least your page cache, before reloading each page in your browser.
Thanks for the hints, jsimonms!