• We get tons of traffic of traffic from a couple of other sites, but not actually views: they are just requests from their WordPress RSS widgets (displaying our feed) every time someone visits their pages. Most of the return codes are 304 (not modified). Many of the requests come via Automattic (ie, WordPress) IPs (one of the sites appears to be hosted there).

    Our htaccess file includes ExpiresByType (and AddType) and “Header set Cache-Control” directives that should obviate most of the repeated requests, right?

    Since they aren’t what htaccess cache settings for the feed should we have? Or is there something else we need? It may be notable that this started to be noted about 4 months ago (suggesting a change in those widgets?).

Viewing 7 replies - 1 through 7 (of 7 total)
  • Moderator bcworkz

    (@bcworkz)

    ExpiresByType should work, but you must use the right MIME type for RSS feeds. Correct would be application/rss+xml, but text/xml is commonly used. Use both. Cache directives are really merely suggestions, browsers could ignore them and re-fetch the data anyway.

    Cache control headers are preferred over Expires. You might try this (untested):

    <ifModule mod_headers.c> 
        # One week (in seconds) for feeds
        <Location "/feed">
            Header set Cache-Control "max-age=604800, public"
        </Location>
    </ifModule>

    Change the max-age time as desired.

    Thread Starter ericr23

    (@ericr23)

    Many thanks. I currently have that directive in <FilesMatch “.(htm|html|js|php|rss|txt|xml)$”>. I’ll try your suggested wrapper.

    I have read that Cache-Control applies only to direct requests of those files and then overrides ExpiresByType.

    Thread Starter ericr23

    (@ericr23)

    Testing …

    Thread Starter ericr23

    (@ericr23)

    I was wondering how the WP Super Cache plug-in might be involved and turned it off for feeds. That changed the responses from 304 to 200, so I turned it back on. So the question remains of how to avoid the repeated requests altogether (ie, cache at Automattic instead of getting repeated 304 responses). Virtually all of the 304 responses involve WordPress feeds (the site has a few non-Wordpress feeds as well). I wonder if setting a header specially in WP is something to try.

    Moderator bcworkz

    (@bcworkz)

    Don’t forget that .htaccess does not know anything about WP. Using FilesMatch will not help with WP feed requests because there are no server files to match against. This is why Location (or LocationMatch) should be used since it can match against HTTPS requests and not server resources.

    It’s up to browsers to interpret caching directives, so results could be variable. I’m guessing which ever header arrives last will take precedence in case of conflict. In any case, you should want to avoid headers that conflict with another. Use Expires or Cache Control as you prefer, but not both for the same request. Or if both are sent, they ought to at least represent the same time frame. My understanding is Cache Control is preferred because it offers more flexibility, but if Expires works for you there’s nothing wrong with using it. But it has to be properly applied, FilesMatch will not do it 😉

    I wonder if setting a header specially in WP is something to try.

    Perhaps, but the best approach is to handle requests before they ever get to WP. Getting WP involved uses a lot more server resource than handling .htaccess directives. I don’t think core WP even knows how to make a proper 304 response, which is in part why caching plugins are useful.

    In theory, 304 responses are only appropriate if the client app makes a conditional request, such as If-Modified-Since. It wouldn’t the only server response that’s served inappropriately 🙂

    In the end, you cannot really minimize feed requests, all you can do is to try to minimize the impact on your server. Probably the best you can do is use a good caching plugin, configure it as best you can, then let it do its thing.

    Thread Starter ericr23

    (@ericr23)

    Thank you. Those are pretty much the conclusions I’ve been arriving at. A 304 response certainly uses less server resource than a 200 response (and with the caching plugin, the 200 response uses less server resource than it would otherwise).

    Thread Starter ericr23

    (@ericr23)

    I added the following to robots.txt, and that has brought it under control.

    User-agent: WordPress
    Crawl-delay: 8
Viewing 7 replies - 1 through 7 (of 7 total)

The topic ‘htaccess cache settings for feeds’ is closed to new replies.