Plugin Author
AITpro
(@aitpro)
Yes, BPS does block HEAD Requests by default. To allow HEAD Requests do these Custom Code steps in this link: http://forum.ait-pro.com/forums/topic/itunes-cannot-read-feed/#post-2787
I followed those instructions to the letter, and tested it out by sending my own request from terminal. Still getting the error.
Plugin Author
AITpro
(@aitpro)
Hmm I wonder if the way cURL is being used is the issue?
Do these manual steps for testing and if this works then I will post permanent Custom Code steps.
1. Go to the htaccess Core >>> htaccess File Editor tab page.
2. Unlock your Root htaccess file if it is currently locked.
3. Click on the “Your Current Root htaccess File” tab.
4. Scroll down in the contents of your Root htaccess file until you come to this section of code: BPSQSE BPS QUERY STRING EXPLOITS
5. Comment out these 2 lines of code using # signs as shown below:
#RewriteCond %{HTTP_USER_AGENT} (havij|libwww-perl|wget|python|nikto|curl|scan|java|winhttp|clshttp|loader) [NC,OR]
#RewriteCond %{HTTP_USER_AGENT} (;|<|>|'|"|\)|\(|%0A|%0D|%22|%27|%28|%3C|%3E|%00).*(libwww-perl|wget|python|nikto|curl|scan|java|winhttp|HTTrack|clshttp|archiver|loader|email|harvest|extract|grab|miner) [NC,OR]
6. Click the Update File button to save your edits.
Test and let me know what happens.
Plugin Author
AITpro
(@aitpro)
Hmm no that would not make a lot of sense. Double check that you did all the Custom Code steps correctly. Also that is a capital letter (eye) “I” correct? curl -I is a HEAD Request. This would be a cURL GET Request (no command-line used):
curl http://findrefuge.com/?feed=seriesengine
Plugin Author
AITpro
(@aitpro)
Just made a HEAD Request to your website with web-sniffer and got a 200 Response. Is the problem still occurring now? If so, then it is probably going to be plugin related. Check your BPS Security Log file and post the security log entry related to what is being blocked/a plugin / the URI.
Sent request from terminal on Mac:
curl -I http://findrefuge.com/?feed=seriesengine
HTTP/1.1 403 Forbidden
Date: Sat, 06 Sep 2014 16:52:34 GMT
Server: Apache
Connection: close
Content-Type: text/html; charset=iso-8859-1
Plugin Author
AITpro
(@aitpro)
Using web-sniffer.net and making a HEAD Request to this URL: http://findrefuge.com/?feed=seriesengine
HTTP Request Header
Connect to 67.227.136.8 on port 80 ... ok
HEAD /?feed=seriesengine HTTP/1.1[CRLF]
Host: findrefuge.com[CRLF]
Connection: close[CRLF]
User-Agent: Web-sniffer/1.1.0 (+http://web-sniffer.net/)[CRLF]
Accept-Encoding: gzip[CRLF]
Accept-Charset: ISO-8859-1,UTF-8;q=0.7,*;q=0.7[CRLF]
Cache-Control: no-cache[CRLF]
Accept-Language: de,en;q=0.7,en-us;q=0.3[CRLF]
Referer: http://web-sniffer.net/[CRLF]
HTTP Response Header
Status: HTTP/1.1 200 OK
Date: Sat, 06 Sep 2014 16:55:58 GMT
Server: Apache
X-Powered-By: PHP/5.3.28
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
ETag: "f87b70f37b9b9ed1fdd33bf865540e97"
Set-Cookie: PHPSESSID=763d931e40fb1473a5b8d4f8a7c9078d; path=/
Last-Modified: Thu, 04 Sep 2014 17:32:00 GMT
Connection: close
Content-Type: text/xml; charset=UTF-8
Went to your feed and played an mp3 podcast successfully using Firefox.
So my question is are the podcasts actually working and is the error still occurring from Apple? I am going to assume that the Terminal check from your Mac is not accurate at this point or is being blocked because of the way it is being sent or some other odd reason. I am able to make a HEAD Request to your site and feed and can view/listen to mp3 podcasts in your feed.
There’s a bunch of entries in the security log that look like this:
[403 GET / HEAD Request: September 5, 2014 5:10 pm]
Event Code: BFHS – Blocked/Forbidden Hacker or Spammer
Solution: N/A – Hacker/Spammer Blocked/Forbidden
REMOTE_ADDR: 17.154.84.42
Host Name: 17.154.84.42
SERVER_PROTOCOL: HTTP/1.1
HTTP_CLIENT_IP:
HTTP_FORWARDED:
HTTP_X_FORWARDED_FOR:
HTTP_X_CLUSTER_CLIENT_IP:
REQUEST_METHOD: GET
HTTP_REFERER:
REQUEST_URI: /?feed=seriesengine
QUERY_STRING:
HTTP_USER_AGENT: iTMS
Plugin Author
AITpro
(@aitpro)
Do you see any Security Log entries for today? The date of that log entry is yesterday: September 5, 2014 5:10 pm
Test using Apple/iTunes itself since Apple/iTunes will tell you if they are still being blocked. I assume the Mac Terminal check is blocked for some other reason. It is probably malformed or is sending the Request in a way that is fubar.
Plugin Author
AITpro
(@aitpro)
This is just an FYI and not related to anything else: http://feedvalidator.org/check.cgi?url=http%3A%2F%2Ffindrefuge.com%2F%3Ffeed%3Dseriesengine
These feed validation “errors” are not critical, but you should look at why they are occurring and fix them.
Plugin Author
AITpro
(@aitpro)
It’s been a couple days since this thread was created. Is this issue/problem resolved? If so, please resolve this thread. If not, please post any additional relevant information.
Plugin Author
AITpro
(@aitpro)
It’s been a couple days since this thread was created.
The intended purpose of the statement above is to date the thread. We will use this method below instead from now on.
Thread Start Date: 9-6-2014
Current Date: 9-8-2014
Sorry. Yes, the problem has been resolved. I checked the feed in itunes, and all the episodes are showing up now. Thanks very much.