search engines
-
my wordfence logs are showing msn and yahoo as blocked
and option to allow them to run as Google’s crawlers
-
Can you post a screenshot of the issue? You might want to relax some of the firewall rules on your site if real crawlers are being blocked.
There are also fake crawlers that pretend to be valid crawlers, so you could also check the IP addresses to be certain that they really belong to MSN or Yahoo.
i cant post a picture
but i can C&PRussia Russia IP: 130.193.48.1 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: spider-130-193-48-1.yandex.com Throttled 114 times starting 29 days 1 hour ago and ending 2 days 14 hours ago. Switzerland Switzerland IP: 194.56.244.224 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: 194.56.244.224 Throttled 2 times starting 7 days 19 hours ago and ending 7 days 19 hours ago. United States Redmond, United States IP: 207.46.13.3 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-3.search.msn.com Throttled 12 times starting 9 months 22 days ago and ending 10 days 4 hours ago. United States Redmond, United States IP: 157.55.39.119 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-119.search.msn.com Throttled 5 times starting 10 days 4 hours ago and ending 10 days 4 hours ago. United States Redmond, United States IP: 157.55.39.178 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-178.search.msn.com Throttled 10 times starting 19 days 3 hours ago and ending 10 days 4 hours ago. Sweden Sweden IP: 195.35.83.145 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: dedi74.aname.net Throttled 170 times starting 10 days 20 hours ago and ending 10 days 19 hours ago. United States Redmond, United States IP: 157.55.39.227 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-227.search.msn.com Throttled 2 times starting 12 days 14 hours ago and ending 12 days 14 hours ago. United States Redmond, United States IP: 207.46.13.45 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-45.search.msn.com Throttled 2 times starting 12 days 14 hours ago and ending 12 days 14 hours ago. United States Redmond, United States IP: 157.55.39.118 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-118.search.msn.com Throttled 153 times starting 2 months 25 days ago and ending 12 days 14 hours ago. United States Redmond, United States IP: 157.55.39.174 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-174.search.msn.com Throttled 1 times starting 15 days 20 hours ago and ending 15 days 20 hours ago. United States Redmond, United States IP: 207.46.13.85 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-85.search.msn.com Throttled 104 times starting 16 days 10 hours ago and ending 16 days 10 hours ago. United States Redmond, United States IP: 157.55.39.99 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-99.search.msn.com Throttled 10 times starting 3 months ago and ending 16 days 11 hours ago. United States Redmond, United States IP: 157.55.39.191 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-191.search.msn.com Throttled 25 times starting 16 days 15 hours ago and ending 16 days 15 hours ago. United States Redmond, United States IP: 157.55.39.222 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-222.search.msn.com Throttled 2 times starting 16 days 15 hours ago and ending 16 days 15 hours ago. United States Redmond, United States IP: 157.55.39.190 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-190.search.msn.com Throttled 2 times starting 16 days 15 hours ago and ending 16 days 15 hours ago. United States Redmond, United States IP: 157.55.39.243 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-243.search.msn.com Throttled 14 times starting 3 months 6 days ago and ending 18 days 19 hours ago. United States Redmond, United States IP: 157.55.39.50 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-50.search.msn.com Throttled 53 times starting 19 days 3 hours ago and ending 19 days 2 hours ago. United States Redmond, United States IP: 157.55.39.49 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-49.search.msn.com Throttled 3 times starting 19 days 3 hours ago and ending 19 days 2 hours ago. United States Redmond, United States IP: 157.55.39.179 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-179.search.msn.com Throttled 21 times starting 19 days 3 hours ago and ending 19 days 3 hours ago. United States Redmond, United States IP: 157.55.39.107 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-107.search.msn.com Throttled 102 times starting 19 days 3 hours ago and ending 19 days 3 hours ago. Germany Germany IP: 148.251.45.130 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: static.130.45.251.148.clients.your-server.de Throttled 15 times starting 4 months 28 days ago and ending 21 days 9 hours ago. United States Redmond, United States IP: 207.46.13.61 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-61.search.msn.com Throttled 66 times starting 22 days 5 hours ago and ending 21 days 22 hours ago. United States Redmond, United States IP: 207.46.13.42 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-42.search.msn.com Throttled 61 times starting 22 days 4 hours ago and ending 22 days 4 hours ago. United States Redmond, United States IP: 207.46.13.94 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-94.search.msn.com Throttled 89 times starting 22 days 5 hours ago and ending 22 days 5 hours ago. United States Redmond, United States IP: 157.55.39.115 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-115.search.msn.com Throttled 103 times starting 22 days 5 hours ago and ending 22 days 5 hours ago. United States Redmond, United States IP: 207.46.13.115 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-115.search.msn.com Throttled 11 times starting 29 days 20 hours ago and ending 22 days 17 hours ago. United States Redmond, United States IP: 157.55.39.238 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-238.search.msn.com Throttled 5 times starting 24 days 3 hours ago and ending 23 days 12 hours ago. United States Redmond, United States IP: 157.55.39.160 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-160.search.msn.com Throttled 3 times starting 24 days 2 hours ago and ending 24 days 2 hours ago. United States Redmond, United States IP: 207.46.13.2 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-2.search.msn.com Throttled 1 times starting 24 days 13 hours ago and ending 24 days 13 hours ago. Russia Russia IP: 178.154.243.74 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: spider-178-154-243-74.yandex.com Throttled 5 times starting 24 days 20 hours ago and ending 24 days 14 hours ago. United States Redmond, United States IP: 157.55.39.28 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-28.search.msn.com Throttled 7 times starting 2 months 25 days ago and ending 24 days 19 hours ago. United States Redmond, United States IP: 207.46.13.93 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-93.search.msn.com Throttled 13 times starting 24 days 19 hours ago and ending 24 days 19 hours ago. United States United States IP: 45.35.20.203 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: 45.35.20.203 Throttled 17 times starting 25 days 5 hours ago and ending 25 days 5 hours ago. United States Redmond, United States IP: 207.46.13.104 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-104.search.msn.com Throttled 103 times starting 26 days 20 hours ago and ending 26 days 19 hours ago. United States Redmond, United States IP: 157.55.39.187 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-187.search.msn.com Throttled 35 times starting 27 days 3 hours ago and ending 27 days 3 hours ago. United States Redmond, United States IP: 157.55.39.188 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-188.search.msn.com Throttled 1 times starting 27 days 3 hours ago and ending 27 days 3 hours ago. United States Redmond, United States IP: 207.46.13.141 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-141.search.msn.com Throttled 222 times starting 8 months 22 days ago and ending 28 days 13 hours ago. United States Redmond, United States IP: 157.55.39.244 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-244.search.msn.com Throttled 12 times starting 3 months 6 days ago and ending 28 days 14 hours ago. United States Redmond, United States IP: 207.46.13.119 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-119.search.msn.com Throttled 82 times starting 1 month ago and ending 28 days 15 hours ago. United States Redmond, United States IP: 157.55.39.67 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-67.search.msn.com Throttled 108 times starting 28 days 15 hours ago and ending 28 days 15 hours ago. United States Redmond, United States IP: 157.55.39.82 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-82.search.msn.com Throttled 2 times starting 28 days 15 hours ago and ending 28 days 15 hours ago. United States Redmond, United States IP: 157.55.39.162 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-162.search.msn.com Throttled 32 times starting 3 months 3 days ago and ending 29 days 9 hours ago. France France IP: 62.210.90.118 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: 62-210-90-118.rev.poneytelecom.eu Throttled 256 times starting 7 months 21 days ago and ending 1 month 3 days ago. United States Redmond, United States IP: 207.46.13.142 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-142.search.msn.com Throttled 6 times starting 1 month 5 days ago and ending 1 month 5 days ago. United States Redmond, United States IP: 207.46.13.121 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-121.search.msn.com Throttled 9 times starting 1 month 5 days ago and ending 1 month 5 days ago. United States Redmond, United States IP: 157.55.39.251 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-251.search.msn.com Throttled 1 times starting 1 month 9 days ago and ending 1 month 9 days ago. United States Redmond, United States IP: 207.46.13.77 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-77.search.msn.com Throttled 4 times starting 1 month 14 days ago and ending 1 month 14 days ago. United States Redmond, United States IP: 207.46.13.40 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-40.search.msn.com Throttled 13 times starting 2 months 16 days ago and ending 1 month 14 days ago. United States Redmond, United States IP: 207.46.13.68 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-207-46-13-68.search.msn.com Throttled 3 times starting 1 month 14 days ago and ending 1 month 14 days ago. United States Redmond, United States IP: 157.55.39.88 Reason: Exceeded the maximum number of page not found errors per minute for humans. Hostname: msnbot-157-55-39-88.search.msn.com Throttled 17 times starting 2 months 21 days ago and ending 1 month 14 days agoi cant post IMG
but its C&P
Russia Russia
IP: 130.193.48.1
Reason: Exceeded the maximum number of page not found errors per minute for humans.
Hostname: spider-130-193-48-1.yandex.com
Throttled 114 times starting 29 days 1 hour ago and ending 2 days 14 hours ago.
Switzerland Switzerland
IP: 194.56.244.224
Reason: Exceeded the maximum number of page not found errors per minute for humans.
Hostname: 194.56.244.224
Throttled 2 times starting 7 days 19 hours ago and ending 7 days 19 hours ago.
United States Redmond, United StatesRussia Russia
IP: 130.193.48.1
Reason: Exceeded the maximum number of page not found errors per minute for humans.
Hostname: spider-130-193-48-1.yandex.com
Throttled 114 times starting 29 days 1 hour ago and ending 2 days 14 hours ago.
Switzerland Switzerland
IP: 194.56.244.224
Reason: Exceeded the maximum number of page not found errors per minute for humans.
Hostname: 194.56.244.224
Throttled 2 times starting 7 days 19 hours ago and ending 7 days 19 hours ago.
United States Redmond, United States
IP: 207.46.13.3
Reason: Exceeded the maximum number of page not found errors per minute for humans.
Hostname: msnbot-207-46-13-3.search.msn.com
Throttled 12 times starting 9 months 22 days ago and ending 10 days 4 hours ago.
United States Redmond, United States
IP: 157.55.39.119
Reason: Exceeded the maximum number of page not found errors per minute for humans.
Hostname: msnbot-157-55-39-119.search.msn.com
Throttled 5 times starting 10 days 4 hours ago and ending 10 days 4 hours ago.
United States Redmond, United StatesAre you using the current version of Wordfence, 6.0.17?
A couple versions ago, we had an issue where crawlers were being counted as humans, so they were being blocked by the “404” firewall rule for humans (if it was set low), and it looks like that might be what is happening on your site. If that’s the case, updating Wordfence should fix it.
If you’re already up to date, let us know, and we’ll look at other possibilities!
yes i have the latest version
and thanksDid it work previously? I see at least one line says one IP was throttled 9 months ago, which was before we had any known issues.
There could be a conflict with another plugin or theme, or there may be something wrong with your database tables. You could try temporarily disabling other plugins and switching to a default theme like twentyfifteen, to check if crawlers are labeled correctly after that.
Alternately, you could try uninstalling Wordfence after enabling the option “Delete Wordfence tables and data on deactivation”, and then reinstalling it. (This will lose your settings unless you use the Export button at the bottom and save the key — but this is the best way to reset everything.)
for now i disabled the plugin
and run iThemes Security
i need to protect the system files as .htaccess
and the plugin iThemes Security give me this optionwordfence is good plugin but dot protect my cure files – free edition
Ok. If you have any other questions, let us know. We haven’t had any other reports of this with the latest version, so there was definitely something unusual happening.
I believe Wordfence and iThemes Security still can work together, as long as the options are set correctly so that they are not both trying to handle the same functions.
The topic ‘search engines’ is closed to new replies.
