blocking good bing bots
-
Hi Bill,
I have been checking the log on my website and found the plugin has been stopping all the good bots from MSN search engines.
I have tried to white list them but they are still getting blocked.
Can you suggest anything I can do to fix it please.
Thanks,
Tracey
-
Hi Tracey,
Please, post in our support page this raw. (from visitors log table)
We need check if is a real bot or fake bot, their IP, their user agent and so on…
http://BillMinozzi.com/support/We use to check the IP before the block if it own from Microsoft (also Google).
PHP have a function to do that:
gethostbyaddr()Cheers,
BillBill,
Here is the info
Your ticket was registered with:
ID: 1530thanks,
Tracey
Hi Tracey,
I wrote the answer to your ticket.
If I understood what you said, that IP is not from Microsoft. It is a fake bot.
I put more details on our ticket.Cheers,
BillBill,
I am talking about
IP 40.77.167.39, 157.55.39.133, 157.55.39.129 and other Bing ones. Not the digital ocean one on the first line.If there was an export feature I send the whole log rather than copying and pasting the first page into a ticket.
Thanks,
TraceyHi,
We use this standard PHP function to check:
gethostbyaddr()
Check if your hosting company don’t block it.If Microsoft server fail to answer correctly, can be blocked. Sometimes can happens. Not always.
You can also whitelist the IP, but usually they change the IP and send the bot again.
You can also add they on the string whitelist.
Cheers,
BillBill,
The plugin is still blocking Bing Bots and they are listed in Stop Bad Bots blocked visits log.
for example
40.77.139.64 is a verified bing bot
this is where we check bing bots
https://www.bing.com/toolbox/verify-bingbotYour log is reporting
17-Aug-2021 19:47:16 Denied 40.77.139.64 Blocked Fake Browser (2) 403 GET Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0bit is also blocking Qwant bots/IPs.
Although we have whitelisted them in Stop Bad Bots it still blocks them. Whitelisting does not appear to work.
Because it is a live site and we are struggling because of the pandemic the last thing we want is Bing, Qwant or other useful bots getting blocked.
The blocks have nothing to do with our hosting company because the blocks are in Stop Bad Bots log.
Any other ideas?
Thanks Tracey
Hi Tracey,
Whitelist only works in premium version.
You told me a couple of weeks you don’t have pro version.Cheers,
BillBill,
Look, there is no point adding a whitelist button next to blocked bots or fake browsers when your software is blocking good bots such as Bing Bot and Qwant. The Whitelist button next to block bots should only appear when one is using a premium version.
You don’t appear to want to fix the plugin as such there is no point having it blocking our customers who use Bing or Qwant.
Not happy and certainly will not upgrade to the pro version if this is your attitude.
Tracey
Hi Tracey,
I wrote above:
If Microsoft server fail to answer correctly, can be blocked. Sometimes can happens. Not always.
“… but usually they change the IP and send the bot again.”If we remove the server check, bad bots will send fake bing bot.
Cheers,
BillHi Tracey,
Please, read our previous post.
I forgot to say.
If you run a search at bing:
“Stop Bad Bots”Our plugin will show up in first place.
It is a fact!
Don’t worry about.
Cheers,
BillHi,
I hope you got this sorted. Its been a while with no reply so gonna go ahead and mark this thread as resolved to help keep the forum organized. Feel free to post again with any further questions or feedback,
Thank you.Bill
The topic ‘blocking good bing bots’ is closed to new replies.