High Algolia traffic

Resolved Topher
(@topher1kenobe)

4 months, 2 weeks ago
We installed this plugin and got it working just fine. The issue is that Algolia traffic is many times higher than we expected. Based on previous search stats we expected to stay within the free tier for a little bit, but almost immediately we’re in the hundreds of dollars per month range.

Algolia support says front end requests should come from the IP of the end user, but all of ours are coming from the host.

We’re using:
- Use Algolia with the native WordPress search template
- Legacy (instantsearch.php)
- Autocomplete is off
Any ideas?

The page I need help with: [log in to see the link]

Viewing 6 replies - 1 through 6 (of 6 total)

Plugin Contributor Michael Beckwith
(@tw2113)

The BenchPresser

4 months, 2 weeks ago

Hey Topher, long time no talk. Hope all is well.

Love the fact that Fangoria is using Algolia + WP Search with Algolia. Anyways.

The links below may have a certain amount of overlap, but by and large they focus on what can be implemented to help with backend based searches, which you’re using if you have the “Use Algolia with the native WP Search template” toggled.

https://support.algolia.com/hc/en-us/articles/10154087058705-How-do-I-prevent-or-reduce-bot-traffic-in-Algolia-Search
https://www.algolia.com/doc/guides/security/api-keys
https://www.algolia.com/doc/guides/security/api-keys/in-depth/api-key-restrictions
https://support.algolia.com/hc/en-us/articles/4406981900433-Monitoring-Search-Operations
https://support.algolia.com/hc/en-us/articles/4406033355793-Handling-High-Levels-of-Bot-Traffic

On the flipside, I’m curious what the search analytics are for the install. Especially for what could be a bunch of spam/junk queries.

I created some function checks to use with pre_get_posts and search queries for various cases like “has emojis”, “non-latin characters”, is just a URL, is just an email, starts with dashes, or just generally odd characters. Helped clean up no-results spam before the request hits Algolia.

You can find these and adapt as needed at https://github.com/WebDevStudios/algolia-snippet-library/blob/main/native-search/reject-spam-like-native-search-queries.md

Let us know if you have any questions or concerns regarding this. Thanks.

Thread Starter Topher
(@topher1kenobe)

4 months, 2 weeks ago

Thanks a bunch Michael. Do you know of any good search query logging plugins? Does Algolia log that stuff?

Plugin Contributor Michael Beckwith
(@tw2113)

The BenchPresser

4 months, 2 weeks ago

If you have access to the Algolia dashboard and whatnot, then https://dashboard.algolia.com/apps/MY APP ID/analytics/overview/ could provide some immediate data, especially for what’s making it through to Algolia itself.

I’m not presently sure of any other logging at the moment. We don’t have anything baked into the Free plugin, but at least in terms of backend search, listening to queries with $query->is_search() and pre_get_posts could potentially open up options for rudimentary logging.

Plugin Contributor Michael Beckwith
(@tw2113)

The BenchPresser

4 months, 2 weeks ago

Good day @topher1kenobe

Any chance to try some of these ideas/resources out yet? If yes, any noticeable difference in the incoming traffic? Hopefully for the better.

Thread Starter Topher
(@topher1kenobe)

4 months, 2 weeks ago

I gave the info the Algolia manager and he took it to the webhost who did a bunch of traffic blocking I think. Regardless, it’s working the way the client wants.

Plugin Contributor Michael Beckwith
(@tw2113)

The BenchPresser

4 months, 2 weeks ago

Awesome to hear. As always, let us know if anything else comes up with the plugin and we’ll do what we can to help.

Viewing 6 replies - 1 through 6 (of 6 total)

You must be logged in to reply to this topic.