Blocking: more guidance needed

  • AlexKalopsia

    (@snakeeater)


    3 weeks ago

    I wanted to comment on Which options should I check to stop AI scraping? but the thread is closed. I have to agree that in the current state it is very difficult to understand what exactly needs to be blocked.

    I can pick:
    Block AI Assistants
    Block AI Data Providers
    Block AI Data Scrapers
    Block AI Search Crawlers
    Block Archivers
    Block Intelligence Gatherers
    Block Scrapers
    Block SEO Crawlers
    Block Undocumented AI Agents

    and yet, there is no info about what each option does. I don’t know what is the difference between AI Data Providers and AI Data Scrapers and what would be the consequence of blocking one or the other.

    In the thread you recommend checking each agent manually (!!!) but it is still unclear to me how would that even work. Say there is one specific that I want to block, the only option given to me is seemingly to block all of them?

    I do really like the entire concept of DarkVisitors/KnownAgents, and setting it up is very easy, but the onboarding is honestly quite confusing and I’d love to get more guidance.

Viewing 1 replies (of 1 total)
  • Plugin Author Known Agents

    (@gavindarkvisitors)

    3 weeks ago

    This is great feedback, thank you Alex.

    For now, the best option is to select “Block AI Data Scrapers” and “Block Undocumented AI Agents”. These are the bots that are explicitly used for AI model training, or very likely to do so.

    The “AI Data Providers” category is relatively new. These are the bots that scrape and re-sell data to third parties. Those third parties can use it for AI model training, or non-training use cases like citations in AI responses. It’s up to you whether you want to block them, since the tradeoff could be fewer human referrals from AI platforms.

    All of this is explained in our Agent Database if you click into the pages, but we will add a link to this list and some simple descriptions in on the plugin settings page in the next version. Again, I appreciate your suggestion here.

    Please let me know if that makes sense, happy to clarify.

Viewing 1 replies (of 1 total)

You must be logged in to reply to this topic.