Black URL

  • Resolved Anonymous User 14978628

    (@anonymized-14978628)


    9 years, 2 months ago

    Hi Jeff,

    Just wanted a quick clarification if possible?

    You Say that if using a caching plugin to:

    Add the blackhole URL (available via the “Robots Rules” setting) to your caching plugin’s “ignore” list

    The blackhole url in the robots rules setting is:

    ?blackhole

    Do i add that? Or the full url that is actually displayed on my page in the source code with the number/letter string on the end e.g.,:

    https://example.com/?blackhole=1234567890

    Also, when adding an exclusion to a caching plugin (wpfastestcache) there is an option to exclude pages with the following rules:

    If_Request_URI – Contains
    If_Request_URI – Is equal to

    Which would be best to use?

    Thanks

Viewing 15 replies - 1 through 15 (of 22 total)
1 2
  • Plugin Author Jeff Starr

    (@specialk)

    9 years, 2 months ago

    Great question. For the exclude rules in a caching plugin, it may take some experimentation (I haven’t yet tested with all caching plugins), but the idea is to whitelist/ignore any URL that contains the string:

    blackhole

    So maybe try using the “If_Request_URI” and then include blackhole should be good. Remember to test well.

    Edited to add: let me know if you discover any other caching plugins that are compatible so I can add them to the list, thank you.

    Edited to add: for the robots.txt rules, you should add exactly what is displayed in the Robots Rules setting (do not exclude the first forward slash).

    • This reply was modified 9 years, 2 months ago by Jeff Starr.
    • This reply was modified 9 years, 2 months ago by Jeff Starr.
    Thread Starter Anonymous User 14978628

    (@anonymized-14978628)

    9 years, 2 months ago

    Hi Jeff, could you tell me, does the number string after ?blackhole remain the same all the time, or does it change?

    e.g.,

    ?blackhole=1234567890

    does “=1234567890” remain the same? If so i can just add the whole url string as an exclusion e.g.,

    https://example.com/?blackhole=1234567890

    Thanks

    Plugin Author Jeff Starr

    (@specialk)

    9 years, 2 months ago

    It’s a security nonce. It changes. Again, I would just use the string that’s provided in the plugin settings unless you know what you are doing and have good reason to do otherwise.

    Thread Starter Anonymous User 14978628

    (@anonymized-14978628)

    9 years, 2 months ago

    ok thanks Jeff, I tested this out and it works out with wpfastestcache, but it doesn’t seem to work with cloudflare enabled even after excluding the blackhole string from the cache (e.g., http://www.example.com/?blackhole*) which is strange.

    With cloudflare enabled, i can visit the blackhole url and get banned from the site. But then when i try to visit the site again i am able to view it. I can browse a few pages and then sometimes it will tell me i am banned, other times it won’t. So not fully functional yet with cloudflare.

    Are there any issues you are aware of with cloudflare and BBQ Pro?

    Plugin Author Jeff Starr

    (@specialk)

    9 years, 2 months ago

    Thanks for the infos. I should mention that, after adding the blackhole string to any cache ignore rules, it also will be necessary to save changes and delete the old cache. Otherwise the cached page will continue to be served in most cases.

    No reported issues with cloudflare and BBQ Pro, which operates completely differently than Blackhole Pro, so there should never be an issue with caching.

    Thread Starter Anonymous User 14978628

    (@anonymized-14978628)

    9 years, 2 months ago

    I tried again this time adding an exclusion in cloudflare and then deleting cache but got same problem. When i visited the blackhole link it told me i was banned, but then when i pasted a url into the browser i was able to browse the site as normal. The ban doesn’t seem to be sticking for some reason?

    Plugin Author Jeff Starr

    (@specialk)

    9 years, 2 months ago

    Yeah probably because of the caching. Did you try excluding the URL and also deleting/resetting the cache?

    Thread Starter Anonymous User 14978628

    (@anonymized-14978628)

    9 years, 2 months ago

    Yes, I excluded the url as http://www.example.com/?blackhole* and cleared cache.

    Is this the right way to exclude the url? That’s what i did with wpfastestcache and it worked.

    Perhaps i should try http://www.example.com/?blackhole=*

    Plugin Author Jeff Starr

    (@specialk)

    9 years, 2 months ago

    As explained, I don’t use cloudflare, so I don’t know the correct syntax. If it were me, I would read the cloudflare documentation, try every possible URL combination, and remember to delete the cache for each new try.

    Thread Starter Anonymous User 14978628

    (@anonymized-14978628)

    9 years, 2 months ago

    Yes, I know you don’t use cloudflare. The wildcard is how cloudlfare explains it should be used which is why i was surprised the exclusion didn’t seem to be working. Will have to play around with this i guess.

    This is from cloudflare:

    If the URL matches: By using the asterisk (*) character, you can create dynamic patterns that can match many URLs, rather than just one

    example: http://www.example.com/*

    Plugin Author Jeff Starr

    (@specialk)

    9 years, 2 months ago

    So then I would try adding both just in case:

    http://www.example.com/?blackhole*
    http://www.example.com/?blackhole=*

    Then save changes and delete the current cache.

    If that doesn’t work, it could be the query-string isn’t recognized by cloudflare. Maybe try googling for any tips on excluding/matching cloudflare query string in URL, or similar. There’s gotta be something out there on the topic.

    Plugin Author Jeff Starr

    (@specialk)

    9 years, 2 months ago

    Gonna go ahead and close this thread to keep things organized. Feel free to post again if needed, I’m glad to help however possible. Thank you.

    Thread Starter Anonymous User 14978628

    (@anonymized-14978628)

    9 years, 2 months ago

    Hey Jeff, haven’t forgotten about this, just got sidetracked with a few things. Will test cloudflare again with the two exclusion urls you mentioned and then report back.

    Plugin Author Jeff Starr

    (@specialk)

    9 years, 2 months ago

    Cool thanks, much appreciated.

    Thread Starter Anonymous User 14978628

    (@anonymized-14978628)

    9 years, 2 months ago

    ok, i have tried both exclusions but problem still remains with cloudflare. I get blocked when i visit the blackhole url, but can then still visit other pages on the site. So unfortunately not working.

Viewing 15 replies - 1 through 15 (of 22 total)
1 2

The topic ‘Black URL’ is closed to new replies.