Problem with my Robots.txt

  • zodiamc

    (@zodiamc)


    8 years, 11 months ago

    I changed 3 of my websites to SSL. Everything worked well, only one site had a problem with a plugin which edited the virtual robots.txt file.

    My Problem now: The robots.txt file is different on http and https and it seems like frozen, cant change anything about that. I tried uploading throught tfp other robots.txt files, used yoast and other plugins. nothing worked, only the virtual file which was generated by the plugin robots.txt rewrite can be seen.

    After i deleted the plugin, nothing changed, and after i installed the plugin, it had again my “customization” on it, which means, that is not fully deleted after i deleted it.

    Can anybody help?

Viewing 7 replies - 1 through 7 (of 7 total)
  • Konstantinos Xenos

    (@xkon)

    8 years, 11 months ago

    Hello,

    Have you tried editing your .htaccess file? Usually theres a RewriteRule in there that tells the robots which .txt to read. Maybe by installing different plugins the .htaccess file got messed up and you have scraps in there from previous installations that cannot be overwritten by newer plugins or more than 1 entries for the same .txt so they overlap eachother somehow.

    I would do this:
    1) remove any plugins that can affect robots.txt
    2) backup your .htaccess file just in case.
    2) view the .htaccess file to rule out any extra robots.txt conditions from in there first.
    3) re-enable your SEO plugin and recheck if the robots.txt is properly read this time.

    Hopefully this will give you some sort of direction.

    Best regards,
    Konstantinos

    Thread Starter zodiamc

    (@zodiamc)

    8 years, 11 months ago

    Thanks i will try that! If not i`m getting professional help from fiverr. My mainkeyword dropped from #4 to #19 on this site. i guess this is one of those “dublicate content” issues…

    Konstantinos Xenos

    (@xkon)

    8 years, 11 months ago

    No problem. Inform us of your findings 🙂 .

    Best regards,
    Konstantinos

    Thread Starter zodiamc

    (@zodiamc)

    8 years, 11 months ago

    Thanks for you help Konstantinos, but i guess my technical knowledge is not enough.

    Deactivated ALL Plugins, i did a full reset on the htaccess file, deleted the “real” robots.txt – but nothing.

    Ah well, i`ll just pay 10$ bucks to get this issue fixed. typcial “beginner” mistake, instead of seeking professional help i tried to fix it myself – i guess i lost some hundreds $ from lost sales <.<

    Konstantinos Xenos

    (@xkon)

    8 years, 11 months ago

    I’m sorry to hear that @zodiamc :/ and for not finding an easy solution for you.

    Feel free to ask if you need anything else.

    Best regards,
    Konstantinos

    Thread Starter zodiamc

    (@zodiamc)

    8 years, 11 months ago

    Hey @xkon,

    just wanted to update you and the community if anyone has the same issue.

    if you cant update your robots.txt file or do anything with it whatsoever, please check your CDN! i would never ever had thought about that, but cloudflare was restricting access rights on my files so i could change anything i wanted but there were no changes made to the robots.txt file.

    after consulting to a professional, he found out that cloudflare was the problem. after disabling it, i could change my virtual robots.txt again.

    Konstantinos Xenos

    (@xkon)

    8 years, 11 months ago

    Oh bummer, you never said anything about using a CDN, but yeah that wouldn’t be a thought even like you said.

    Thanks for informing us though and glad that everything worked out fine.

    Don’t forget to mark the topic as resolved to keep the forums nice and tidy!

    Best regards,
    Konstantinos

Viewing 7 replies - 1 through 7 (of 7 total)

The topic ‘Problem with my Robots.txt’ is closed to new replies.