Another Crawler problem with Sitemap

Solution: Secure that curl can be used by your WordPress instance

My Crawler stopped working at some time. I havn’t noticed it, but during a migration I validated everything, and spotted that it didn’t work anymore.

It’s throwing the classic “No valid sitemap parsed for crawler.”

However, the given sitemap URL is working just fine, as it has been for a few years. I have tested it in my browser, in a php instance (see below) and with curl on the server itself.

I’m using the the Google Sitemap Generator plugin, which normally works (and is listed as a supported sitemap): https://ww.wp.xz.cn/plugins/google-sitemap-generator/

The “test script” provided in other topics, works just fine. It’s fetching the sitemap (domain removed), and var_dumping it as expected.

<?php
require( './wp-load.php' );

$response = wp_remote_get( 'domain.com/sitemap.xml' );

echo '<pre>';
var_dump($response);
echo '</pre>';

I’ve checked the last 10 tickets related to the same error message. but I’ve not found any interesting. If I turn on “Debug”, I don’t get anything interesting related to the “crawler” process.

Pressing “Refresh Crawler Map” in “Map” also throws: “No valid sitemap parsed for crawler.”

Info:
CentOS + Ubuntu server tested + CyberPanel
Page is behind Cloudflare DNS + Cache

• This topic was modified 2 years, 12 months ago by exetico. Reason: Added a few details
• This topic was modified 2 years, 12 months ago by exetico.