Make WordPress Core

Opened 2 months ago

Closed 2 months ago

#66083 closed defect (bug) (duplicate)

wp-sitemap.xml returns HTTP 404 on sites with no published posts

Reported by: fabwebmaster Owned by:
Priority: high Milestone:
Component: Sitemaps Version:
Severity: major Keywords:
Cc: Focuses:

Description (last modified by sabernhardt)

On any WordPress site without published posts of the built-in 'post' type,
requests to /wp-sitemap.xml (and all sub-sitemaps) return HTTP 404 while
still serving a complete, valid sitemap in the response body.

Search engines evaluate the status code, so the sitemap is effectively
ignored. Google Search Console reports "Sitemap could not be read — General
HTTP error, 404" for affected sites.

This affects a common configuration: business and portfolio sites that use
pages and custom post types but never publish blog posts.

Steps to reproduce

  1. Fresh WordPress install, pretty permalinks enabled
  2. Trash or unpublish all posts of type 'post'
  3. Request /wp-sitemap.xml

Expected: HTTP 200
Actual: HTTP 404, body contains valid <sitemapindex>

Cause

On a sitemap request, WP_Query still runs the default main query:

  SELECT ... FROM wp_posts
  WHERE post_type = 'post' AND post_status IN ('publish','private')
  ORDER BY post_date DESC LIMIT 0, 10

With no posts, $wp_query->posts is empty. WP::handle_404() then sets 404,
because sitemap requests are not exempted the way robots.txt and favicon
requests are (via is_robots() / is_favicon()).

WP_Sitemaps::render_sitemaps() runs on template_redirect, i.e. after
handle_404(), which is why the body is correct while the status code is not.

On sites that do have published posts, the query returns results and
handle_404() takes the early return — the sitemap works by side effect
rather than by design.

Debug output

Filter on pre_handle_404, site WITH posts (returns 200):

  is_home: false, is_paged: false, post_count: 6

Filter on pre_handle_404, site WITHOUT posts (returns 404):

  is_home: false, is_paged: false, post_count: 0

Both sites:
matched_rule ^wp-sitemap\.xml$, matched_query sitemap=index,
query_vars {"sitemap":"index"}, wp_sitemaps_enabled true, no callbacks
registered on pre_handle_404.

Verified across 20 installations

  8 sites with published posts     → all HTTP 200
  8 sites without published posts  → all HTTP 404
  3 sites appeared to be exceptions but do have posts (excluded from the
    sitemap by a wp_sitemaps_post_types filter, still present in the database)

Different hosts, different server stacks (Apache and nginx), different plugin
sets, PHP 8.3 and 8.4, WordPress 7.1. The only variable that correlates is
whether the site has published posts.

Ruled out: caching plugins, security plugins (Wordfence), consent plugins,
SEO plugins (none installed), multilingual plugins, theme code, rewrite rule
regeneration, PHP version, PHP handler.

Suggested fix

Exempt sitemap requests in WP::handle_404(), consistent with the existing
is_robots() and is_favicon() exemptions — or have WP_Sitemaps register a
pre_handle_404 callback when a sitemap query var is present.

Workaround

add_filter( 'pre_handle_404', function ( $preempt, $query ) {
    if ( '' !== $query->get( 'sitemap' ) || '' !== $query->get( 'sitemap-stylesheet' ) ) {
        status_header( 200 );
        return true;
    }
    return $preempt;
}, 10, 2 );

Change History (2)

#1 @sabernhardt
2 months ago

  • Description modified (diff)
  • Focuses accessibility removed

Related/duplicate: #61293

#2 @westonruter
2 months ago

  • Milestone Awaiting Review
  • Resolution → duplicate
  • Status new → closed

This is actually a duplicate of #65945 and is slated for 7.1.1.

@fabwebmaster If not, please let us know. Otherwise, I'm closing this.

Please test the ​7.1.1-RC1 which was just released.

Note: See TracTickets for help on using tickets.