When the permalink structure ends with a slash, a slash is appended to the URL of robots.txt.

[Toro_Unit]

When the permalink structure ends with a slash,

The URL is http://example.org/robots.txt/

The trailing slash should be removed.

[Toro_Unit]

added unit test

[davidbaumwald]

With 5.3 RC1 releasing today, this ticket is being moved to Future Release. If any committer feels this can be worked in to 5.3 before the deadline or can assume ownership in the 5.4 cycle, feel free to move it back up.

[joostdevalk]

We should really fix this. @SergeyBiryukov can you get this in?

[whyisjake]

In 48153:

Canonical: Ensure that when permalinks are set to a trailing slash, that robots.txt doesn't.

Fixes #48025.

Props Toro_Unit, joostdevalk, SergeyBiryukov.

[whyisjake]

In 48155:

Canonical: Add tests for trailing slashes for robots.txt.

See #48025.

Props Toro_Unit, joostdevalk, SergeyBiryukov.

[SergeyBiryukov]

In 48166:

Canonical: Move trailing slash handling for robots.txt and sitemaps to a single condition.

Give the unit test a more descriptive name.

Follow-up to [48153], [48155].

See #48025.