Opened 18 years ago
Closed 18 years ago
#6891 closed defect (bug) (fixed)
Do not add trailing slash to the robots.txt URL
| Reported by: |
|
Owned by: | |
|---|---|---|---|
| Milestone: | 2.6 | Priority: | normal |
| Severity: | normal | Version: | 2.5.1 |
| Component: | General | Keywords: | has-patch |
| Focuses: | Cc: |
Description
The /robots.txt is redirected to /robots.txt/ if configured. This is in violation of the robot exclusion standard, which specifies
This file must be accessible via HTTP on the local URL "/robots.txt".
It is also not network nice because it generates extra traffic for the additional request after redirection.
Attachments (1)
Change History (3)
Note: See
TracTickets for help on using
tickets.
I spotted this the other day, Didnt get around to submitting a trac ticket though, Thanks for bringing it up and reminding me :)