Make WordPress Core

Opened 6 years ago

Last modified 2 months ago

#49661 reviewing defect (bug)

mails, Howdy, wp_mail() and filters spreading

Reported by: arena's profile arena Owned by: sirlouen's profile SirLouen
Milestone: Future Release Priority: normal
Severity: normal Version: 1.5.1.2
Component: Mail Keywords: good-first-bug has-test-info has-patch has-unit-tests
Focuses: Cc:

Description

Hello,

The different mails that are build to be sent with function wp_mail() have different ways of doing it. Some are using str_replace(), others are using sprintf(), have a different vocabulary (SITENAME, SITE_NAME, ###SITENAME### ...)

Last but not least, some have different filters : from No filters to subject and/or content and/or mail headers and here again with different vocabularies ( filter_prefix_content or filter_prefix_message) ...

So when you want to change some mails that belong to a group of mails (admin, network, privacy ..), you have to dig in the code and eventually ask for specific changes for this or that mail.

With privacy, and locally for legal purpose, related mails must be sent by a new profile in the organisation, by some sort of a new WP "role" that is still to be created : the cop ... for Chief Of Privacy . In Europe, we call him a dpo.

The following patch proposes to have all mails equals (except some specific ones related to wp updates).

Subject and Mail body are templates.
The {{Mustache}} standard is adopted.
All mails have an id and belong to a group.

This is an encapsulation of wp_mail() with some conventions to get the best results.

With the patch, i attach here

  • a zip file that is a set of .eml files (with some logging in it for debugging purpose)

As a proof of concept, i also attach the three plugins i developped

  • logphpmailer.php : to save eml files
  • wp_mailer_filter.php : sample plugin filtering mail using the new filters of wp_mailer class (here for privacy mails)
  • wp_mailer_logger.php : appends some logging to mail content using the new filters of wp_mailer class

the patch also fixes a bug i detected on the Reply-To mail header (double quotes doubled, added by wp AND phpmailer)

I know this is a big one. Due to CoVid, i am stuck at home in front of my keyboard.

Regards

Attachments (7)

#wp_mailer.patch (143.8 KB) - added by arena 6 years ago.
the patch
wp_mailer_mails.zip (24.8 KB) - added by arena 6 years ago.
the mails tested with some logging
logphpmailer.php (796 bytes) - added by arena 6 years ago.
logphpmailer.php
wp_mailer_filter.php (1.2 KB) - added by arena 6 years ago.
wp_mailer_filter
wp_mailer_logger.php (3.5 KB) - added by arena 6 years ago.
wp_mailer_logger
#wp_mailer-v2.patch (143.8 KB) - added by arena 6 years ago.
v2 of patch
49661.patch (1.4 KB) - added by SirLouen 3 months ago.
Adding Unit Tests for PR 9543

Download all attachments as: .zip

Change History (36)

@arena
6 years ago

the patch

@arena
6 years ago

the mails tested with some logging

@arena
6 years ago

logphpmailer.php

@arena
6 years ago

wp_mailer_filter

@arena
6 years ago

wp_mailer_logger

#1 @arena
6 years ago

Related #46424 1)

#2 @arena
6 years ago

v2 of patch to fix text missing

--- wp-includes/user.php

< +As {{USERNAME}}, this notice confirms that your email address on {{SITENAME}} to {{NEW_EMAIL}}.
---

+As {{USERNAME}}, this notice confirms that your email address was changed on {{SITENAME}} to {{NEW_EMAIL}}.

@arena
6 years ago

v2 of patch

#3 @SirLouen
5 months ago

Generally, it's very complex to get this kind of features getting through (specially nowadays).

Although I like most of the proposal, but with some caveats:

  1. I'm not 100% confident about Mustache in 2025. PHP support was dropped a time ago, and for example, other projects like CLI that adopted Mustache had to fork the PHP project to keep maintaining it in newer versions. But all of a sudden the maintainer returned like 1 month ago and merged a bunch of pending PRs in the repo… Still, personally I would not trust PHP-Mustache and would rather opt for Twig, which is better maintained, but unlikely because I doubt there will be much love in WP for such, or simply use a custom templating or microtemplating).
  1. Another problem I'm seeing in this report is that you want to approach all in one place. Ideally multiple reports should be done, one per idea, one at a time. I support most of the ideas you are proposing, although some others are better suited to be left for extenders.

To avoid digging further on this without results (like has already happened) and spending a ton of time on something that maybe will not even be ever executed, I think we could start from one point and continue from there, firstly selecting a potential templating engine to normalize all emails structure feels a great feature for me.

Also, I've been thinking a lot lately on the idea of encapsulating WP_Mail into a class, specially for testing purposes. Nowadays, as a pluggable is a complete mess. But here we have two topics in the table, as I say, one a time is the right way to proceed.

I would like to review your patch for this specific task, and probably open a new report just to approach this. If we can manage to make this happen, we could move into further improvement as the ones suggested afterwards, one at a time. Anyway, I want to hear about other WP members opinions on templating. Currently, only JavaScript microtemplating is being used for many of the admin interfaces.

#4 @SirLouen
4 months ago

Related Tickets
#53829 - Filtering
#46229 #51717 - Templating (+HTML Mails)
#29513 - Encapsulation

#5 @SirLouen
4 months ago

  • Keywords good-first-bug has-test-info needs-patch added; has-patch removed
  • Milestone changed from Awaiting Review to Future Release
  • Type changed from feature request to defect (bug)
  • Version changed from 5.4 to 1.5.1.2

Bug Reproduction Report

Description

✅ This report validates that the issue can be reproduced.

Environment

  • WordPress: 6.9-alpha-60093-src
  • PHP: 8.2.29
  • Server: nginx/1.29.1
  • Database: mysqli (Server: 8.4.6 / Client: mysqlnd 8.2.29)
  • Browser: Chrome 139.0.0.0
  • OS: Windows 10/11
  • Theme: Twenty Twenty-Five 1.3
  • MU Plugins: None activated
  • Plugins:
    • Micro Email Testing 1.0.0
    • Test Reports 1.2.0

Reproduction Instructions

  1. Create a comment to a post
  2. Add a reply to that comment
  3. Add a second reply to the comment with the same email (there is a bug #33717 that wont sent notification emails in the comment goes to the moderation queue)
  4. Check the received email, more specifically the Reply-To
  5. 🐞 It's being wrongly formatted like:
Reply-To: "\"tester@example.org\"" <tester@example.org>

The name is wrongly being picked + double quotes.

Actual Results

  1. ✅ Error condition occurs (reproduced).

Additional Notes

After reviewing the patch, I can say that despite this was very ambitious, it would bring massive backward compatibilities issues. I think there is a good amount of content to make feature or canonical plugin I've been thinking around. I was considering on closing as maybelater for these reasons, but then I noticed that between the code, there is a bug that I've been able to reproduce, so we should be focusing on this bug for this ticket.

The problem is happening among these lines:
https://github.com/WordPress/wordpress-develop/blob/dd2d7efd9dc11a362056be6cd7fc75ccd6b0cf22/src/wp-includes/pluggable.php#L1751-L1761

Here, there is not only the double quotes formatting issue, but using the email instead of the name, that we also could have (Both email and name fields are mandatory). First introduced in [3035], #1593

Supplemental Artifacts

Raw email received:

Return-Path: <test@example.com>
Received: from localhost (wordpress-develop-php-1.wordpress-develop_wpdevnet. [172.20.0.3])
        by 93e77d70be1e (Mailpit) with SMTP
        for <test@example.com>; Sun, 17 Aug 2025 16:23:45 +0000 (UTC)
Date: Sun, 17 Aug 2025 16:23:45 +0000
To: test@example.com
From: "Mr. Tester" <test@example.com>
Reply-To: "\"tester@example.org\"" <tester@example.org>
Subject: [WordPress Develop] Comment: "Hello world!"
Message-ID: <IzHNN1HuqMA90LeQ9cRI3IT38sLlzkOuncMuCLX4k@localhost>
X-Mailer: PHPMailer 6.10.0 (https://github.com/PHPMailer/PHPMailer)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8

New comment on your post "Hello world!"
Author: Mr. Tester (IP address: 172.20.0.1, 172.20.0.1)
Email: tester@example.org
URL: 
In reply to: http://localhost:8889/wp-admin/comment.php?action=editcomment&c=3#wpbody-content
Comment: 
This is a test example

You can see all comments on this post here:
http://localhost:8889/2025/08/17/hello-world/#comments

Permalink: http://localhost:8889/2025/08/17/hello-world/#comment-4
Trash it: http://localhost:8889/wp-admin/comment.php?action=trash&c=4#wpbody-content
Spam it: http://localhost:8889/wp-admin/comment.php?action=spam&c=4#wpbody-content
Last edited 2 months ago by SirLouen (previous) (diff)

#6 @jdeep
4 months ago

I would like to pick up this bug ticket.

#7 @SirLouen
4 months ago

@jdeep go for it :)

This ticket was mentioned in PR #9543 on WordPress/wordpress-develop by @jdeep.


4 months ago
#8

  • Keywords has-patch added; needs-patch removed
The Reply-To header in comment reply notification emails was being 
manually formatted with quotes, similar to the From header. 
However, PHPMailer treats these headers differently:

- From headers are set via `setFrom()`, which applies proper quoting
  and encoding internally.
- Reply-To headers in core were added as custom headers via 
 `addCustomHeader()`, which passes the value through `encodeHeader()`
  and escapes quotes a second time.

This mismatch caused author names with quotes to be encoded as:
Reply-To: \"Author\" <author@example.com>

This commit fixes this issue.

See Trac #49661

Trac ticket: #49661

#9 follow-up: @jdeep
4 months ago

@SirLouen I believe a simple one-liner fix should be okay for this. Since name field is mandatory, we can use that in the Reply-To header.

#10 in reply to: ↑ 9 @SirLouen
3 months ago

  • Owner set to SirLouen
  • Status changed from new to reviewing

Replying to jdeep:

@SirLouen I believe a simple one-liner fix should be okay for this. Since name field is mandatory, we can use that in the Reply-To header.

Hey @jdeep apologies for the late review.

It looks good to me. I'm sending you here some Unit Tests to add to the PR.

Also as an extra advice: Generally, as a rule of thumb, try to add unit tests if you feel you can do them, but if you can't, don't worry, add the tag needs-unit-test and someone will take care of them. (And always remember that Unit Tests should be done without using AI, if you can't do them without AI better leave them undone, it's very clear when AI is being used and that is bad reputation, because 99% of the times AI PHPUnit unit tests are utterly bad quality).

Last edited 3 months ago by SirLouen (previous) (diff)

@SirLouen
3 months ago

Adding Unit Tests for PR 9543

#11 @SirLouen
3 months ago

  • Keywords needs-testing has-unit-tests added

This ticket was mentioned in Slack in #core-test by sirlouen. View the logs.


3 months ago

This ticket was mentioned in Slack in #core-test by nikunj8866. View the logs.


3 months ago

This ticket was mentioned in Slack in #core-test by sirlouen. View the logs.


3 months ago

#15 @sajjad67
3 months ago

Test Report

Description

This report validates whether the indicated patch works as expected.

Patch tested: https://github.com/WordPress/wordpress-develop/pull/9543

Environment

  • WordPress: 6.9-alpha-60093-src
  • PHP: 8.2.29
  • Server: nginx/1.29.1
  • Database: mysqli (Server: 8.4.6 / Client: mysqlnd 8.2.29)
  • Browser: Chrome 140.0.0.0
  • OS: Windows 10/11
  • Theme: Twenty Twenty-Five 1.3
  • MU Plugins: None activated
  • Plugins:
    • Email Testing 1.0.0
    • Test Reports 1.2.0

Actual Results

  1. ✅ Issue resolved with patch.

Additional Notes

  • After applying the patch the wrongly formatted name is now showing correctly.

Supplemental Artifacts

Raw email received (before applying the patch):

  Return-Path: <from@example.com>
  Received: from localhost (wordpress-develop-php-1.wordpress-develop_wpdevnet. [172.18.0.3])
          by eb84c58a3dd4 (Mailpit) with SMTP
          for <test@example.com>; Mon, 15 Sep 2025 16:17:39 +0000 (UTC)
  Date: Mon, 15 Sep 2025 16:17:39 +0000
  To: test@example.com
  From: Sajjad Hossain Sagor <from@example.com>
  Reply-To: "\"test2@gmail.com\"" <test2@gmail.com>
  Subject: [WordPress Develop] Comment: "Hello world!"
  Message-ID: <3nM4BSP7SRjFyhsvdFKFjB2ogzNxwoKosnAx5f4M@localhost>
  X-Mailer: PHPMailer 6.10.0 (https://github.com/PHPMailer/PHPMailer)
  MIME-Version: 1.0
  Content-Type: text/plain; charset=UTF-8

  New comment on your post "Hello world!"
  Author: Sajjad Hossain Sagor (IP address: 172.18.0.1, 172.18.0.1)
  Email: test2@gmail.com
  URL: 
  In reply to: http://localhost:8889/wp-admin/comment.php?action=editcomment&c=5#wpbody-content
  Comment: 
  Hi

  You can see all comments on this post here:
  http://localhost:8889/index.php/2025/09/15/hello-world/#comments

  Permalink: http://localhost:8889/index.php/2025/09/15/hello-world/#comment-6
  Trash it: http://localhost:8889/wp-admin/comment.php?action=trash&c=6#wpbody-content
  Spam it: http://localhost:8889/wp-admin/comment.php?action=spam&c=6#wpbody-content

Raw email received (after applying the patch):

  Return-Path: <from@example.com>
  Received: from localhost (wordpress-develop-php-1.wordpress-develop_wpdevnet. [172.18.0.3])
          by eb84c58a3dd4 (Mailpit) with SMTP
          for <test@example.com>; Mon, 15 Sep 2025 16:15:35 +0000 (UTC)
  Date: Mon, 15 Sep 2025 16:15:35 +0000
  To: test@example.com
  From: Sajjad Hossain Sagor <from@example.com>
  Reply-To: Sajjad Hossain Sagor <test@gmail.com>
  Subject: [WordPress Develop] Comment: "Hello world!"
  Message-ID: <9aMmh7TOyHScstfPRoS7sPFBlXu6KlTTAXdELGobhQ@localhost>
  X-Mailer: PHPMailer 6.10.0 (https://github.com/PHPMailer/PHPMailer)
  MIME-Version: 1.0
  Content-Type: text/plain; charset=UTF-8

  New comment on your post "Hello world!"
  Author: Sajjad Hossain Sagor (IP address: 172.18.0.1, 172.18.0.1)
  Email: test@gmail.com
  URL: 
  In reply to: http://localhost:8889/wp-admin/comment.php?action=editcomment&c=1#wpbody-content
  Comment: 
  Hello

  You can see all comments on this post here:
  http://localhost:8889/index.php/2025/09/15/hello-world/#comments

  Permalink: http://localhost:8889/index.php/2025/09/15/hello-world/#comment-5
  Trash it: http://localhost:8889/wp-admin/comment.php?action=trash&c=5#wpbody-content
  Spam it: http://localhost:8889/wp-admin/comment.php?action=spam&c=5#wpbody-content

This ticket was mentioned in Slack in #forums by sajjad67. View the logs.


3 months ago

#17 @jdeep
3 months ago

Apologies for the late reply. @SirLouen I have applied your Unit Test patch to the PR.

This ticket was mentioned in Slack in #core-test by sirlouen. View the logs.


3 months ago

#19 @mosescursor
3 months ago

Bug Reproduction Issue “45516”

Reproduction Report

Description

This report validates whether the issue can be reproduced. I have failed to reproduce the bug and so couldn't apply patch

Environment

  • WordPress: 6.9-alpha-60093-src
  • PHP: 8.2.29
  • Server: nginx/1.29.1
  • Database: mysqli (Server: 8.4.6 / Client: mysqlnd 8.2.29)
  • Browser: Chrome 140.0.0.0
  • OS: macOS
  • Theme: Twenty Twenty-Five 1.3
  • MU Plugins:
    • test-mail.php
  • Plugins:
    • Test Reports 1.2.0
    • WPForms Lite 1.9.7.3
    • WP Mail SMTP 4.6.0

Actual Results

  1. ❌ Error condition does not re-occur (not - reproduced).

Additional Notes

  • I set up Docker
  • Installed WordPress trunk using the wordpress-develop environment (Docker).
  • Added an MU Plugin to send test emails with MailPit
  • Installed and activated the WP Mail SMTP plugin.
  • Configured SMTP with Mailpit (mailpit:1025).
  • Sent test emails from the WP Mail SMTP tool.
  • Verified messages in Mailpit’s web UI (http://localhost:8025).

Supplemental Artifacts

Plugin Code: This is the code I used in the Plugin (test-mail.php)

<?php
add_action('init', function() {
    if (isset($_GET['sendmail'])) {
        $to = 'you@example.com';
        $subject = 'Test Email';
        $message = 'Testing reply-to header bug.';

        $headers = [
            'Reply-To: "test2@gmail.com" <test2@gmail.com>'
        ];

        wp_mail($to, $subject, $message, $headers);

        echo "Mail sent (check headers).";
        exit;
    }
});

Received Email headers for test email

From	WordPress Develop  <test@example.com>
To	<you@example.com>
Reply-To	"test2@gmail.com" <test2@gmail.com>
Subject	Test Email
Date	Fri, 19 Sep 2025, 11:17 am (620 B)
Tags	
Add tags...

Email from Blog Post comment

From	WordPress Develop  <test@example.com>
To	<test@example.com>
Subject	[WordPress Develop] Please moderate: "Hello world!"
Date	Fri, 19 Sep 2025, 11:38 am (1.2 kB)
Tags	
Add tags...
Text
Headers
Raw
Link Check
A new comment on the post "Hello world!" is waiting for your approval
http://localhost:8889/?p=1

Author: Moses Cursor (IP address: 140.82.121.6, lb-140-82-121-6-fra.github.com)
Email: testing@email.com
URL: https://email.com
Comment: 
Looks good

Approve it: http://localhost:8889/wp-admin/comment.php?action=approve&c=2#wpbody-content
Trash it: http://localhost:8889/wp-admin/comment.php?action=trash&c=2#wpbody-content
Spam it: http://localhost:8889/wp-admin/comment.php?action=spam&c=2#wpbody-content
Currently 1 comment is waiting for approval. Please visit the moderation panel:
http://localhost:8889/wp-admin/edit-comments.php?comment_status=moderated#wpbody-content

This ticket was mentioned in Slack in #core-test by sirlouen. View the logs.


3 months ago

#21 @dilip2615
3 months ago

Patch Testing Report – #49661

Patch tested: 49661.3.diff (adds/updates unit tests for wp_mail() filters)

Environment

  • WordPress: 6.9-alpha-60093-src
  • PHP: 8.2.28
  • Server: nginx/1.29.1
  • Database: mysqli (Server: 8.4.6 / Client: mysqlnd 8.2.28)
  • Browser: Chrome 140.0.0.0
  • OS: Windows 10/11
  • Theme: Twenty Twenty 2.9
  • MU Plugins: None activated
  • Plugins:
    • Test Reports 1.2.0

Steps
1) Downloaded and refreshed patch (fixed old paths from phpunit/tests → tests/phpunit).
2) Applied patch cleanly with git apply --index 49661.3.diff.
3) Ran targeted tests:

  • npm run test:php -- --group mail
  • Verified execution of new test cases for wp_mail().

4) Reviewed assertions related to wp_mail() filters.

Results

  • All mail group tests passed (OK (29 tests, 70 assertions)).
  • Patch applies cleanly on trunk after path refresh.
  • Added tests behave as expected, no regressions observed.

Conclusion

  • Patch is valid, refreshed, and confirmed working.
Last edited 3 months ago by dilip2615 (previous) (diff)

#22 follow-up: @SirLouen
3 months ago

@dilip2615 I think that you got confused like @mosescursor

The final patch is the PR #9543

Give it a second check and retest. You don't need to test the Unit Tests. Here we have to be actually testing what I explain here (the comments part and check if the Reply-To address is clean or not after the patch.

#23 in reply to: ↑ 22 @dilip2615
3 months ago

Okay @SirLouen I will follow your instructions.

This ticket was mentioned in Slack in #core-test by sirlouen. View the logs.


2 months ago

This ticket was mentioned in PR #9999 on WordPress/wordpress-develop by @SirLouen.


2 months ago
#25

After further review and testing, I have observed that the problem is not within Reply-To in the wp_notify_postauthor function but the fact that currently the parsing of anything not From (and even the From) is extremely poor. In Trac #62940 we aim to improve this significantly, by simplifying the code and unifying the parsing for all To, From, Bcc, CC and Reply-To headers.

For this reason, this PR will be blocked until #62940 is merged.

The only "improvement" that this extremely simple PR brings, is adding the commenter name to the Reply-To and the corresponding Unit Test, testing that the expected output is correct (but this will only pass, again when #62940 is merged).

#26 @SirLouen
2 months ago

  • Keywords needs-testing removed

#27 follow-up: @SirLouen
2 months ago

@dilip2615 @mosescursor no more testing here. This will be blocked by #62940. Once it is unblocked, I will ping you to proceed with the last testing before merging.

#28 in reply to: ↑ 27 @dilip2615
2 months ago

Okay I will do as you say Replying to SirLouen:

@dilip2615 @mosescursor no more testing here. This will be blocked by #62940. Once it is unblocked, I will ping you to proceed with the last testing before merging.

#29 @mosescursor
2 months ago

Okay, will be waiting

Note: See TracTickets for help on using tickets.