Make WordPress Core

Opened 12 months ago

Closed 9 months ago

Last modified 9 months ago

#58358 closed task (blessed) (fixed)

Measure change in performance of a single commits

Reported by: joemcgill's profile joemcgill Owned by: swissspidy's profile swissspidy
Milestone: 6.4 Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch has-unit-tests
Focuses: performance Cc:

Description

Automated performance metrics were added in [55459]. Currently, each time this automated workflow runs, it records two sets of metrics: one for the current change, and one used as a baseline for normalizing results between changes. Currently the baseline measurement is always taken against the 6.1.1 tag in WordPress (the most recent stable release when the workflow was created). In order to better understand the performance impact of a single change, it would be useful to compare directly against the previous commit hash (e.g., trunk) as part of the same workflow, using the same GitHub worker infrastructure to reduce variability between separate runs of the workflow.

Goals of this ticket

  1. Collect a set of performance metrics for trunk prior to a commit in addition to the two that are being tested now.
  2. Calculate the percentage change between trunk before and after the current change that is being tested
  3. Include the percentage change in the workflow summary logs

Change History (12)

@swissspidy commented on PR #5000:


9 months ago
#3

Just realized that this is PR number 5000 :D That's _alot_ of PRs.

@swissspidy commented on PR #5000:


9 months ago
#4

Since #4999 has been committed, could you please incorporate the metrics TTFB, LCP, and LCP - TTFB?

That should already be the case now that I merged the latest changes from trunk. You can see the results here: https://github.com/swissspidy/wordpress-develop/pull/33#issuecomment-1679084103

#5 @swissspidy
9 months ago

  • Milestone changed from Awaiting Review to Future Release
  • Owner set to swissspidy
  • Status changed from new to assigned

#6 @swissspidy
9 months ago

  • Milestone changed from Future Release to 6.4

@oandregal commented on PR #5000:


9 months ago
#7

Leave a comment on the PR with the results (https://github.com/swissspidy/wordpress-develop/pull/33#issuecomment-1679084103)

Would you be open to leave this for a follow-up, so we can merge the rest of it ASAP. This PR provides an enormous value, so the faster we merge, the better.

@swissspidy commented on PR #5000:


9 months ago
#8

Yes, definitely open to that.

@swissspidy commented on PR #5000:


9 months ago
#9

Removed the PR comment part now.

I think it looks good, but I'll wait for a final review from @desrosj

#11 @swissspidy
9 months ago

  • Resolution set to fixed
  • Status changed from assigned to closed

In 56506:

Build/Test Tools: Compare results in performance measurement workflow.

This extends the performance test workflow added in [55459] to also run tests against the target branch (if running on a pull request) or the previous commit (if running on trunk).

Those results are then compared with the ones from the current commit, and the difference is displayed as a GitHub Actions workflow summary for convenience.

Props mukesh27, flixos90, desrosj, joemcgill, swissspidy.
Fixes #58358, #58359.
See #56150.

Note: See TracTickets for help on using tickets.