Make WordPress Core

Opened 7 months ago

Closed 4 months ago

Last modified 4 months ago

#58360 closed task (blessed) (fixed)

Measure additional load time performance metrics on each commit

Reported by: joemcgill's profile joemcgill Owned by: swissspidy's profile swissspidy
Milestone: 6.4 Priority: normal
Severity: normal Version:
Component: Build/Test Tools Keywords: has-patch has-unit-tests
Focuses: performance Cc:

Description

In [55459] we added a new workflow that records server timing metrics for each commit. While server timing is an important metric, it only tells part of the story. As @oandregal pointed out in his writeup about measuring performance metrics across different versions of WordPress, we often need to take a wholistic picture of how a change affects the user experience by considering additional performance metrics like LCP, TBT, and CLS, which are user-centric and can be calculated using lab data.

Metrics to consider for inclusion

  • Largest Contentful Paint (LCP)
  • Total Blocking Time (TBT)
  • Cumulative Layout Shift (CLS)

Implementation notes

Our current performance workflow executes npm run test:performance, which already uses the test-e2e framework from wp-scripts to measure server timing, so adding additional front-end measurements can be implemented similarly to how it's been implemented here and used in the Gutenberg repo.

Change History (15)

#1 @flixos90
6 months ago

I would say, given that load time performance is where WordPress struggles the most (LCP passing rate of WordPress sites is far lower than CLS, INP, or FID passing rates, see https://cwvtech.report) we should focus on additional load time metrics first.

I would suggest using LCP like you are saying, but also TTFB. While that should give us similar values like the wp-total Server-Timing metric, it is measured in a more holistic way also encompassing the request so it provides another reference point which IMO is useful to have alongside wp-total: Effectively TTFB is server response time as seen from the client, while wp-total is just the WordPress-specific response time as seen from the server.

Regarding the implementation: While Gutenberg uses the implementation you mentioned, I think it would be worth considering using https://github.com/GoogleChrome/web-vitals instead, as that is a library specifically to measure those metrics. Rather than manually implementing it and having to ensure our implementation remains up to date with Web Vitals changes, using that library would simplify things by avoiding that. This command is an example of using that library in a headless browser, and I think we could follow a similar approach in our CI workflow.

#2 @oandregal
6 months ago

Thanks for the follow-up.

In the original ticket at #57687, I mentioned tracking LCP and LCP-TTFB in addition to TTFB. The editor tracks all these three because they provide different perspectives on performance (perceived performance for the user, client-only, and server-only respectively). It'd be great to have these as a common set tracked both in the editor and core.

This ticket was mentioned in PR #4999 on WordPress/wordpress-develop by @swissspidy.


4 months ago
#3

  • Keywords has-patch has-unit-tests added; needs-patch removed

Collects TTFB, LCP, and LCP - TTFB for the two existing performance tests in core, using PerformanceObserver.

Tracking more CWV metrics quickly requires the use of a dedicated library, which can be looked into in a separate PR.

This is something to get off the ground quickly.

Trac ticket: https://core.trac.wordpress.org/ticket/58360

#4 @swissspidy
4 months ago

  • Milestone changed from Awaiting Review to 6.4

This PR is a start at collecting TTFB & LCP (and LCP - TTFB).

For collecting more stuff in the future I'd recommend using Lighthouse, as it's meant for lab tests like this. The web-vitals library is more targeted at collecting the data in the field.

#5 @flixos90
4 months ago

  • Owner set to swissspidy
  • Status changed from new to assigned
  • Summary changed from Measure additional performance metrics on each commit to Measure additional load time performance metrics on each commit

Thanks @swissspidy for the PR, looks great so far!

+1 on focusing on load time metrics now, since they have been our primary focus for a while. I'm changing the title accordingly.

It would be good to then open dedicated follow-up tickets, potentially even separate ones for layout stability metrics (CLS) and interactivity metrics (TBT, FID, INP).

@youknowriad commented on PR #4999:


4 months ago
#6

I'm happy to add these metrics whenever this is committed.

@oandregal commented on PR #4999:


4 months ago
#7

Thanks for working on this, Pascal! Eager to have the client-side metrics also reported for core.

@oandregal commented on PR #4999:


4 months ago
#8

I've noticed there is a 6ms difference between wpTotal and TTFB. I know _why_ it happens, though I wonder if wpTotal is still necessary?

@swissspidy commented on PR #4999:


4 months ago
#9

I've noticed there is a 6ms difference between wpTotal and TTFB. I know _why_ it happens, though I wonder if wpTotal is still necessary?

I think it's valuable to know which portion of TTFB can be attributed to the server and which one to the browser/network. Worst case, if we find it to be unnecessary we can we keep reporting the value but just hide it from the dashboard.

#10 @swissspidy
4 months ago

  • Resolution set to fixed
  • Status changed from assigned to closed

In 56399:

Build/Test Tools: Measure additional load time metrics in performance tests.

Three new metrics are being collected and reported as part of this change:

  • Time To First Byte (TTFB) - the time between the request for a resource and when the first byte of a response begins to arrive
  • Largest Contentful Paint (LCP) — the render time of the largest image or text block visible within the viewport
  • The difference between the two (LCP minus TTFB)

Props joemcgill, flixos90, oandregal, mukesh27, youknowriad, swissspidy.
Fixes #58360.

#11 @swissspidy
4 months ago

Created #59119 and #59120 as follow-up tickets.

@swissspidy commented on PR #4999:


4 months ago
#12

Committed in https://core.trac.wordpress.org/changeset/56399

Thanks everyone!

@youknowriad Could you handle updating the dashboard accordingly? That would be great!

@youknowriad commented on PR #4999:


4 months ago
#13

So can you confirm the "keys" of the new metrics:

  • timeToFirstByte
  • largestContentfulPaint
  • lcpMinusTtfb


Right? Do you want to show all of them in the dashboard?

@swissspidy commented on PR #4999:


4 months ago
#14

Yes, those are the keys. They should match the ones from Gutenberg.

I think it's fine to show them all in the dashboard for now. Once we add more metrics we can revisit :-)

@youknowriad commented on PR #4999:


4 months ago
#15

I've added them 👍

Note: See TracTickets for help on using tickets.