Google stresses the importance of using actual user data to assess Core Web Vitals instead of relying only on lab data from tools like PageSpeed Insights (PSI) and Lighthouse.
This reminder comes as the company prepares to update the throttling settings in PSI. These updates are expected to increase the performance scores of websites in Lighthouse.
Field Data vs. Lab Data
Core Web Vitals measure a website’s performance in terms of loading speed, interactivity, and visual stability from the user’s perspective.
Field data shows users’ actual experiences, while lab data comes from tests done in controlled environments using tools like Lighthouse.
Barry Pollard, a Web Performance Developer Advocate at Google, recently emphasized focusing on field data.
In a LinkedIn post, he stated:
“You should concentrate on your field Core Web Vitals (the top part of PageSpeed Insights), and only use the lab Lighthouse Score as a very rough guide of whether Lighthouse has recommendations to improve performance or not…
The Lighthouse Score is best for comparing two tests made on the same Lighthouse (e.g. to test and compare fixes).
Performance IS—and hence LH Scores also ARE—highly variable. LH is particularly affected by where it is run from (PSI, DevTools, CI…), but also on the lots of other factors.
Lighthouse is a GREAT tool but it also can only test some things, under certain conditions.
So while it’s great to see people interested in improving webperf, make sure you’re doing just that (improve performance) and not just improving the score”
Upcoming Changes To PageSpeed Insights
Pollard discussed user concerns about PageSpeed Insights’s slow servers, which can cause Lighthouse tests to take longer than expected.
To fix this, Google is changing the throttling settings in PageSpeed Insights, which should lead to better performance scores when the update is released in the coming weeks.
These changes will affect both the web interface and the API but will not impact other versions of Lighthouse.
However, Pollard reminds users that “a score of 100 doesn’t mean perfect; it just means Lighthouse can’t help anymore.”
Goodhart’s Law & Web Performance
Pollard referenced Goodhart’s Law, which says that when a measure becomes a goal, it stops being a good measure.
In the web performance context, focusing only on improving Lighthouse scores may not improve actual user experience.
Lighthouse is a helpful tool, but it can only assess certain aspects of performance in specific situations.
Alon Kochba, Web Performance and Software Engineer at Wix, added context to the update, stating:
“Lighthouse scores may not be the most important – but this is a big deal for Lighthouse scores in PageSpeed Insights.
4x -> 1.2x CPU throttling for Mobile device simulation, which was way off for quite a while.”
Lighthouse scores may not be the most important – but this is a big deal for Lighthouse scores in PageSpeed Insights.
4x -> 1.2x CPU throttling for Mobile device simulation, which was way off for quite a while. https://t.co/ZLrwsDQGmO
— Alon Kochba (@alonkochba) December 6, 2024
Key Takeaway: Prioritize User Experience
As the update rolls out, website owners and developers should focus on user experience using field data for Core Web Vitals.
While Lighthouse scores can help find areas for improvement, they shouldn’t be the only goal.
Google encourages creating websites that load quickly, respond well, and are visually stable.
Featured Image: GoodStudio/Shutterstock