Differences in server response time
Variation in content, for example due to A/B tests or advertisements
Differences across test devices and test locations
Inaccuracies during data collection
Google has written a detailed guide on Lighthouse variability. You can run tests several times and look at the average to get a more consistent assessment.
Data Accuracy: Observed Vs Simulated Data
One common reason for discrepancies between page speed testing tools is the way the data is collected. In a lab test the network is throttled to a fixed speed, typically to match a slower mobile data connection. The way this throttling is achieved can impact your measurements.
PageSpeed Insights uses an approach called simulated throttling. Measurements are collected on a fast network connection. After that, a simulation of a slow 4G connection is applied to estimate how the page might have loaded on a mobile device.
You can install the Site Speed Chrome extension to view the original observed metrics when running a test on PageSpeed Insights.
Screenshot of Lighthouse reported and observed metrics, September 2024
Simulated data can sometimes be unreliable, as the Lighthouse simulation doesn’t handle all real life edge cases that can happen when opening a website.
For example, in this test we can see that the Largest Contentful Paint metric is reported as one second worse than the values observed when opening the page in Chrome.
However, the original values for the First Continental Paint and for the Largest Contentful Paint metrics were identical. This suggests that the simulated metrics could potentially not match what real users experience.
You can check the settings section of the Lighthouse report to see if the metrics were measured as reported or if a simulation has been applied.
Screenshot of Lighthouse settings, September 2024
If you want to get reliable page speed data, the free DebugBear page speed test is built to provide the most accurate insight. Collecting real measurements takes a bit longer than running a simulation, but it will also help you make the best decisions when optimizing your website speed.
Why Des The Real User Data Not Match The Lighthouse Test Rresults?
When testing your website on PageSpeed Insights you’ll often find that the real user metrics are much better than those reported by the synthetic Lighthouse test. Why is that?
That’s because the Lighthouse test uses a very slow network connection. The CrUX Core Web Vitals data looks at the slowest 25% of user experiences on your website, but typically, even those visits come from a device that has a decent network connection.
So, a bad Lighthouse performance score doesn’t necessarily mean that you’ll fail Google’s Core Web Vitals assessment. But it can indicate that some users are having a poor experience and that there’s more room for improvement.
Screenshot of real user and lab-based performance metrics in PageSpeed Insights, September 2024
5. Use The PSI API To Automate Performance Testing
Got a lot of pages on your website you want to test? You can use the PageSpeed Insights API to automatically run website tests in bulk.
The API provides more detailed performance metrics and details on each Lighthouse audit. For example, you can use the API to see the most common performance recommendations across your website.
There’s even a way to access PageSpeed Insights data directly in Google Sheets.
JSON page speed data reported by the PageSpeed Insights API, September 2024
6. Know When To Reach For A Different Tool
PageSpeed Insights is a great tool to run a quick performance for a specific URL on your website. However, as we’ve seen above, this data comes with some limitations.
If you just want to get a site-wide overview of Core Web Vitals on your website, the quickest way to find this data is using Google Search Console.
Search Console will show you exactly how many pages on your website are slow or need to be improved.
Screenshot of Core Web Vitals data in Google Search Console, September 2024
Need to dive deep into CPU performance, for example to optimize the new Interaction to Next Paint metric?
The Performance tab in Chrome’s developer tools provides a detailed analysis of all kinds of CPU processing that happens on your website.
Screenshot of a website performance profile in Chrome DevTools, September 2024
Finally, if you want to optimize how different resources are loaded on your website, the DebugBear website speed test can be invaluable.
This test can provide a detailed report on what resources are loaded by your website, when they load, and how they impact rendering.
Screenshot of a website request waterfall in DebugBear, September 2024
How To Always Stay Ahead Of Your Website Speed
PageSpeed Insights and other performance tests are a great starting point for optimizing your website. However, without continuous monitoring, you risk reintroducing problems without noticing.
DebugBear is a monitoring platform for Core Web Vitals that lets you continuously test both your own website and those of your competitors.
Screenshot of the DebugBear performance dashboard, September 2024
In addition to scheduled lab testing, DebugBear also keeps track of Google CrUX data and collects real user analytics directly on your website.
The real user data provides a wide range of insight to not just help you keep track of performance but actively improve it:
See what LCP subpart is causing the biggest delay for your visitors
Find specific interactions and scripts that cause a poor Interaction to Next Paint score
Identify specific countries or devices where performance is worse than usual
Screenshot of real user monitoring data in DebugBear, September 2024
Deliver A Great User Experience
PageSpeed Insights is a helpful tool for any website owner, not just telling you how fast your website is in the real world, but also giving you concrete advice on how to optimize it.
However, if you’d like to go beyond the data PSI provides and test your website continuously, you can sign up for a free 14-day DebugBear trial.
This article has been sponsored by DebugBear, and the views presented herein represent the sponsor’s perspective.
Ready to start optimizing your website? Sign up for DebugBear and get the data you need to deliver great user experiences.
Image Credits
Featured Image: Image by DebugBear. Used with permission.