Exact where we can, sketched where we must: how Reliable computes percentiles

Almost every performance dashboard ends up showing a percentile. It is also where dashboards most often go quietly wrong. Percentiles get averaged together, missing data gets drawn as zero, and time buckets drift out of line with each other. The chart still looks fine, but the number on it describes nothing real.

Reliable computes percentiles in two completely different ways. Web vitals are computed exactly, from every raw sample. WebSocket timings are computed from sketches with a bounded error. This post explains why, walks through the math of the exact path, and covers the two rules every trend chart in Reliable follows.

#Why p75

The headline number for each web vital (LCP, INP, CLS, FCP and TTFB) is the 75th percentile, p75: the value that three out of four page loads beat.

• p50 hides the slow quarter. A median can look great while one visitor in four has a miserable experience.
• p99 is dominated by the worst devices and networks. It moves a lot from day to day for reasons you cannot fix, so it is a poor thing to set targets against.
• p75 is what Google uses. Core Web Vitals are assessed at the 75th percentile of page loads, so p75 lines up with how search ranking and Chrome's own tooling will judge the same site.

#Collecting the samples

We did not write our own web vitals code. The Reliable SDK uses Google's web-vitals library (its attribution build), maintained by the Chrome team. The definitions are subtler than they look. Largest Contentful Paint has to track changing candidates. Cumulative Layout Shift groups shifts into session windows. Interaction to Next Paint picks one interaction out of many. Reimplementing those would only have made our numbers disagree with everyone else's.

For each page load, the library reports one final value per metric along with its rating (good, needs improvement or poor). The SDK sends those as they are, and the backend stores each one as a row in vitals_events: project, metric, value, rating, page path, device type, connection type and time.

#Store the raw rows, compute on read

The obvious alternative is to pre-aggregate: keep one p75 per project, per metric, per day. It is smaller and faster to read. We decided against it, and the reason is a property of percentiles:

You cannot build a percentile of a combination out of the percentiles of its parts.

A stored daily p75 cannot give you a weekly p75. It cannot give you the p75 of one page, of mobile users only, or of 3G connections, unless you decided to store exactly that slice ahead of time. Raw rows answer every one of those questions exactly, including ones nobody has asked yet.

Raw rows are affordable here because web vitals are low-volume. A page load produces about five numbers, and an index on (frontend_project_id, metric, occurred_at) means every query only reads one project's rows inside its time window. Compare a WebSocket, which can carry 100 messages a second for an hour. There, storing every value is not an option, and that is where the second method comes in.

#The math of PERCENTILE_CONT

The headline p75 is one query over the last 24 hours:

SELECT metric,
       PERCENTILE_CONT(0.75) WITHIN GROUP (ORDER BY value) AS p75,
       COUNT(*) FILTER (WHERE rating = 'good')              AS good,
       COUNT(*) FILTER (WHERE rating = 'needs_improvement') AS needs_improvement,
       COUNT(*) FILTER (WHERE rating = 'poor')              AS poor
  FROM vitals_events
 WHERE frontend_project_id = $1
   AND occurred_at >= NOW() - INTERVAL '24 hours'
 GROUP BY metric;

Postgres offers two percentile functions. PERCENTILE_DISC returns an actual sample. PERCENTILE_CONT interpolates between the two samples around the target position:

sort the n values: x1 <= x2 <= ... <= xn
position h = 1 + q * (n - 1)                     // q = 0.75 for p75
p = x[floor(h)] + (h - floor(h)) * (x[floor(h) + 1] - x[floor(h)])

Take eight LCP samples, in milliseconds: 1200, 1400, 1500, 1800, 2100, 2600, 3900, 7200. Then h = 1 + 0.75 x 7 = 6.25, between the 6th value (2600) and the 7th (3900):

PERCENTILE_CONT(0.75) = 2600 + 0.25 * (3900 - 2600) = 2925 ms
PERCENTILE_DISC(0.75) = 2600 ms                  // first value with at least 75% at or below it

We use the continuous version. On small samples it moves smoothly as data arrives, instead of jumping from one sample to the next.

#Two questions that agree

The dashboard shows two things side by side: the p75 with its grade, and a bar showing what share of page loads the library rated good, needing improvement and poor. They answer different questions, but they are tied together by a simple fact:

The p75 is at or under the threshold exactly when at least 75% of page loads are at or under it.

In the example, the LCP threshold for "good" is 2500 ms. Five of the eight loads are at or under it, which is 62.5%. That is less than 75%, and consistently, the p75 of 2925 ms is over the threshold. The grade and the bar can never tell contradictory stories. (Strictly, this holds exactly for the discrete percentile, and to within the interpolation for the continuous one.)

The thresholds are the standard Core Web Vitals ones, and boundaries count as passing, the same as in the web-vitals library:

metric   good       needs improvement    poor
LCP      <= 2.5 s   <= 4.0 s             above
INP      <= 200 ms  <= 500 ms            above
CLS      <= 0.1     <= 0.25              above
FCP      <= 1.8 s   <= 3.0 s             above
TTFB     <= 0.8 s   <= 1.8 s             above

#Rule one: never average a percentile

Trend charts show one p75 per hour (for the last 24 hours) or per day (for 7 or 30 days). Each point is computed from the raw rows in that bucket:

SELECT DATE_TRUNC('day', occurred_at) AS bucket, metric,
       PERCENTILE_CONT(0.75) WITHIN GROUP (ORDER BY value) AS p75
  FROM vitals_events
 WHERE frontend_project_id = $1 AND occurred_at >= $2
   AND metric IN ('lcp', 'cls', 'inp')
 GROUP BY bucket, metric;

What we never do is derive one percentile from others. Here is why. Say Monday had 1,000 page loads with an LCP p75 of 2.0 s, and Tuesday had 50 loads with a p75 of 6.0 s. Averaging the two p75s gives 4.0 s, which is "poor". Now work out the real p75 over all 1,050 loads. You need 787.5 loads at or under the answer. Monday alone has 750 loads at or under 2.0 s, so at most 38 more are needed, and they can come from Monday. The true p75 is at most Monday's 79th percentile: a little above 2.0 s, and "good".

The average gave Tuesday's 50 loads the same weight as Monday's 1,000 and turned a good week into a poor one. Percentiles only combine through the underlying data. On the exact path, that means recomputing from raw rows. On the sketch path, it means merging distributions, never percentiles.

#Rule two: a missing value is not zero

Every trend chart is drawn on a fixed grid of buckets generated in code, not from whatever rows the query happened to return. That is 24 hourly buckets, 7 daily ones or 30 daily ones. The grid always ends with the current bucket, even though it is still filling up, so the chart shows today and not just up to yesterday. The query results are then matched onto the grid by bucket start time.

Some buckets have no data: a quiet hour, a day with no traffic. What goes in that slot depends on what the series measures:

• Counts (sessions, errors, page views) get 0. No sessions happened, and zero is the true answer.
• Percentiles and averages get null, and the chart leaves a gap. There is no p75 of nothing. Drawing it as zero would be worse than wrong: an LCP of 0 ms reads as a perfect score, so the quietest day would look like the fastest.

%% caption: Figure 1. From page loads to a trend line. Buckets come from a fixed grid; empty ones become gaps for percentiles and zeros for counts.
flowchart TD
    A["Page load in the browser"] --> B["web-vitals library: final LCP, INP, CLS, FCP, TTFB"]
    B --> C["One row per metric in vitals_events"]
    C --> D["PERCENTILE_CONT(0.75) per bucket, from raw rows"]
    G["Fixed grid: 24 hours, 7 days or 30 days, ending now"] --> E["Match results onto the grid"]
    D --> E
    E --> F{"Bucket has data?"}
    F -->|"yes"| H["Plot the p75"]
    F -->|"no, percentile"| I["null: gap in the line"]
    F -->|"no, count"| J["0: nothing happened"]

Each bucket also carries its exact start time, as a timestamp. For the 24-hour view, the browser uses that to label hours in the viewer's own time zone instead of the server's.

#When exact is too expensive: sketches

Everything above depends on being able to keep every sample. For WebSocket timings we cannot. A single long-lived connection can produce hundreds of thousands of values, and the messages themselves are too private to store. So the Reliable SDK summarises each connection into DDSketches: histograms whose buckets grow geometrically, giving every percentile a guaranteed relative error of at most 5%. Two sketches merge exactly by adding their bucket counts, which is what makes per-project baselines possible. The full story, with the math, is in the WebSocket deep dive.

The choice between the two methods comes down to one question:

%% caption: Figure 2. How we choose. Exact is always preferred; a sketch is the fallback when raw data cannot be kept.
flowchart TD
    Q["A new metric needs percentiles"] --> V{"Can we afford to keep every raw value?"}
    V -->|"yes, low volume and allowed to store"| X["Store raw rows, PERCENTILE_CONT on read"]
    V -->|"no, too much data or too private"| S["Mergeable sketch (DDSketch), bounded relative error"]
    X --> R1["Exact, any slice, any window, decided later"]
    S --> R2["Within 5%, merges exactly, fixed at collection time"]

                        web vitals                 WebSocket timings
volume                  ~5 values per page load    up to ~100 values per second per socket
stored as               raw rows in Postgres       DDSketch per connection, merged per project
percentile error        none                       at most 5% relative
new slices later        yes, any column            only what the sketch was keyed by
combining periods       recompute from rows        add bucket counts

#How we decided

• Exact by default. A percentile you can recompute from raw data is one you can always check. We only reach for an approximation when storage or privacy forces it.
• Use the reference implementation for definitions. Our web vitals match what Chrome and Google's tools report, because they come from the same library.
• Decide what "no data" means for each series up front. Zero for counts, null for everything else. It is one line in each query path, and it prevents a whole class of charts that lie.
• Grids in code, not in SQL results. Generating the buckets first and filling them in keeps every series aligned, keeps the current period visible, and makes gaps explicit.

#What to take away

• If you can afford the raw data, keep it and compute percentiles on read. It is the only way to slice later without regret.
• Never average percentiles, not across days, pages or servers. Recompute from the data or merge distributions.
• p75 at or under a threshold means at least 75% of samples are at or under it. Use that to keep grades and distributions consistent.
• Missing is not zero. Draw a gap.