What QA score distributions reveal — and the mean conceals

Quality · ~6 minute read

A team QM mean of 87% can hide four agents struggling at 65% and eight excelling at 92%. Most of the developmental decisions worth taking in quality depend on the shape of the score distribution, not its centre — and the team mean is the one number guaranteed to conceal the shape.

The patterns worth recognising

QM data has characteristic shapes. The bimodal team — two clusters, common after a new product or process where some agents have adapted and others haven’t; the mean sits between the clusters and represents neither. The skewed distribution — a long tail of exceptional or failing calls pulling the mean away from where most of the work actually sits.

Then the two that point at the raters rather than the agents: the compressed distribution, all scores clustering tightly around one value — often rater conservatism (‘I always give threes’) masquerading as consistency — and the polarised distribution, scores spread across the full range through inconsistent scoring rather than genuinely varied performance. Both are calibration signals, not performance signals.

Summary statistics that match the shape

For most contact-centre QM data, median, IQR and percentiles beat mean and standard deviation — they are robust to the skew that quality scores almost always carry. Reserve the mean for distributions that are genuinely symmetric, and state the median alongside it for transparency. For ordinal scales, the mode tells you what the typical rating actually is.

The most actionable numbers are usually the tails. The bottom decile is where coaching investment produces disproportionate change; the top decile is where the behaviours worth spreading live. A coaching programme aimed at the bottom decile, measured by whether the tail moves, is more sensitive and more honest than one measured by whether the mean shifts — the mean barely registers four agents improving in a team of fifteen.

Plot before you summarise

The discipline is to look at the picture before the number. A monthly histogram of overall scores per team makes bimodality jump out and drift visible. Side-by-side box plots compare teams, months or tenure bands at a glance. Per-item distributions surface the items some agents have understood and others haven’t. An agent-by-item heatmap separates agents weak across the board from agents weak on one specific thing — two very different coaching conversations.

Apply the misleading-mean test to any QM report: is the distribution roughly symmetric? Is there bimodality? Are outliers driving the mean? Is the spread itself the story? If any answer is yes, show the distribution, not just the number. ‘Team mean 87%’ and ‘bimodal — four agents around 65% needing focused coaching, eight at 92% to recognise and reinforce’ are the same data; only one of them supports a decision.

The drift nobody tracks

Single-period analysis misses the slowest and most important pattern: a distribution whose centre is moving or whose spread is widening over months. A team can hold a stable mean while its spread doubles — the strong getting stronger, the weak drifting away — and a mean-only report shows a flat line throughout.

Track the shape over time, not just the centre. Stacked monthly histograms, or a simple time series of median, IQR and the two deciles, catch the divergence early enough to act on it. By the time drift shows up in the mean, it has usually been running for a quarter.

What distribution-reading unlocks

This is not a reporting nicety — it is the foundation for the advanced layer. Coaching prioritisation by bottom decile needs the tail visible. Item-level diagnosis needs per-item shapes. Calibration health monitoring needs compressed and polarised patterns recognised for what they are. Predictive QM models routinely find that distribution-shape features — median, IQR, percentiles — carry more signal than the mean alone.

The mature QM analyst reads distributions as a routine skill and reports the shape wherever it differs materially from the central tendency. The novice reports a team mean — technically true, operationally misleading — and loses the story the data was telling.

Reading QM distributions Patterns to recognise ▸ Bimodal — two stories, one number ▸ Right-skewed — exceptional tail ▸ Left-skewed — failure tail ▸ Drifting — shape moving over time ▸ Compressed — rater conservatism ▸ Polarised — rater inconsistency The response ▸ Plot before summarising ▸ Median + IQR + percentiles ▸ Mean only when symmetric ▸ Surface bottom & top deciles ▸ Track shape over time ▸ Compressed/polarised → calibrate The operational story lives in the shape, not the mean

The closing principle

Read the distribution before you summarise it. The team mean is rarely wrong and rarely useful; the shape — bimodal, skewed, compressed, drifting — is where the coaching decisions, the calibration signals and the early warnings actually live.

See also