Diagnostic, anomaly, predictive — the QM analytics ladder

Quality · ~7 minute read

Most QM functions stop at description — the dashboard says the score moved — or jump straight to action: coach the agents. The advanced function climbs a ladder: diagnostic (why did it move?), anomaly detection (flag it the week it happens), and predictive (which contacts, agents and calls are at risk next?). Each rung has its own discipline, and skipping rungs is how coaching effort ends up in the wrong place.

Diagnostic — from ‘it moved’ to ‘why’

The diagnostic run starts by confirming the move is real — SPC, not eyeballing — because diagnosing noise wastes everything that follows. Then drill down (which agents, items, contact types, raters, periods?) and decompose (a 4-point team drop might be three agents driving 60% of it). Then the step most functions skip: check rater, sample and data-quality causes before agent causes. The scoring may have drifted, the sample composition may have shifted, a pipeline may have dropped a week of events — all cheaper to verify than coaching is to deliver, and all unfair to coach agents for.

One worked case from the book: a team’s empathy score dropped 4 points; the old programme would have coached the team. Diagnosis localised the drop to credit-check calls after a product launch had forced a procedural opening — a process-design cause, confirmed by the same pattern appearing in other teams handling the product. The fix was routed to product, not to coaching. Six weeks later scores recovered, and the team’s coaching capacity hadn’t been spent on a process problem.

Anomaly detection — the automated front-end

Anomaly detection multiplies the analyst’s reach: instead of the QM lead spotting drift in the monthly pack, the system flags it within the week. Six streams are worth automating — operational score shifts, rater drift (generosity or harshness creeping into one rater’s pattern), agent-level changes, call-level oddities, compliance fails (real-time fast-track), and calibration signals. Match the method to the stream: rules for known failure modes like compliance, statistical detection as the workhorse, unsupervised ML where speech-analytics signals combine with scores into patterns no single threshold catches.

The trap that kills most deployments is alert fatigue. Twenty-five alerts a day across a QM team trains everyone to ignore them; tune to a deliberate false-positive budget — a few per analyst per day — route by severity, and log every alert’s disposition so thresholds improve on data. And keep a human in the loop: no flag triggers action automatically, ever.

Predictive — anticipating the outcome

The use-cases that fit QM data are specific: complaint risk within 7 days of a scored call; repeat-contact risk; escalation risk mid-call; coaching-need prediction per agent per scorecard area; and — in speech-analytics operations — which calls from a 100%-scored population most warrant a human reviewer’s time. Each couples a prediction to a named decision with a threshold for action.

The workflow disciplines are non-negotiable: time-aware data splits so the model never learns from the future; class-imbalance handling (complaints are rare — a model predicting ‘no complaint’ for everything scores 95% accuracy and is useless); a simple baseline the model must beat (one operation’s ‘flag if score below 70’ rule gave 9% precision; the model had to justify itself against that, and reached 34%); calibration so 70% means 70%; and production monitoring, because unmonitored models degrade silently.

Fairness — the discipline agent-level analytics demands

Anomaly flags and predictions about agents carry stakes that operational metrics don’t. A false positive enters someone’s record and triggers coaching for a non-issue; models trained on historical scoring inherit any historical rater bias; and flagged agents attract more scrutiny, generating more flags — a feedback loop. One operation’s quarterly fairness check found agents on one shift over-flagged: the cause was a harsh-scoring rater on that shift, not the agents — exactly the wrong people would have been actioned.

The defences: human-in-the-loop confirmation on every agent-level flag; verify rater, sample and data causes before agent action; conservative thresholds at agent level (under-flag rather than over-flag); periodic analysis of flag rates against protected characteristics; and transparency — agents should know the flagging exists and how it is used. Secret surveillance, once discovered, poisons the whole programme’s relationship with the agent population.

Climbing the ladder in order

The rungs depend on each other. Predictive models need the diagnostic discipline to interpret their flags and the SPC foundation to separate signal from noise; anomaly detection needs the diagnostic run applied to every confirmed flag; all three need the data model underneath. An operation that buys a predictive model without the diagnostic layer gets risk scores nobody knows how to act on.

Built in order, the ladder transforms the function: drift caught in weeks instead of quarters, compliance fast-tracked, coaching priorities surfaced from data rather than the monthly pack, and — at the top — a measured reduction in the outcomes the operation cares about. One complaint-risk model, properly baselined, calibrated and fairness-checked, was credited with a 19% reduction in customer-initiated complaints within six months.

The closing principle

Diagnostic before anomaly, anomaly before predictive, fairness throughout. Each rung multiplies the QM function’s reach — but only when the rung below is solid, and only when agent-level analytics is run with the care the stakes demand.