How do I know whether an improvement actually worked?

Question

Accepted Answer

Judge it against the KPI it was meant to move, tracked over time, not against effort or enthusiasm. Use a feedback loop and a regular operational review: compare performance with the target, analyse the trend, and look at the real effect on customers. If the data says the change is not working, be willing to adjust it or stop it.

Start with the measure you chose before the change. A KPI is tied to a strategic objective, so the question is specific: did this number move, in the right direction, after the change? Read it as a trend, not a snapshot. The chapter's line is that a single data point is a snapshot and a trend line is a movie. Performance is a story told over time, so one good week proves very little.

Then check that the metric itself is honest. A team can hit its number and still fail the customer, as the chapter's premature celebration case shows: clicks rose sharply while customers could not complete their task. Add measures of the real outcome, such as task completion rate and customer satisfaction, alongside the headline number.

Use an operational review to make the judgement formal: review performance against the KPIs, analyse the trends, and score the work against a rubric that defines what good looks like. The chapter's scheduling system case shows the hard end of this discipline. A service team replaced a clunky scheduling system, and the data showed response time and scheduling accuracy getting worse, not better. The leader scrapped the new system and started over, this time involving the team in choosing. The system they picked was simpler, and productivity and morale rose. Knowing whether an improvement worked means being willing to act on the answer.

How do I know whether an improvement actually worked?

Related questions