Measuring Behaviour Change

Most L&D teams don’t need convincing that completion rate is the wrong lens for a performance-based organisation. Say it out loud in strategy meetings and people nod enthusiastically but there’s no framework on how to make it happen

Now that personalisation, conversation, reflection and learning can be personalised with AI, it would seem that this would make measurement easier and should be baked into behavioural outcome measurements. Picture a coaching tool for example.  It holds a real dialogue, follows up on last week’s commitment of a simulated conversation, quietly shifts how a manager handles the actual  difficult conversation. The manager’s confidence grows. Performance is addressed and the bad performer now has clear direction on what’s expected. The manager builds on their confidence to address things as they arise rather than ducking the issue that is affecting their team and morale more generally. The individual has an opportunity to grow and develop.

Now picture it sitting in that same quarterly review next to a content platform, judged on completion rate, average session length and cost per seat. Nobody thinks that’s the right test, but it gets used anyway, because it’s the test that’s already built into the delivery.

Behaviour Metrics

Personalised development, historically delivered by a human coach or mentor, is generally rooted in individual development, budgeted separately, and measured on completely different metrics such as  retention, changes in performance ratings as observed by managers, peers or team members, business outcomes tracked over quarters rather than individual learning sessions.

Burke and Hutchins’ 2007 review found no meaningful correlation between how much people rated a training experience and whether they actually applied it afterward (cited via Work-Learning Research).

Satisfaction, a metric many use  doesn’t predict what actually happens in the workplace – just the experience of the training itself. The widely cited estimate from Baldwin and Ford’s 1988 review, popularised by Broad and Newstrom, put transfer from training investment into lasting job behaviour at no more than around 10%, a figure that’s been refined since, but the underlying pattern has held

Human coaching’s evidence base rests on entirely different indicators. The ICF Global Coaching Client Study found 86% of organisations tracking coaching ROI had recouped their investment, with a median return around 7:1 (ICF, Coaching Statistics: The ROI of Coaching), measured through retention and business outcomes over time.  As this is a self-selected sample, the multiple could be taken with with a pinch of salt, but personalised development was never evaluated on a one-time learning event in the first place.

The real question is what is the correct measure for behaviour change , and whether you’d trust it enough to defend it. Is it quantitative as well as qualitative.

The Metrics That Show Development

Here’s what could be put on the scorecard instead.

• Applied commitment: Did someone try, this week, something that came up in a prior session? A named behaviour tied to a named session is a real signal. A vague “yes, helpful” isn’t.
• Observed shift: Does the person’s manager notice a change without being prompted? This one takes a full quarter to show up, but it’s the closest thing to ground truth, since it comes from someone who is seeing the behaviour shift day to day.
• Goal movement: Is there measurable movement on a goal the person created single session moving the needle is noise. Movement that holds across two or three check ins is signal.
• Retention and performance indicators: The same lagging indicators leadership development has always used, tracked on the same multi quarter horizon, not compressed into a first month verdict.

None of these show up in week one. That’s the point.

AIDA Is Built to Be Measured on Behaviour, Not Completion

We built AIDA, our AI-powered development tool, around this exact gap. AIDA holds adaptive, individual conversations and follows up over time rather than delivering a fixed content path, so we report on it the way we’d want to be judged: applied commitments, manager observed shifts, goal movement, retention, on the timeline those things actually take to show up.

Get in touch with our team to find out more.