Why KPIs are not the answer for complex systems (part 1)

There's a saying in politics: "When you are sick to the teeth of repeating a message, people are just starting to hear it". By that yardstick, Simon Guilfoyle would make a great politician. For 30 months straight he has maintained a blog with one overriding theme: that numerical targets create dysfunctional behaviour and should be eliminated.

Targets and KPIs are so ingrained into the management mindset that it never occurs to many people to question their necessity. This applied to me too: When I first heard Simon's message, I felt that such a blanket condemnation couldn't be justified. However, after a short conversation with Simon, he completely won me over. This is Simon's key argument:

[Numerical performance] data [cannot] establish a causal link between the number of outputs and whether the job gets done well. This is particularly relevant where managers are forced to rely on a proxy measure ...

In the public sector, accurate performance measurement is even more problematic [because it is] difficult to measure the outputs generated [and] notoriously difficult to establish a causal link between a specific activity and an eventual outcome ...

Where there are insufficient levels of staff ... it will be impossible to meet these targets, regardless of effort ... [If the] capability of the system prevents it from performing to the levels demanded ... setting targets will not change the capacity of the system.

There are three sorts of objective measurements:
(a) Input measures - observe what is being fed into the system (requests, raw materials, etc)
(b) Performance measures - observe an aspect of the system in operation (temperature, staff on leave, etc)
(c) Outcome measures - observe the services delivered and products leaving the system

In a contracting model you are operating in a black box system, where you can only measure inputs and outputs. For example, "I give you some original materials and a sum of money; you return with a set of bound reports."

In a traditional employment model, managers treated their systems as "white box" environments, where they can inspect and control all aspects of the operations of their system, and are expected to improve performance within it.

But in complex environments, you are operating in a "grey box". You can observe behaviour inside the system, and choose to directly intervene in its operations, but cause and effect aren't deterministic.

In a non-deterministic world, KPIs. JUST. DON'T. MAKE. SENSE. And let's be clear by what we mean by "KPI". It's an outcome that managers and teams are expected to "hit" continuously, or by a specific point in time.

In a complex space (and this is an ever-increasing percentage of the world), the focus shifts from control to coherence. In short: You don't make something happen; you enable it to happen. There are ways to make the propensity for something to happen more likely, but never a guarantee.

By holding people accountable for a target, you only guarantee that people will manipulate measurements of the target instead. It's their only reliable way to succeed.

In Part 2, we'll look more at how performance can be evaluated in an environment where success is as much about circumstances and luck as it is about the effort people put in.

This is Part 1 of a series on performance management for complex systems.

Did you know...

Our expertise in complex systems analysis, combined with a deep understanding of technology and modern, agile management and leadership techniques makes knowquestion uniquely positioned to find strategic solutions to your tough problems. Contact us today.

Comments

Anonymous (not verified) — Sat, 23/11/2013 - 01:33

Excellent article! So refreshing to hear from a public sector management perspective--we've struggled with bringing meaning to our KPIs for years!

Thank you!

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <small> <blockquote> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <br>
  • Lines and paragraphs break automatically.

More information about formatting options