Defining KM RoI in terms of critical failure cost

As we all know, trying to pin down a tangible return on investment (RoI) for Knowledge Management initiatives is often difficult.

What we do know is that KM implemented properly reduces risk profiles. For example, less chance of having to re-learn a process because your critical staff member just moved to Rome, or less chance of a critical failure driven by inadequate communication.

Sameer Patel focuses on Enterprise 2.0 and social computing rather than KM, but the same issues apply here as well. He has an interesting take on defining RoI in these circumstances:

The standard justification here is: X hours per employee saved x Y# of employees x Some cost per hour = Savings ... The problem [is] that none of them are really what a smart buyer can justify to a senior executive as “return”. What’s worse, software was sold a decade ago in this fashion and the scars from unattained real dollar productivity savings from CRM, Portal and KM implementations are still very visible in the enterprise ...

Enterprise social software needs to be sold based on one simple end goal: How the income statement will look like before and after the investment ... model just 2 representative use cases at narrow functional levels ... for instance "on 5 occasions, our software enabled a sales rep to find a subject matter expert/a white paper/an up sell opportunity, resulting in total sales of $20 million" is much more tangible for an IT director to take to her CFO.

I believe this is sound advice for any KM project, not just for E2.0-style initiatives. However, this post triggered a key realisation. KM should not be aiming to achieve "x minutes per day" style savings. Rather, our job is to save "$X million over 5 years" by reducing the number of critical incidents where problem solving failure and/or knowledge loss occur. Otherwise, Fogbank happens. There are three components to this equation for critical KM failure in an organisation:

  • the current risk probability
  • the level that KM can reduce that risk
  • the true cost of a critical KM failure

Let's assume for the sake of argument that KM can produce a positive RoI for Company X. So if Company X's directors don't accept the value of KM, there are three possible reasons:

  • they are underestimating the current risk of failure
  • they don't believe the quoted level for how KM can reduce that risk
  • they are underestimating the true cost of failure

Now, it should be possible to establish reasonable metrics for components 1 and 3. (Most companies will privately admit to their worst failures, and we can extrapolate the costs of these failures fairly easily.) Multiplying these two components gives KM a maximum budget.

Let's turn this into an example: If Company X has lost $10,000,000 over 3 years due to preventable KM failures, a KM program is a no brainer if it can reduce that amount by just 20% on a annual budget of $400,000 ($2 million saved vs $1.2 million outlaid).

Now for the scary bit - this gives KM a solid target to aim for and hence, full accountability. The one remaining problem is that because KM deals with low probability events, one bad year will throw the numbers right out. Similarly, one good year may look like a resounding success.

So management will need to be convinced to commit to KM for 3 or 4 years to make sure that overall trends are heading in the right direction.

Did you know...

Our expertise in complex systems analysis, combined with a deep understanding of technology and modern, agile management and leadership techniques makes knowquestion uniquely positioned to find strategic solutions to your tough problems. Contact us today.

Comments

Matt Moore (not verified) — Mon, 16/03/2009 - 08:22

Stephen - You are absolutely right - in theory. I agree that mins of time saved per employee measures are generally rubbish but these are generally used as windown dressing for projects that have already got approval for a different reason.

Looking at critical incidents implies a much longer time line than most senior managers seem to care about (5 years is about right). It also implies that you are measuring this stuff - which takes guts because it's about measuring (and thus admitting to) your screw ups.

It also assumes that critical incidents are events that follow a normal distribution. That may not be the case (suggest Taleb's The Black Swan as some background reading here).

Something like this does happen in some high-risk, capital-intensive industries (e.g. oil drilling, power plants, army medical practices) already. But this won't just be the outcome of a single project. It requires a company-wide orientation to better knowledge-based risk management.

Anonymous — Mon, 16/03/2009 - 08:22

That last comment was me. Can you fix your comments so posters can be identified? Or is this an outpost of Bloggers Anonymous? Cheer Matt Moore

Stephen Bounds — Mon, 16/03/2009 - 08:33

Hi Matt,

Good point. Hadn't checked that bit yet (this blog is still pretty new!). I've retroactively edited your comment to add your name, and allowed people to put in a name for all future comments. Hope that's OK!

I think you're pretty much on the money, by the way. But I was wondering about your comments that critical events are normally distributed. Why would that need to be the case to allow justification for a budget based on average expenses?

Matt Moore (not verified) — Mon, 16/03/2009 - 09:37

Because critical incidents vary wildly in terms of size & frequency. The frequency is important because that drives the timeframe you need to sample to make your business case. No point in choosing a 6 month timeframe if a potentially catastrophic event happens every 6 years approx. And you have to understand the relative impact of different risks. Unfortunately this is often done using a normal distribution model which is rarely valid. Hence your calculations will be off.

The other issue is that you may get a Black Swan after you start your KM program. Which will completely stuff your business justification (& possibly your business).

Two key books here:
- The Black Swan (as noted early)
- Risk Intelligence by Apgar.

Stephen Bounds — Mon, 16/03/2009 - 18:11

Ah, I see where you're coming from. This method wouldn't work if there was only one critical event every 6 years, you would need 50-60 years just to work out a baseline!

But when I say "critical", I don't mean that there were necessarily catastrophic results.

What I mean by a critical incident is one where knowledge loss or a problem solving failure was unequivocally a factor in causing a serious problem in a project or process (with accompanying financial loss). Most businesses with sub-optimal KM and a relatively dynamic environment would expect to have, say, up to 10 of these a year.

Remember that Black Swans may be unpredictable based on evidence, but they can be guarded against through resilient organisational design. So we can't just shrug our shoulders and say "didn't see that one coming". A resilient organisation, by definition, should be able to react to any circumstance, whether forecast or not.

Gordon Rae (not verified) — Tue, 17/03/2009 - 02:53

The most important risk factor in KM is the knowledge walks out the door. The most pragmatic, way to measure ROI on KM is to look at payroll and consultancy costs, staff turnover, and business continuity. What knowledge do you need to run your organization? Do you have a resilient, uninterruptible supply of that knowledge?

Neil Olonoff (not verified) — Tue, 17/03/2009 - 19:20

Steven
I very much agree with the premise. It's what I've been talking about in terms of "catastrophic knowledge failures," and then indicating that there are millions of smaller knowledge failures occurring every day unnoticed by the press. Examples of the "catastrophic" failures include the Challenger disaster, Fogbank (as we know now) etc. But thousands of smaller failures occur every day.

There's a distinction, though, to be made between the very process-driven activities (medical procedures, manufacturing processes, and engineering activities) and the messier "knowledge work" activities that take places in offices world wide. How do we look at those sorts of activities and compute lower failure rates? Is it even possible?
Regards,
Neil

Joe Firestone (not verified) — Thu, 19/03/2009 - 13:56

Hi Stephen,

Nice post. I like the main thought and have commented on it at some length here:

http://kmci.org/alllifeisproblemsolving/archives/knowledge-management-an...

I'll look forward to your reply.

Best,

Joe

Joe Firestone (not verified) — Thu, 19/03/2009 - 14:02

Hi Matt and Stephen,

I've got a review of Risk Intelligence here:

http://www.adaptivemetricscenter.com/Review%20of%20Risk%20Intelligence.html

Best,

Joe

Stephen Bounds — Thu, 19/03/2009 - 19:43

Hi Gordon,

Thanks for your comment. I agree with you, but the difficulty is that if you measure at the payroll level then there will be lots of other extraneous factors "polluting" the relative change in costs.

If organisations instead worked out (perhaps as a routine part of an After Action Review or Project Closure) the amount of time and money wasted through lost knowledge for initiatives, this might provide a more direct measure of the impact of KM in reducing lost knowledge.

Stephen Bounds — Thu, 19/03/2009 - 19:51

Interesting ideas Neil.

Part of the answer, I feel, is an openness in organisations to admitting when less-than-optimal outcomes have occurred in retrospect and tracking these in some way.

A reasonable trigger of knowledge loss or problem solving failure is when an issue gets escalated to the CEO level (or perhaps their 2IC). When that happens, this should be examined as a symptom of where an organisation's systems have broken down.

James Grey (not verified) — Fri, 20/03/2009 - 15:21

Stephen,
in the current financial circumstances, I think a horizon of 5 years is way too far in the future to consider. We are looking at how we can apply current knowledge quickly to realise business benefits. So rather than looking for failures and how to fix them, looking for gaps and how to fill them. By doing this you can quickly realise benefits for the organisation based on cash saved or generated rather than claiming you prevented something that migh or might not have happened in the future.

Cheers,
James.

Robert Webber (not verified) — Fri, 25/10/2013 - 21:38

I agree that trying to pin down a tangible return on investment for Knowledge Management initiatives is often difficult. Sameer Patel is actually right. It is always better to focus on Enterprise 2.0 and social computing rather than KM.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <small> <blockquote> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <br>
  • Lines and paragraphs break automatically.

More information about formatting options