Realisation rate as a measure for the success of KM

Steve Matthews presents the synthesis of a discussion following a posting on Matthew Parson's book discussing two key drivers: leverage and realisation rate.

On the subject of realisation rates, Stuart Kay (Knowledge Manager at Baker McKenzie Australia), one of the early thinkers and writers on KM ROI in law firms argues that "If you simply benchmark KM against gross write downs, you may find yourself ruining the credibility of your KM effort as a result of a couple of poor hires a significant proportion of whose time is written down and over which you have no control or influence. The measure will not be accurate - it will be arbitrary, and you will essentially be gambling with your credibility."

I totally agree and this is one of the basic challenges of any performance management framework that looks beyond the traditional measures and aims at managing intangibles such as knowledge (or image or any other intangible). We can never be certain that there is not a lot of other factors influencing the target of our measures but we basically got to start somewhere. But maybe we should start somewhere below realisation rates. E.g. is there a correlation between usage of KM systems and hours billed in the first place or can junior lawyers be brought up to speed and billing more fees in practice areas where KM is further ahead?

It is important to discuss issues like realisation rates and leverage, but maybe at the stage of measuring KM where most firms I know about currently are, this is beyond a clear and arguable link. I believe performance management is about insight leading to action. If that insight is uncertain (due to many other factors potentially influencing the outcome), how can we determine the right action?

Sometimes these very general benchmarks without a framework of causal links and measures remind me of the ROI calculations of software vendors, which some of you may be familiar with (along the lines of "we implemented XYZ CMS and it saved us $5 mio in two months"). This is where Stuart Kay's notion of "gambling with your credibility" comes back in my mind.