Christmas cookies are everywhere, they come in a hundred varieties, and you need to sample eight of each. It makes perfect sense to you now, but come January 1, the mystery of where you gained 30 pounds begins. Such things can happen in metrics as well. Tom Cagley discusses the presence of gluttony in software metrics, and why it is clearly not a good thing.
Put the Cookie Down!
It is possible to put on excess metrics the same way you carry excess weight. This happens when people decide to measure things that serve no current function, but do so anyway because they “might need it someday.” Cagley sees two problems with this rationale.
The first and most obvious one is that it wastes the effort of the measurement team. If they are busy collecting data that nobody needs, it means they have less time to do meaningful analysis of the data that the business does need. Without suitable time for analysis, people are more prone to succumb to “cognitive bias” and see what they want in the numbers, instead of what is actually there. This is a problem that only gets worse with time, not better.
The second problem with the rationale is that it affects the measurement team’s credibility:
Collecting data that is warehoused just in case it might be important causes those who provide the measures and metrics to wonder what is being done the data. Collecting data that you are not using will create an atmosphere of mystery and fear. Add other typical organizational problems, such as not being transparent and open about communication of measurement results, and fear will turn into resistance.
There should be a feedback loop in place so that people understand why given measurements are being made. If there is no good answer for why the measurement is being taken, then you need to stop measuring it. Likewise, if you can imagine the taste of an almond candy cane cookie on your tongue even when you have not been near one all day, it is safe to say you have sampled enough of them by now. You can read Cagley’s full post here: http://tcagley.wordpress.com/2014/08/08/needs-edit-the-seven-deadly-sins-of-metrics-programs-gluttony/