My good friend Jack Ganssle commented over at EETimes (also available on the TechOnline India site, with different comments) about my recent post on process standards. In it, Jack cautions against relying on “a strong feeling that ‘things are better.’” He recommends using measurements to bring it back to the realm of engineering.
Bob Pease, analog engineer at National Semiconductor and writer at EDN Magazine, used to say, “when something seems funny, measure the amount of funny.” That’s easier done in the engineering domain than the people domain, of course.
These two simple guidelines will help:
1. Measure the things you really care about. Too many people collect numbers that are based on guesses or on indefinable units. If you’re measuring productivity, figure out a way to measure delivered, working features, not “story points” or other estimates. Jack gave an example where, to increase the number of circuit boards being produced, the technicians were not bothering to repair the boards that didn’t work. They were tossing them aside, creating a pile of waste inventory. In Jack’s example, the productivity measurement only measured part of what was desired.
2. Use measurements to illuminate, not as a goal. As Michael Bolton says, good metrics allow you to ask better questions. They don’t answer them. The productivity numbers mentioned above didn’t say how productive the workers were being, because it didn’t show the wasted inventory.
Measuring things is a great way to sharpen the observations. It’s still an observation, though. Sometimes you can observe things that you can’t measure. In that case of a “strong feeling of ‘things are better,’” you may not be able to measure how much better. But you can still ask the data question, “What have you seen or heard that makes you think things are better?”
Observations without measurements are still valuable.