Appropriate Measures

Penrock recently established a metric for judging the quality of my blog. It was, however, not a very good metric. He's measuring quality by number of entries. Fortunately for you, I can see past the actual metric to the ultimate goal. Whether I am meeting that goal is debatable, but so far I have refrained from taking the easy way out by pandering directly to the metric.

Metrics can be useful; with a good set of metrics you know where you stand and how far you have to go. Metrics give you something to aim for, which is sort of the problem. It's too easy to just aim for the metric and ignore the overall purpose. Thus it is important to make sure that your goals and the metrics that support them are completely aligned. Metrics that are poorly defined can be met without actually bringing you any closer to your goal. In extreme cases, poorly defined metrics can act counter to the original goal.

For example, consider my friend Dave. Dave designs software that is intended to help get species off the "Endangered Species List." Now, as it turns out, there are two ways in which a species can be removed from that list. The first is for the animal to increase in numbers until it is no longer considered endangered. This process is what Dave's software is intended to track.

Dave, who is a very helpful person, claims that he is so dedicated to this cause that he uses his spare time to help remove species from the list. He's doing what is within his power as a private citizen, so instead of a computer he claims to use a rifle (or in the case of baby seals which may or may not even be on the list, a club). He figures it is easier to get animals off the list by pushing them out the bottom than by attempting to carry them up to the top.

Now before you start sending Dave hate mail, it is worth pointing out (for those of you who are not familiar with my friend) that Dave's stories have roughly the same relationship to truth as Penrock's. So odds are good he's not actually spending his weekends trying to tip endangered species into extinction. However, that's technically a valid response when the goal is to reduce a metric solely defined as "number of species currently considered endangered."

So, metrics are important, but it is more important to have the right metrics. Faulty metrics can be ruinous. Dave's example might be a little extreme, but there are plenty of actual cases of such dangerous metrics (the No Child Left Behind Act springs immediately to mind). This is true for personal projects and it is certainly true for public policy . Unfortunately, since the ultimate metric in public policy is the vote, we may be in trouble.