Measure the Right Thing

Happy New Year!  There's an interesting phenomenon that accompanies each new year.  It generally goes something like this: set lofty goal (i.e. have that beach body I always wanted), set metric to measure progress, start working on goal, fail to meet metrics, give up.  Every single year millions of people worldwide set resolutions and promptly break them.  We're creatures of habit, so no real surprise - see The Power of Habit by Charles Duhigg.

My family has a fun tradition.  We get together each new years eve and we set predictions for the coming year.  We then review last year's predictions and see just how many we got right.  This year, if I recall correctly, we got about 40% right.

When my brother-in-law announced the percentage that we got correct for 2012, we started thinking about how we could bump that number up.  Naturally, instead of predictions like "there will be an earthquake of 4.5+ magnitude in Utah" - which was predicted this year - predictions started flying (particularly from the younger generation) that were more like "I will go to the bathroom this next year."  

I'm not sure we were measuring the right thing.  You see... people game the system.  In the QA world, it's like rewarding people for finding bugs.  If you measure and reward based on number of bugs found, you'll suddenly find that your team is suddenly a group of bug-detecting-geniuses.  There are a lot more bugs in the software, but not necessarily because they were there to start with.

Rather than aiming to get 100% correct on our predictions, we should look at adding some sort of probability factor for each prediction.  "I will go to the bathroom this year" - 99.999% probability leads to a certain weight - say 1.  Whereas "There will be an earthquake of 4.5+ magnitude in Utah" - lower probability (I'm just going to ballpark 15%) resulting in a higher weight - say 7.  Thus, next year we would get a much clearer picture of crazy predictions come true.  I will say that last year I predicted Joe Paterno would die.  We only knew at the time that he had a treatable form of lung cancer.

We need to be careful how measure things to make sure we're tracking what we want to be tracking.

What is one of your goals for the new year?  How are you going to track them?  Comment below or message me directly.