The measure of measuring

Here’s Goodhart’s law: “When a measure becomes a target, it ceases to be a good measure.”

There’s a lot juice in that statement. The sentiment is hardly rare, but Goodhart’s law managed to bottle it. I wish I found it on the shelf earlier. Back in corporate life, I’ve probably written 10-page explanations and given 15-minute lectures in meetings that tried, and failed, to express the same thing.

I had more success presenting the concept in other environments, such as when I worked as a flight instructor. Student pilots often struggled with this measurement thing. For example, if you tell a student to maintain a certain airspeed, their eyeballs can almost bore holes in the airspeed indicator with their laser focus. Meanwhile, other measurable parameters, such as heading, altitude, propeller RPM, manifold pressure, and so on, start deviating from where they should be.

Measurement per se isn’t the problem, but measurement that lacks meaningful context can be a trap.

Of course, for a student pilot, grappling with such matters is a learning experience. They eventually get in synch with the bigger picture, and they learn to get seemingly contradictory parameters to work in harmony with each other. When this stuff finally flows together, it’s an “ah ha!” moment. All those measurements that seemed so hard to chase in the past have finally become safely tamed and duly obedient.

Although the “ah ha!” seems simple theoretically, and, heck, it is simple theoretically, there’s no way to get there with just theory. It takes actual hands-on practice to get there. The learning occurs as an experience. Having an experience, and “knowing” something from a book, are not the same thing.

Moving away from flying, I’ll pull Saipan’s hotels into this gig, since I see the same factors in play. If you think about your favorite hotel in Saipan, you’ll consider that whoever is managing it is juggling a mixture of things that can be measured and things that can’t be measured.

In the measurement realm, hotels have accounting software, and various metrics for operational matters, and various indexes and ratios for financial measures, and so on.

But there’s other stuff that’s important and that can’t be measured. You can’t quantify how inviting the lobby is, how friendly the service is, how enjoyable a guest’s experience was, or what general impression the market has of your hotel’s brand. Sure, you could ask people to rate such things, and that might be a good idea, but it’s a tacit admission that these things can’t be directly measured.

This is why good hotel managers get a lot of respect. They have to have a head for what the numbers say and also a mind for what people feel.

Well, now that I’ve gone from flying stories and down to the hotel lobby, I might as well saunter to the beach, plant my chair, and kick off June with a very cool thought indeed.

Here’s the thought: The most important things in our lives probably can’t be measured at all.

If you’d like some academic backup for such an assertion, I’ll offer this ancient Chinese wisdom:

“Good reckoning does not use counters.”

We can’t place a value on our family, for example. The value is usually so high there’s no number for it, and there’s no index for measurement.

And for all the talk about happiness, there’s no way to measure it that I’m aware of. Some studies solicit self-reported data, but, again, that’s just evidence that we’re dealing with something that can’t be directly measured on its own.

An economist might say that happiness is that which is increased when a person is able to indulge a preference. This makes sense at one level, yet on another level I think we’ve all seen cases where indulgence leads to what looks like unhappiness. I guess we’re talking about definitions here, not really measurements, so, well, never mind.

Moving back to Goodhart’s law, I think it is worth minding. It seems like modern life generates more and more measurements to throw at us. It’s up to us to determine how valid these measurements really are.