EASY MEASURES AREN’T NECESSARILY THE RIGHT MEASURES

We’ve been watching the Ken Burns documentary series about the history of the Vietnam War. It’s a difficult series to watch, but fascinating, nonetheless.

One point that the documentary makes is that, unlike other wars in which success was measured by the land overtaken, in Vietnam that didn’t apply as neither side ever really made any physical progress until after the end of the war, when North Vietnam prevailed and one nation emerged.

As a result, during the war, the measurement used was the number of dead bodies that could be counted (notwithstanding that it wasn’t always clear which side the bodies came from). The point Burns makes is that this was the easiest thing to measure and so was the metric upon which important decisions were made.

This struck a resilient chord for the two of us, thinking not of wars, but of the way states and localities often measure their success. They often focus on data that’s easier to collect but may be less helpful in achieving a desired result.

Take for example, crime rates, which are the measures upon which many police departments determine the degree to which they’re accomplishing their work. We won’t question that everyone wants less crime, but that measure leaves a great deal to be desired. For one thing, people can be fearful of reporting crimes (perhaps because of concern over retaliation or even when they don’t want to be identified by the authorities themselves). This can easily mean that there’s somewhat less crime than that which is being reported. The opposite problem also occurs when an apparent increase in crime comes from a shift in procedure or a change in reporting, leading to greater fears although the actual crime incidence hasn’t changed.

In our view, there may be somewhat better ways to measure crime-fighting success – and that’s the safety felt by the residents of a community.  Regular surveys can uncover this kind of data, but in many communities, surveys can be difficult to administer and unless great care is taken may not even reach the portion of the population that is most vulnerable to crime.

Then there are measures of success of tax incentives. Typically, cities or states turn to the number of new jobs created. But sometimes, these jobs are temporary, leaving a false impression. This is particularly true when it comes to incentives for data centers. Many new jobs are created when the centers are being constructed, but once they’re up and running, the jobs rapidly decline; it really doesn’t take too many people to keep a data center running.

It’s easy to proclaim victory when the first wave of jobs comes in, with few entities publicizing the reduction of jobs that follows, even though these are numbers that are available.

Homelessness provides another good example. Many places measure their success at dealing with this onerous problem by counting the number of shelter beds filled or the number of people who are processed into the system. But it would be more useful to consider the time-to-transition from temporary housing to more permanent abodes. Of course, this is much harder to measure, because fragmented state, local and non-profit systems often don’t share data, making it difficult to determine the results of a policy or how it’s implemented.

The purchase of new large IT systems can also be problematically measured. An entity may take credit for completing the installation of a new hardware or software system. But timeliness, while very important, doesn’t consider whether or not the new technology functions as intended. We can’t begin to count the number of times in which we’ve interviewed people at all levels of government who are woefully disappointed that new technology promises haven’t been kept. In that case, it matters less whether the IT was installed on schedule. It’s like having a train arrive at your station on time but then get stuck on the tracks.

In all these instances, there are no magic wands that leaders can wave to produce results measurements that can be put to good use. Our warning is simple: Be aware that the measurements you’re using are flawed and be careful about the ways they are used to make decisions.

#StateandLocalManagement #StateandLocalPerformanceManagement #StateandLocalGovernmentPerformance #StateandLocalGovernmentPerformanceMeasurement #StateandLocalGovernmentData #CityGovernmentPerformance #CityGovernmentManagement #CityGovernmentPerformanceMeasurement #CountyGovernmentManagement #CountyGovernmentPerformanceMeasurement #CityPerformanceManagement #CountyPerformanceManagement #StateandLocalCrimeMeasurement #StateandLocalCrimeData #StateandLocalTaxIncentiveResults #MeasuringResults #StateandLocalHomelessnessMeasurement #StateandLocalHomelessnessData #MeasuringStateLocalTaxIncentiveSuccess #BandGReport #BarrettandGreeneInc