Several years ago – right before the start of the pandemic -- our book, The Promises and Pitfalls of Performance-Informed Management was published by Rowman & Littlefield.
This past weekend, we had a somewhat jarring reminder of the pitfall aspect of that title in a recently-released City of Austin performance audit: The audit, titled Strategic Direction 2023: Progress on Economic Opportunity and Affordability Outcome, takes issue with the timeliness and quality of the performance measures used to measure strategic outcomes in the Texas state capital.
The subhead of the audit, was sobering: “Issues With Performance Measures, Monitoring and Delays Affected the City’s Ability to Measure and Report Progress Towards a Key Strategic Outcome.”
While the audit noted that its exploration of the use of performance measurement only concerned one of six strategic objectives, it noted that the conclusions drawn likely applied “to the other five outcome areas as well.”
Here are a handful of problems cited that underscore fairly typical challenges with state and local government performance measurement:
· There were issues with the timeliness of reported data. In fact, only 23% of measures covered the most recently completed fiscal year. The audit labeled 46% of data as “old data.”
· While the goal was to measure equity, too few measures examined progress with an equity lens. While the five-year strategic planning document stated that “Race is the primary predictor of outcomes,” only 23% of economic opportunity/affordability measures could be disaggregated by race/ ethnicity,
· When surveyed by the auditor, department teams raised concerns about data quality. According to the audit, of 24 comments received, “the majority (67%) expressed negative opinions about the data quality of performance measures.” Problems focused on data sources that were difficult to update in a timely way; dependence on unreliable third-party data, and a lack of connection between the measures used and the work of their own departments.
· The performance measures lacked targets. “The lack of targets makes it very difficult to determine where the City has been successful or where more effort is needed,” auditors wrote.
· Over half the measures focused on “community indicators”, in which results depend on a wide variety of external factors with potential results difficult to connect with government funding, management or policy choices.
There’s a clear silver lining around the storm clouds that these problems represent. Left unexamined and unaddressed, the “promises” part of our book’s title can be little more than the clichéd “best laid plans,” that often go awry.
In fact, we want to give credit to the City of Austin Audit Office for following up on how the performance measurement effort is working. The problems cited just go to echo a message that we have delivered over time (and in our book) that performance measurement is hard to do, and that it heavily depends on timely and high-quality data.
This kind of audit should not discourage the effort, which is very necessary, but help to improve performance measurement in this and other cities, counties and states.
#StateandLocalGovernmentManagement #StateandLocalPerformanceAudit #StateandLocalGovernmentPerformanceMeasurement #StateandLocalPerformanceManagement #StateandLocalPerformance #StateandLocalGovernmentData #PublicSectorDataQuality #CityofAustin #AustinAuditOffice #PerformanceMeasurementPitfalls #PerformanceMeasurementChallenges #DataTimeliness #DataQuality #EconomicOpportunity #StateandLocalStrategicOutcomes #ThirdPartyDataReliability #PerformanceMeasurementTargets #PromisesandPitfallsofPerformanceInformedManagement #BarrettandGreene #CityPerformanceMeasurement #CommunityIndicators