BEYOND THE DATA

Not long ago we were at a presentation by an academic about a topic that was of genuine interest to us. As is often the case, she had many interesting slides, showing all of the data she had gathered over months of research.

But when one of the members of the audience inquired about the meaning behind the data, her answer was simple. “I don’t know. I only know the numbers.”

This was frustrating to us, and we suspect other folks in the audience. But it’s been our experience at various conferences, webinars and other presentation venues that a great deal of academic work doesn’t seem to take the next step forward, beyond doing meticulous research in order to find out what the research really means.

Don’t misunderstand, we know that academic research is often limited by time and money. But we’d like to make a modest proposal (and maybe this isn’t practical, but that doesn’t stop us from suggesting it): After the heavy number crunching is done, we’d like academics to have a handful of discussions with experts in the field to see how they would interpret the information they’ve gathered.

The responses that could be garnered through this extra step wouldn’t need to be published as gospel – but could at least give users of the published work a sense of what it might mean, creating a starting point for their own reflections.

We hasten to add that we’re not saying that all research should be qualitative. We’d just like the important findings of quantitative research to be presented in a real-world context of some kind.

Some of those reading this B&G Report will remember the Government Performance Project (GPP), which was a Pew-Charitable Trusts effort that was published in Governing Magazine, with the goal of evaluating the management capacity of the states (and occasionally cities and counties).

The research included lengthy survey instruments that covered the areas of human resources, budgeting, performance management, information technology and infrastructure, in an effort to evaluate the management capacity of states. We were deeply involved in its inception, research and writing. (Actually, it was loosely based on a similar, far less rigorous exercise that we did for the now defunct publication, Financial World.)

The GPP included a cadre of very smart academics with small teams of graduate students to go through the survey instruments – had scores of sub criteria that were then pulled together to analyze how the governments fared on clear-cut criteria, and then brought a full step forward to come up with overall evaluation. That could then be translated into grades.

But we also had a team of journalists, who did many hundreds of interviews with people in all fifty states and tried to get answers and context for all of the sub-criteria that the academics were using. This wasn’t always easy, but we were able to get people on the phone, to work with the journalistic team.

The journalistic input provided two benefits.

First, it sometimes discovered flaws in information that came from the surveys. While the journalists made every effort to reach people who were at the higher levels of government, sometimes the surveys were filled out by interns who didn’t necessarily provide the most accurate answers.

When this happened, the academic team was able to re-evaluate, and the two groups came together to find a consensus that we thought approximated the truth.

On the other side of the coin, sometimes interviews were inclined to whitewash their stories and put the full emphasis on the positives. The academics provided an intellectual grounding that provided accountability.

The other advantage was that the journalists were able to tell the stories that allowed readers to get more than “just the numbers.” We didn’t just provide the total grades and the grades in the criteria, but we were able to publish the stories behind the most intriguing results.

We apologize for this little trip down memory lane, but this exercise and the collaboration between journalists and academics has been a highlight of our careers, and taught us a great deal about the potential flaws in relying just on interviews (the way that journalists are inclined to do), as well as the booby-traps that can be hidden in seemingly accurate and complete data.

The results bore out the process, and we were aware that a number of states actually took action based on the findings. And whether you’re an academic or a journalist what could be better than that?

#StateandLocalGovernmentManagement #StateandLocalGovernmentPerformance #CityGovernmentManagement #CityGovernmentPerformance #CountyGovernmentManagement #CountyGovernmentPerformance #GovernmentPerformanceProject #PewCharitableTrusts #GoverningMagazine #FinancialWorld #StateandLocalGovernmentData #GovernmentDataAndRealWorldContext #StateandLocalPerformanceManagement #StateandLocalPerformanceMeasurement #CityPerformanceManagement #CountyPerformanceManagement #JournalistAcademicCollaboration #StateandLocalInformationTechnology #StateandLocalInfrastructureManagement #StateandLocalHumanResources #StateandLocalHR #StateandLocalGovernmentBudgeting #CityBudgeting #CountyBudgeting #StateandLocalFinancialManagement #CityFinancialManagement #CountyFinancialManagement #GradingStateManagement #GradingCityManagement #GradingCountyManagement #AcademicJournalistPartnership #StateandLocalData #StateandLocalDataMeaning #BandGReport #BarrettandGreeneInc