“Randomized controlled trials (RCTs) have been called the gold standard for identifying the impacts of public service-delivery procedures,” write Harry Hatry and Batia Katz in a new report from the Urban Institute. But they can be very expensive and time consuming.
This makes them “impractical for many public service agencies, especially small government agencies and most NPOS,” write Hatry and Katz, “These problems have deterred agencies in local, state, or federal governments and NPOS from using the procedures.
Fortunately for some, there’s growing interest in a far simpler version of RCTs dubbed “Mini-RCTs,” which can be used for internal agency decision making (even if they don’t have sufficient rigor to be used as clear evidence for external use).
In both RCTs and Mini-RCTs, the report explains:
· Comparisons are made between service recipients who receive a new or modified service-delivery procedure and service recipients who did not receive the procedure.
· The selection of clients into each group is done randomly
· During the trial period, conditions are controlled so that none occur that could significantly affect the validity of the comparisons.
Hatry and Katz write that Mini-RCTs can be used for a number of purposes that are particularly timely right now, including determining the value of emerging new technologies; ways to recruit customers or volunteers and approaches to raise program revenues.
Some of the elements of Mini-RCTs include a willingness to get findings that fall short of ultimate outcomes but can use “intermediate outcomes that are somewhat easier and cheaper to measure . . .” and “indicate meaningful progress toward the end outcomes sought." In addition, they can abbreviate the length of the trial used and the number of participants involved. Of course, there are tradeoffs. As the report states, “The larger the number (of participants) the more accurate the findings will be, but the cost of data collection will be higher.”
The report concludes that, “Mini-RCTs have the potential to yield many small-scale benefits that add up to major national improvements in the effectiveness, efficiency, and equity of public services—by providing more accurate information on what service-delivery practices work and do not work. A major additional benefit is that Mini-RCTs can stimulate innovation and creativity by giving decisionmakers at organizations the option of testing changes before fully implementing them. Ultimately, Mini-RCTs in public service program management could broadly benefit service delivery and help customers across all types of organizations.”
#StateandLocalGovernmentPerformanceManagement #StateandLocalGovernmentPerformance #EvidenceBasedPolicyMaking #UrbanInstitute #HarryHatry #KatiaBatz #MiniRCT #StateandLocalGovernmentManagement #PublicSectorDataAnalysis #StateandLocalPerformanceMeasurement #PublicSectorProgramEvaluation #PublicSectorManagement #GovernmentInnovation #MiniRCTs