The first year provided promising benefits to legislative and executive branch leaders. The process has added more rigor to the state’s performance-based budgeting process by creating a clear standard for evaluating new funding requests. The information has proved particularly useful for legislators who were often confronted with making funding recommendations with limited information on program effectiveness. The process has also challenged agencies to carefully consider the new programs they propose to ensure that they are supported by strong evidence and include a plan for monitoring implementation.
During the first year, no program met all of the criteria, primarily because PEER staff determined that the supporting research provided in the program requests did not meet the standards for rigorous evidence as defined by Mississippi Code 27-103-159.2 However, some programs—whose funding requests lacked rigorous research but were otherwise considered promising—were approved to create a one-year pilot program. During that time, they must conduct research on the program’s effectiveness. Programs will be considered for continued funding the following year, depending upon the evaluation results. In cases in which outcomes require additional time to be measured, researchers will present interim data.
This approach—allowing agencies and program vendors to conduct research on pilot programs—is part of an effort to encourage internal capacity building, which the state believes is critical for effective evidence-based policymaking. “One of the things that stands in the way of evidence-based government is capacity,” said Kirby Arinder, research methodologist for the PEER committee. “If the agencies can be shown how to use rigorous research to select programs or modify existing programs, they can better incorporate it into their work.”
Mississippi encountered several challenges in implementing the new requirements for screening budget requests. Although the seven elements were designed to be applied to new intervention programs—in areas that traditionally have been rigorously evaluated and have a strong evidence base—the state received some requests for new programs that were not interventions and that were in policy areas where strong research on effectiveness is lacking. For example, one request sought to fund the training of a new cohort of state troopers, which is not an intervention program. Many requests for new program funding also came in during the legislative session from vendors outside the formal budget process, restricting the time available for PEER committee staff to closely analyze each request.
Triplett said these and other issues will be resolved as agencies become more familiar with the purpose of the seven elements and the type of evidence that should be provided, and as outside vendors come to understand the requirements and time frame for submitting requests. “The first year was a new process for everyone, and we are seeking to target the initiative better in future years,” said Triplett. To support agencies with their requests, PEER staff members regularly refer them to the Results First Clearinghouse Database for examples of intervention programs with high-quality research documenting their effectiveness.
As the seven elements are better understood statewide, Mississippi leaders are optimistic that this tool will be an important aspect of the state’s overall shift toward evidence-based policymaking and performance-based budgeting. Triplett believes that state agencies will learn the process as they create inventories of their programs in compliance with Mississippi Code Section 27-103-159 (2014), an undertaking with which the PEER committee staff is assisting. The legislation also requires agencies to establish a baselining procedure for evaluating programs that don’t meet the standards of “evidence-based” or “research-based” and a method for conducting cost-benefit analyses of all programs. “We can see that they are learning about what constitutes an evidence-based program through the inventory process,” said Triplett.