Policymakers seeking the best return on taxpayer dollars are increasingly focusing on cost-effective programs that have been proven to achieve desired outcomes. Evidence-based policymaking, which relies on rigorous analysis of program results to inform budget, policy, and management decisions, is one strategy gaining support among public leaders who want to reduce wasteful spending, expand successful programs, and strengthen accountability.
With state and local governments often challenged by budget shortfalls or changes in leadership priorities, laws that promote evidence-based programs are one of several strategies that can help institutionalize these practices and save taxpayer dollars over the long term.1 Several states have passed laws to prioritize evidence-based programs, including providing incentives for using such programs, requiring agencies to inventory and categorize existing programs, and prohibiting funding of those shown to be ineffective.2
This brief highlights laws in three states—Washington, Oregon, and Tennessee—mandating the use of evidence-based programs and practices, and documents each state’s experience, the impact of these efforts, and lessons learned. The analysis found that the laws drove state agencies to develop baseline information on existing services provided, create new data systems to monitor program implementation and outcomes, and prioritize funding to programs backed by evidence. The states’ success with this method has led policymakers to expand consideration of evidence-based approaches in areas beyond their initial ones.
The experiences in Washington, Oregon, and Tennessee provide important lessons for states interested in expanding their use of evidence-based programs. These insights include:
- Engage key stakeholders. Leaders from the three states noted the importance of gaining support early in the process from key stakeholders, including state agency leadership and contracted service providers charged with carrying out the legislative requirements.
- Funding percentage mandates can be powerful tools for change. Requirements to direct designated percentages of funds to evidence-based programs can be an effective tool in promoting evidence use. In Tennessee and Oregon, stakeholders reported that mandates helped promote reforms and establish valuable goals for agencies and providers. However, percentage mandates aren’t a necessity. Washington saw value in forgoing percentage requirements and giving agencies flexibility, as stakeholders noted that uniform requirements to fund evidence-based programs could have created incentives for agencies to overprescribe certain services.
- Require monitoring of program fidelity. Programs implemented with fidelity to the original design are significantly more likely to achieve positive outcomes than those that deviate from them.3 Washington and Tennessee required that programs be implemented with fidelity to their designated design and that agencies establish systems to regularly track and report this information. Several Oregon departments have created advanced data systems, recognizing the need to monitor program implementation.
- Include provisions for evaluating “homegrown” programs—those not based on a specific model. In Oregon and Tennessee, program leaders used assessment tools to determine whether providers were implementing services associated with effective, research-based practices, even if such programs were homegrown. Tennessee leaders in particular noted that this was important to building support among providers.4 Washington enables providers to submit evaluations of homegrown programs to a panel of experts; programs with strong evaluations may be included in the state’s inventory of evidence-based interventions.
- Establish a process for verifying compliance with evidence mandates. Options for this include surveying providers, developing an inventory of programs and comparing each to an approved list of evidence-based interventions, or using tools such as the Standard Program Evaluation Protocol or Correctional Program Checklist to assess providers. Each state profiled in this brief used a different process to gather and verify this information, and in some cases the method was left to the agencies to determine, creating inconsistent reporting of information.
- Consider available resources when requiring evidence-based programs. Although these can achieve better outcomes than other programs, they can also require additional expertise to implement. The three states noted challenges in building and retaining staff expertise, creating data systems that support fidelity monitoring, and evaluating programs in areas where few evidence-based program models exist. Washington had the benefit of the Washington State Institute for Public Policy (WSIPP) and the Evidence Based Practice Institute (EBPI) at the University of Washington to aid implementation. Stakeholders from Oregon and Tennessee noted that similar institutions would have helped in their states.
Washington: Required four state agency divisions that deliver prevention and intervention services for children’s mental health, child welfare, and juvenile justice to substantially increase their use of evidence-based programs using research and inventory methods developed by WSIPP and EBPI. The divisions were also directed to establish monitoring and quality control procedures to ensure that programs were effectively implemented.
Oregon: Required five state agencies to spend an increasing percentage of their funding on evidence-based programs, reaching 75 percent of available funds by the 2009-11 biennium. The law also required biennial reports to the Legislature that assess each of the agency’s programs and its progress toward meeting that percentage goal.
Tennessee: Required the Department of Children’s Services to target up to 100 percent of funding to evidence-based juvenile justice programs by 2012-13. The law says the department should require all program provider contracts to include evidence-based services, as well as monitoring and quality control procedures.
Legislation: H.B. 2536, passed in 2012.
Policy areas: children’s mental health, child welfare, and juvenile justice.
Key requirements: Each state division must establish a baseline of existing programs and substantially increase their use of evidence-based programs.
Over the past two decades, Washington has created a strong foundation for using evidence systematically to inform budget and policy development. In 1983, the Legislature created WSIPP to conduct and aggregate research on social policy programs. The institute has developed a cost-benefit model that calculates the return on investment in funding evidence- and research-based programs and promising practices.5 In 1997, the Legislature expanded its use of evidence-based strategies by enacting the Community Juvenile Accountability Act (CJAA), which is among the first laws in the nation to mandate that agencies direct funding toward cost-effective programs in juvenile courts.6 In 2007, the Legislature created EBPI to help develop effective practices in children’s mental health and improve access to services for children.
Building on this foundation, the Legislature passed H.B. 2536 in 2012 to significantly increase the use of evidence-based programs in the state’s child behavioral health, child welfare, and juvenile justice systems. These programs are administered by the Department of Social and Health Services—which includes the Behavioral Health Administration, Rehabilitation Administration, and Children’s Administration, along with the separate Washington State Health Care Authority. Although these agencies were already taking limited steps to implement evidence-based programs and practices, the Legislature wanted to expedite this progress by replicating the reforms initiated by the CJAA.7
“Change comes hard, and everyone believes they are already doing the best thing” for children and families, said Mary Lou Dickerson (D), a former state representative who was the champion and prime sponsor of H.B. 2536 and other critical legislation supporting evidence-based practices.8 “The concept I wanted to push was that over time, we would replace unproven practices with those that are evidence-based.”
The Legislature also sought to establish a baseline to track which evidence- and research-based programs were available statewide. To accomplish this, it required WSIPP and EBPI to create formal definitions of evidence, systematically review scientific research to identify effective programs in each policy area, and develop inventories categorizing programs based on their evidence of effectiveness. The agencies were directed to use these inventories to prepare a baseline assessment of the programs they administered and establish fidelity monitoring and quality control procedures.
Although the initial version of the bill included mandates that specific percentages of funding be directed to evidence-based programs, the Legislature changed its approach after receiving public feedback during committee hearings. Community-based provider representatives were concerned that the bill would negatively affect providers who were not yet implementing proven programs.9 The final legislation stipulated that each agency attain “substantial increases” in their use of evidence-based programs above the baseline assessment and submit a report to the Legislature by 2013 detailing strategies, timelines, and costs for achieving this increase.10
Each of the agencies affected by the law had unique structures and varied on the use of evidence-based programs when the law was enacted. For these reasons, its impact has also varied.
Behavioral Health Administration
The legislation had the greatest impact on the Behavioral Health Administration (BHA). Although the agency had implemented some evidence-based programs before the law went into effect, it lacked a good metric for tracking these programs and relied on informal surveys from providers for this information.11 The law spurred the BHA to create a standardized process for tracking the use of evidence-based programs over time, which has become a valuable tool for managers. After WSIPP updates the list of evidence-based practices each year, service providers must update their electronic codes to track and report program utilization to agency leaders.12
The BHA was the only state agency to create internal targets for evidence-based programs based on the law. According to Gregory Endler, former program administrator in the Children and Youth Behavioral Health Unit, the agency knew that it had the potential for significant improvement and sought to increase the use of evidenceand research-based programs by 7.5 percent a year, eventually reaching 45 percent.13
Although meeting the targets has been challenging, the agency has made progress, including embedding its goals in provider contracts and using corrective action plans for providers who have not met the thresholds.14 To help support the providers, the BHA collaborated with the University of Washington in 2015 to identify key barriers to successful implementation. The university also provides targeted technical assistance to the agency.15
The Washington State Rehabilitation Administration was already advanced in evidence-based policymaking at the time of the 2012 act, based on its experience in implementing the 1997 CJAA, which directed the agency to target funding to programs backed by rigorous research in reducing recidivism in the juvenile justice system. By 2012, the agency was spending 74 percent of its treatment program funds on evidence-based programs.16
The legislation benefited the division’s work in several ways. It prompted increased dialogue about evidence-based policymaking, particularly with provider organizations and communities across the state that were initially resistant to the approach. It also clarified the division leadership’s intention to invest scarce resources in programs that were backed by strong research. “We were already doing the intent of the bill, but it enabled us to further our conversations and expand to subpopulations that didn’t have evidence-based programs,” said Cory Redman, acting director of the Office of Juvenile Justice, who noted that this dialogue promoted greater consistency within the division, such as ensuring that juvenile justice drug courts across the state followed a similar model.17