Evidence-Based Decision-Making

The Evidence-Based Decision-Making in Juvenile Justice Certificate Program is designed to help juvenile justice practitioners apply the extensive body of research around what works to reduce recidivism and use structured decision-making tools to improve efficiency and outcomes of their juvenile justice systems. Building on the Juvenile Justice System Improvement Project (JJSIP) and Juvenile Justice Reform and Reinvestment Initiative (JJRRI) demonstration projects, the Certificate Program aims to support state and local jurisdictions interested in implementing a comprehensive, evidence-based decision-making platform to improve youth outcomes, protect public safety, and reduce system costs.

Phase 1:

Participants in the Certificate Program will receive instruction from researchers, policymakers and practitioners with experience integrating structured decision-making tools and practices into a holistic platform for juvenile justice decision-making. With guidance from instructors and completion of checklists throughout the training, participants will leave the Certificate Program with a detailed understanding of the components of the evidence-based decision-making platform and a plan for assessing and enhancing their readiness to implement the platform in their jurisdictions.

Topics covered include:

Aligning system policies, procedures, practices and approaches to support sustainable implementation of an evidence-based decision-making platform;
Operationalizing a rigorous and validated risk and need assessment process;
Utilizing dispositional matrices to guide decisions regarding supervision levels, placements and services;
Evaluating the effectiveness of programs for reducing recidivism, including through the use of the Standardized Program Evaluation Protocol™; and
Developing effective quality assurance and data evaluation mechanisms to ensure program fidelity and achievement of desired outcomes.

Phase 2:

In the year following the Certificate Program, participants will also receive on-site and off-site targeted technical assistance (TA) to help them assess their site’s implementation readiness and identify and address gaps, develop a vision that incorporates evidence-based decision-making, and prepare a Capstone Project that outlines the goals, objectives and timeline for implementing an evidence-based decision-making platform in their jurisdiction. Upon approval of the Capstone Project, participants will receive an Executive Certificate from Georgetown University, membership into CJJR’s Fellows Network and ongoing support from its staff.

Phase 3:

At the end of the Capstone year, each site will be given the opportunity to apply for two additional years of intensive TA through a separate contract to support them in implementing the evidence-based decision-making platform in their jurisdiction. Sites that are selected for this next phase of implementation TA will have clearly demonstrated during the course of their Capstone Project that they are likely to complete implementation in the two-year continuation period.

CJJR has partnered with Vanderbilt University’s Peabody Research Institute to provide this professional development opportunity.

Curriculum

The Evidence-Based Decision-Making Certificate Program is a 5-day period of intensive period of study for jurisdictions that seek to improve outcomes for youth in the juvenile justice system by creating an evidence-based decision-making platform. Participants will receive instruction from researchers, policymakers, and practitioners with experience integrating structured decision-making tools and practices into a holistic platform for juvenile justice decision-making. With guidance from instructors and completion of checklists throughout the training, participants will leave the Certificate Program with a detailed understanding of the components of the evidence-based decision-making platform and a plan for assessing and enhancing their readiness to implement the platform in their jurisdictions.

The curriculum includes the following modules:

Module 1: Introduction and Research on Youth in the Juvenile Justice System

During this session, instructors and participants will review research on the distinctive characteristics of adolescent development that are related to delinquent behavior, identification of precursors and pathways to delinquency, the influence of risk and protective factors on offending, the patterns of offenses and offenders that typically appear in juvenile justice systems, the nature of recidivism, and promotion of positive youth development. This introductory session will serve as the foundation for the rest of the Certificate Program.

Module 2: Essential Features of an Evidence-Based Decision-Making Platform

This session will examine the details of what it means to have a comprehensive evidence-based platform for juvenile justice decision-making. Both the prevention and graduated sanctions components of OJJDP’s Comprehensive Strategy for Serious, Violent, and Chronic Juvenile Offenders will be discussed, with particular focus on how to create an array of programs that addresses the needs and risk levels of the population of youth served so that the right services and supports are provided at the right time for the right youth. The session will include a review of structured decision-making tools, such as risk and needs assessments, disposition matrices, and program evaluations, that are vital to effectively managing the flow of juveniles through the system and minimizing recidivism. It will end with a brief discussion of how these tools and processes can be integrated to create a coherent evidence-based decision-making process.

Module 3: Systems Alignment

Implementation of structured decision-making tools often changes the way juvenile justice agency staff interact with each other and with their external stakeholders. These changes may require that juvenile justice agencies align their existing policies, procedures, and practices to support the use of the tools. This session will offer examples from the implementation experiences of the JJSIP and JJRRI sites to highlight the critical role of systems alignment (e.g., to address staffing levels and assignments, workforce development, and training) in the implementation of an evidence-based decision-making platform. Participants will also review the importance of multi-system collaboration, community engagement and long-term strategic planning to successful implementation. Participants will have the opportunity to discuss systems alignment issues that may arise in their jurisdiction and share ideas on ways to address them.

Module 4: Operationalizing Risk and Needs Assessment

This session will allow participants to examine more closely the most current research on risk and needs assessment and analyze (1) the way their jurisdiction targets a youth’s risks and needs in service provision, (2) the quality of data derived from their risk and needs assessment, and (3) the usefulness of risk and needs information in planning a framework for graduated sanction arrangements and interventions. This session will give participants an understanding of the critical importance of risk and needs assessment to all of the components of the evidence-based decision-making platform.

Module 5: Building a Disposition Matrix

The session will also focus on how the use of a disposition recommendation matrix is used to match youth to services, monitor the service array, and anticipate service expenditures. Participants will discuss issues related to developing and implementing a disposition matrix in their jurisdictions. Staff from the Florida Department of Juvenile Justice will review the steps they took to develop their disposition recommendation matrix, evaluate outcomes, provide feedback to stakeholders, and modify the tool to enhance its effectiveness.

Module 6: The Standardized Program Evaluation Protocol

The sessions that comprise this module will describe the SPEP^TM and how to implement it within the broader frame of the evidence-based decision-making platform. The topics to be covered will include:

The Use of Evidence-Based Practices: This section will review the varying definitions of what is “evidence-based” and how to use research to improve program practice along the entire juvenile justice continuum.

How the SPEP^TM Works: This section will highlight the research findings that informed the development of the SPEP^TM. The key characteristics of effective programs, such as program type, amount and quality of service, and risk level of youth served will be discussed. Consideration will be given to what needs to be in place to use this knowledge to evaluate programs in a system, such as a strong management information system and risk and need assessments.

Evaluating Programs with the SPEP^TM: This section will walk participants through the program evaluation process, including defining what constitutes a program, classifying existing programs into categories supported by evidence of effectiveness, generating program ratings, and understanding the resulting scores. During this session, participants will get hands-on experience by applying the SPEP^TM tool to an existing program in their system.

The Performance Improvement Process: This section will describe how service providers, working collaboratively with juvenile justice agencies and partners, can improve their SPEP^TM score in order to align their services with the characteristics of the most effective programs found in the research. Challenges such as provider acceptance, understanding, and buy-in as well as cost, fidelity and system sustainability will be explored. The discussion will also focus on ways that service providers and public agency officials can build constructive alliances and work together to develop performance improvement strategies.

Data-Driven Decision Making: This section will provide guidance on how to generate the necessary data on an ongoing basis, connect the SPEP^TM with existing data systems and automate the program ratings. A simulation will be provided to illustrate this process. The objective is to use the collective data systems for improved macro and policy level planning, including budgeting, staffing, and program development.

Module 7: Data Management and Evaluation

This session will expand on the preceding discussion by focusing on the individual and program level data that are required to support and evaluate the evidence-based decision-making model. It will include a review of the key variables needed to generate the SPEP^TM scoring system, construct and evaluate the disposition matrix, and assess outcomes, such as recidivism and costs. It will include discussion of the various ways that the JJSIP and JJRRI sites have approached data collection and review the types of research questions they have addressed.

Module 8: Quality Assurance (QA)

Successful implementation of an evidence-based, decision-making platform requires on-going monitoring to ensure that evidence-based tools are used with fidelity and incorporated appropriately and consistently in decision-making. The session will discuss a range of important QA activities, including monitoring the collection, reliability, and use of risk and needs data; overseeing the case planning process to determine whether case plans account appropriately for a youth’s criminogenic needs; tracking how staff use dispositional and service matrices to make recommendations regarding supervision and services; and providing oversight to the SPEP^TM process and related performance improvement process. Session leaders will describe the role of QA staff in recommending policies and procedures for improving staff training, evaluating staff and provider performance, collecting and analyzing data, sharing information with staff and stakeholders, and aligning systems to support evidence-based decision-making.

Module 9: Leadership and Sustainability

Managing the implementation of an evidence-based decision-making platform requires leaders who are committed to using research and data to inform practice, who can effectively communicate that commitment to staff and stakeholders, and who sustain their commitment over time. This session will highlight how leaders have guided successful implementation efforts by:

Creating a vision for juvenile justice reform that incorporates evidence-based decision-making;
Involving staff in all aspects of planning and implementation;
Training staff so they can consistently and accurately use evidence-based tools;
Allocating resources to ensure that there are sufficient staff to support the data collection, analytic, and quality assurance tasks necessary for implementing and sustaining the evidence-based platform;
Convening information-sharing sessions that encourage buy-in by affording opportunities for staff and external stakeholders to ask questions and provide feedback on the use of structured decision-making tools;
Creating partnerships with and achieving buy-in from key stakeholders to implement and sustain the work; and
Advocating for practices, policies, and legislation that help to institutionalize and sustain evidence-based decision-making as a central part of juvenile justice reform.

Module 10: Experienced Practitioner Panel

A panel of leaders will discuss how their jurisdictions successfully improved the way the juvenile justice systems serve youth through integration of structured decision-making tools and evidence-based decision-making. Panelists will provide participants with examples of successes and challenges in the work and engage in an interactive discussion around what participant teams seek to do through their own development and implementation of an evidence-based decision-making platform. After the panel, practitioners will stay to work with teams as they review checklists and develop action steps.

Module 11: Review of Checklists and Completion of Action Steps

At the end of the program, the teams will discuss how to integrate the tools discussed (risk and needs assessments, disposition matrices, SPEP^TM, etc.) into a system-level comprehensive strategy, and ultimately, into routine practice. With assistance from instructors and experienced practitioners, and aided by “implementation readiness” checklists, each team will develop a set of steps for assessing readiness to implement the components of an evidence-based decision-making platform. Teams will present their plan to the group for feedback and discussion.

Application Guidelines

Each team should be comprised of between seven and nine members and should include representatives from the following constituencies:

Directors, deputies and/or managers from probation and corrections agencies,
Staff responsible for training,
Staff responsible for data collection and quality assurance,
A member of the judiciary,
A representative from the public defender’s office,
A representative from the prosecutor’s office, and
Directors or managers from service providers.

Note that teams should comprise the individuals who will be most heavily involved in the implementation of the jurisdiction’s evidence-based decision-making platform. In addition, we strongly suggest that the teams include at least one member from the judiciary and the director of the jurisdiction’s juvenile justice agency, e.g., chief probation officer and/or state director.Teams that can demonstrate most or all of the following characteristics will be given preference for participation:

A strong history of juvenile justice reforms,
A year or more of experience using a validated risk assessment instrument,
A needs assessment procedure or combined risk and needs assessment tool in place,
The capacity to collect data over a sustained period of time—ideally, through the use of an existing automated information system–in order to complete SPEP^TM scoring and other data collection tasks,
A basic framework for a continuum of graduated sanctions/services,
Commitment to quality system improvements, and
The willingness to use research to maximize program outcomes and inform future funding decisions.

This program is NOT accepting applications from students who do not also hold a professional role in a child serving organization.

More Information

As part of the Certificate Program, participants are required to develop and implement a Capstone Project during the twelve-month period following the completion of their Certificate Program session.

Learn More

The primary goal of all of our Certificate Programs is not simply knowledge, but effective implementation in a local reform effort. We study “what works” and collaborate with leaders to bridge the gap between theory and practice.

Learn More