Evidence-Based Decision-Making

The Evidence-Based Decision-Making in Juvenile Justice Certificate Program is designed to help juvenile justice practitioners apply the extensive body of research around what works to reduce recidivism and use structured decision-making tools to improve efficiency and outcomes of their juvenile justice systems. Building on the Juvenile Justice System Improvement Project (JJSIP) and Juvenile Justice Reform and Reinvestment Initiative (JJRRI) demonstration projects, the Certificate Program aims to support state and local jurisdictions interested in implementing a comprehensive, evidence-based decision-making platform to improve youth outcomes, protect public safety, and reduce system costs.

Phase 1: Participants in the Certificate Program will receive instruction from researchers, policymakers and practitioners with experience integrating structured decision-making tools and practices into a holistic platform for juvenile justice decision-making. With guidance from instructors and completion of checklists throughout the training, participants will leave the Certificate Program with a detailed understanding of the components of the evidence-based decision-making platform and a plan for assessing and enhancing their readiness to implement the platform in their jurisdictions.

Topics covered include:

Aligning system policies, procedures, practices and approaches to support sustainable implementation of an evidence-based decision-making platform;
Operationalizing a rigorous and validated risk and need assessment process;
Utilizing dispositional matrices to guide decisions regarding supervision levels, placements and services;
Evaluating the effectiveness of programs for reducing recidivism, including through the use of the Standardized Program Evaluation Protocol™; and
Developing effective quality assurance and data evaluation mechanisms to ensure program fidelity and achievement of desired outcomes.

Phase 2: In the year following the Certificate Program, participants will also receive on-site and off-site targeted technical assistance (TA) to help them assess their site’s implementation readiness and identify and address gaps, develop a vision that incorporates evidence-based decision-making, and prepare a Capstone Project that outlines the goals, objectives and timeline for implementing an evidence-based decision-making platform in their jurisdiction. Upon approval of the Capstone Project, participants will receive an Executive Certificate from Georgetown University, membership into CJJR’s Fellows Network and ongoing support from its staff.

Phase 3: At the end of the Capstone year, each site will be given the opportunity to apply for two additional years of intensive TA through a separate contract to support them in implementing the evidence-based decision-making platform in their jurisdiction. Sites that are selected for this next phase of implementation TA will have clearly demonstrated during the course of their Capstone Project that they are likely to complete implementation in the two-year continuation period.

CJJR has partnered with Vanderbilt University’s Peabody Research Institute to provide this professional development opportunity.

Capstone Project
Tuition & Application
Application Guidelines
Curriculum & Instructors
What our participants are saying

Capstone Project

As part of Phase 2 of the Certificate Program training, participants are required to complete a Capstone Project – a set of actions designed to initiate or enrich collaborative efforts to prepare them to implement an evidence-based decision-making platform in their jurisdiction. During the one-year Capstone Project, participants will receive TA to help them assess and enhance their jurisdiction’s implementation readiness, identify and improve identified weak spots, develop a vision for juvenile justice reform that incorporates evidence-based decision-making, and prepare a plan that outlines the goals, objectives, and timeline for implementing an evidence-based decision-making platform in their jurisdiction.

In order to develop their Capstone Project, participants will draw upon and refine the plan created at the end of the Certificate Program to assess and enhance readiness to implement an evidence-based decision-making platform in their jurisdiction. Each team will be provided two-days of on-site consultation, as well as distance TA through phone, email and webinar consultation. Subject matter experts will be available to provide TA, and will be matched based on the needs identified after completing the checklists. Ideally, participants will complete the following tasks during the Capstone year:

Identify a broad-based stakeholder team to support implementation of the platform components, define leadership responsibilities, identify work groups to address specific tasks, and outline staffing responsibilities;
Complete a thorough inventory and assessment of existing policies, procedures, and practices related to the implementation of an evidence-based decision-making platform that will include the following tasks:
- Gauging the level of staff and stakeholder support for implementing an evidence-based platform that involves the routine use of structured decision-making tools and practices;
- Evaluating the use of risk and needs assessment tools in decision-making to assess the validity and reliability of the tools and the fidelity with which tools are being used to inform decision-making;
- Assessing the availability and quality of data necessary to implement the structured decision-making tools and processes included in the evidence-based decision-making platform, evaluate outcomes, and guide the development of policy and practice;
- Documenting the array of providers and services available to youth in the juvenile justice system; and
- Documenting the existence and nature of quality assurance procedures for monitoring the validity and reliability of the structured decision-making tools, defining and measuring outcomes, and recommending procedures and policies to align policies and practices to support evidence-based decision-making.

Complete a gap analysis and develop a plan for addressing any gaps in existing policies, procedures and practices identified during the assessment phase, thereby building capacity and readiness to implement the evidence-based decision-making platform;
Create a vision for juvenile justice reform that incorporates the use of structured decision-making tools and practices; and
Outline an action plan and timeline for implementing an evidence-based decision-making platform in their jurisdiction.

A one- to two-page Capstone Summary will be due approximately one month after the Certificate Program. The final Capstone Project Proposal is due approximately three months after the end of the Certificate Program. As indicated above, CJJR and Vanderbilt will be working with each team to support the development of their Capstone Project.

CJJR will request formal progress updates every six months after the Capstone Year to track progress and offer assistance. Based on these updates and implementation progress, CJJR recognizes the individual or team that has made the most significant progress in improving outcomes for youth in their community with the Capstone of the Year Award. First distributed in March 2012, this award marks an annual CJJR practice to honor and recognize the success, innovation, and tremendous work of our CJJR Fellows through their Capstone Projects.

Learn more about the Capstone of the Year Award.

Tuition & Application

The tuition for this program is $4,500 per person. Tuition includes the Phase I five-day Certificate Program training, and the Phase II Capstone Year TA during which each team will be provided two-days of on-site TA by two consultants, as well as regular distance TA through phone, email and webinar consultation. Tuition does not include the Phase III Implementation TA, which will require a separate application and contract for teams who demonstrate a readiness to implement an evidence-based decision-making platform in their jurisdiction following the Capstone year.

Tuition subsidies are available for participants with demonstrated financial need and with heightened readiness to utilize the curriculum to undertake changes in their local community. Read more about tuition.

Please direct any questions to jjreform@georgetown.edu.

Application Guidelines

Each team should be comprised of between seven and nine members and should include representatives from the following constituencies:

Directors, deputies and/or managers from probation and corrections agencies,
Staff responsible for training,
Staff responsible for data collection and quality assurance,
A member of the judiciary,
A representative from the public defender’s office,
A representative from the prosecutor’s office, and
Directors or managers from service providers.

Note that teams should comprise the individuals who will be most heavily involved in the implementation of the jurisdiction’s evidence-based decision-making platform. In addition, we strongly suggest that the teams include at least one member from the judiciary and the director of the jurisdiction’s juvenile justice agency, e.g., chief probation officer and/or state director.Teams that can demonstrate most or all of the following characteristics will be given preference for participation:

A strong history of juvenile justice reforms,
A year or more of experience using a validated risk assessment instrument,
A needs assessment procedure or combined risk and needs assessment tool in place,
The capacity to collect data over a sustained period of time—ideally, through the use of an existing automated information system–in order to complete SPEP^TM scoring and other data collection tasks,
A basic framework for a continuum of graduated sanctions/services,
Commitment to quality system improvements, and
The willingness to use research to maximize program outcomes and inform future funding decisions.

This program is NOT accepting applications from students who do not also hold a professional role in a child serving organization.

Curriculum & Instructors

The Evidence-Based Decision-Making Certificate Program is a 5-day period of intensive period of study for jurisdictions that seek to improve outcomes for youth in the juvenile justice system by creating an evidence-based decision-making platform. Participants will receive instruction from researchers, policymakers, and practitioners with experience integrating structured decision-making tools and practices into a holistic platform for juvenile justice decision-making. With guidance from instructors and completion of checklists throughout the training, participants will leave the Certificate Program with a detailed understanding of the components of the evidence-based decision-making platform and a plan for assessing and enhancing their readiness to implement the platform in their jurisdictions.

The curriculum includes the following modules:

Module 1: Introduction and Research on Youth in the Juvenile Justice System

During this session, instructors and participants will review research on the distinctive characteristics of adolescent development that are related to delinquent behavior, identification of precursors and pathways to delinquency, the influence of risk and protective factors on offending, the patterns of offenses and offenders that typically appear in juvenile justice systems, the nature of recidivism, and promotion of positive youth development. This introductory session will serve as the foundation for the rest of the Certificate Program.

Module 2: Essential Features of an Evidence-Based Decision-Making Platform

This session will examine the details of what it means to have a comprehensive evidence-based platform for juvenile justice decision-making. Both the prevention and graduated sanctions components of OJJDP’s Comprehensive Strategy for Serious, Violent, and Chronic Juvenile Offenders will be discussed, with particular focus on how to create an array of programs that addresses the needs and risk levels of the population of youth served so that the right services and supports are provided at the right time for the right youth. The session will include a review of structured decision-making tools, such as risk and needs assessments, disposition matrices, and program evaluations, that are vital to effectively managing the flow of juveniles through the system and minimizing recidivism. It will end with a brief discussion of how these tools and processes can be integrated to create a coherent evidence-based decision-making process.

Module 3: Systems Alignment

Implementation of structured decision-making tools often changes the way juvenile justice agency staff interact with each other and with their external stakeholders. These changes may require that juvenile justice agencies align their existing policies, procedures, and practices to support the use of the tools. This session will offer examples from the implementation experiences of the JJSIP and JJRRI sites to highlight the critical role of systems alignment (e.g., to address staffing levels and assignments, workforce development, and training) in the implementation of an evidence-based decision-making platform. Participants will also review the importance of multi-system collaboration, community engagement and long-term strategic planning to successful implementation. Participants will have the opportunity to discuss systems alignment issues that may arise in their jurisdiction and share ideas on ways to address them.

Module 4: Operationalizing Risk and Needs Assessment

This session will allow participants to examine more closely the most current research on risk and needs assessment and analyze (1) the way their jurisdiction targets a youth’s risks and needs in service provision, (2) the quality of data derived from their risk and needs assessment, and (3) the usefulness of risk and needs information in planning a framework for graduated sanction arrangements and interventions. This session will give participants an understanding of the critical importance of risk and needs assessment to all of the components of the evidence-based decision-making platform.

Module 5: Building a Disposition Matrix

The session will also focus on how the use of a disposition recommendation matrix is used to match youth to services, monitor the service array, and anticipate service expenditures. Participants will discuss issues related to developing and implementing a disposition matrix in their jurisdictions. Staff from the Florida Department of Juvenile Justice will review the steps they took to develop their disposition recommendation matrix, evaluate outcomes, provide feedback to stakeholders, and modify the tool to enhance its effectiveness.

Module 6: The Standardized Program Evaluation Protocol

The sessions that comprise this module will describe the SPEP^TM and how to implement it within the broader frame of the evidence-based decision-making platform. The topics to be covered will include:

The Use of Evidence-Based Practices: This section will review the varying definitions of what is “evidence-based” and how to use research to improve program practice along the entire juvenile justice continuum.

How the SPEP^TM Works: This section will highlight the research findings that informed the development of the SPEP^TM. The key characteristics of effective programs, such as program type, amount and quality of service, and risk level of youth served will be discussed. Consideration will be given to what needs to be in place to use this knowledge to evaluate programs in a system, such as a strong management information system and risk and need assessments.

Evaluating Programs with the SPEP^TM: This section will walk participants through the program evaluation process, including defining what constitutes a program, classifying existing programs into categories supported by evidence of effectiveness, generating program ratings, and understanding the resulting scores. During this session, participants will get hands-on experience by applying the SPEP^TM tool to an existing program in their system.

The Performance Improvement Process: This section will describe how service providers, working collaboratively with juvenile justice agencies and partners, can improve their SPEP^TM score in order to align their services with the characteristics of the most effective programs found in the research. Challenges such as provider acceptance, understanding, and buy-in as well as cost, fidelity and system sustainability will be explored. The discussion will also focus on ways that service providers and public agency officials can build constructive alliances and work together to develop performance improvement strategies.

Data-Driven Decision Making: This section will provide guidance on how to generate the necessary data on an ongoing basis, connect the SPEP^TM with existing data systems and automate the program ratings. A simulation will be provided to illustrate this process. The objective is to use the collective data systems for improved macro and policy level planning, including budgeting, staffing, and program development.

Module 7: Data Management and Evaluation

This session will expand on the preceding discussion by focusing on the individual and program level data that are required to support and evaluate the evidence-based decision-making model. It will include a review of the key variables needed to generate the SPEP^TM scoring system, construct and evaluate the disposition matrix, and assess outcomes, such as recidivism and costs. It will include discussion of the various ways that the JJSIP and JJRRI sites have approached data collection and review the types of research questions they have addressed.

Module 8: Quality Assurance (QA)

Successful implementation of an evidence-based, decision-making platform requires on-going monitoring to ensure that evidence-based tools are used with fidelity and incorporated appropriately and consistently in decision-making. The session will discuss a range of important QA activities, including monitoring the collection, reliability, and use of risk and needs data; overseeing the case planning process to determine whether case plans account appropriately for a youth’s criminogenic needs; tracking how staff use dispositional and service matrices to make recommendations regarding supervision and services; and providing oversight to the SPEP^TM process and related performance improvement process. Session leaders will describe the role of QA staff in recommending policies and procedures for improving staff training, evaluating staff and provider performance, collecting and analyzing data, sharing information with staff and stakeholders, and aligning systems to support evidence-based decision-making.

Module 9: Leadership and Sustainability

Managing the implementation of an evidence-based decision-making platform requires leaders who are committed to using research and data to inform practice, who can effectively communicate that commitment to staff and stakeholders, and who sustain their commitment over time. This session will highlight how leaders have guided successful implementation efforts by:

Creating a vision for juvenile justice reform that incorporates evidence-based decision-making;
Involving staff in all aspects of planning and implementation;
Training staff so they can consistently and accurately use evidence-based tools;
Allocating resources to ensure that there are sufficient staff to support the data collection, analytic, and quality assurance tasks necessary for implementing and sustaining the evidence-based platform;
Convening information-sharing sessions that encourage buy-in by affording opportunities for staff and external stakeholders to ask questions and provide feedback on the use of structured decision-making tools;
Creating partnerships with and achieving buy-in from key stakeholders to implement and sustain the work; and
Advocating for practices, policies, and legislation that help to institutionalize and sustain evidence-based decision-making as a central part of juvenile justice reform.

Module 10: Experienced Practitioner Panel

A panel of leaders will discuss how their jurisdictions successfully improved the way the juvenile justice systems serve youth through integration of structured decision-making tools and evidence-based decision-making. Panelists will provide participants with examples of successes and challenges in the work and engage in an interactive discussion around what participant teams seek to do through their own development and implementation of an evidence-based decision-making platform. After the panel, practitioners will stay to work with teams as they review checklists and develop action steps.

Module 11: Review of Checklists and Completion of Action Steps

At the end of the program, the teams will discuss how to integrate the tools discussed (risk and needs assessments, disposition matrices, SPEP^TM, etc.) into a system-level comprehensive strategy, and ultimately, into routine practice. With assistance from instructors and experienced practitioners, and aided by “implementation readiness” checklists, each team will develop a set of steps for assessing readiness to implement the components of an evidence-based decision-making platform. Teams will present their plan to the group for feedback and discussion.

Core instructors for the program may include:

Mike Baglivio, Director of Research and Program Development, G4s Youth Services, LLC
Shay Bilchik, JD, Research Professor and Director, Center for Juvenile Justice Reform, Georgetown University McCourt School of Public Policy
Stephanie Bradley, Ph.D., Director, Principal Investigator, EPISCenter, Pennsylvania State University
Michelle Brogden, Director of Evaluation and Quality Assurance, Delaware Division of Youth Rehabilitative Services
Gabrielle Chapman, Ph.D., Research Associate, Peabody Research Institute, Vanderbilt University
Nancy Dietz, Director, Delaware Division of Youth Rehabilitative Services
Mark Greenwald, Director of Research and Data Integrity, Florida Department of Juvenile Justice
Jeffrey Gregro, Deputy Chief, Berks County Juvenile Probation Office
James C. “Buddy” Howell, Ph.D., Comprehensive Strategies for Juvenile Justice, LLC
Mark Lipsey, Ph.D., Director, Peabody Research Institute, Vanderbilt University
Laura Moneyham, Deputy Secretary, Florida Department of Juvenile Justice
Beth Rosica, Ph.D. Vice President, Administration, VisionQuest National Ltd.
Keith Snyder, JD, former Executive Director, Pennsylvania Juvenile Court Judges Commission

For more information about the curriculum, download the 2017 Evidence-Based Decision-Making Application Packet.

What our participants are saying

“The Department’s Implementation of the Juvenile Justice System Improvement Project (JJSIP) fundamentally altered the course of juvenile justice in Florida. Participating in training workshops facilitated through Georgetown University in conjunction with Vanderbilt University, like the Evidence-Based Decision-Making Certificate Program, led the Department to implement a number of system reforms including an increased impetus on matching youth to the right service, at the right time, and in the correct dosage. This included the development of data-driven disposition recommendations that have contributed to recidivism reductions across the delinquency continuum in Florida.”

Christy Daly, Secretary, Florida Department of Juvenile Justice

“Berks County has benefitted tremendously from our participation in the Juvenile Justice System Improvement Project (JJSIP). It’s changed the way we have been able to identify gaps in services and then develop an array of services to meet the needs of youth at all risk levels. Our relationships with community partners and service providers have also become stronger. Through SPEP we have improved the quality of the services being delivered and ensured we are making appropriate referrals to the right programs for the right amount of time.”

Jeffrey Gregro, Deputy Chief, Berks County, Pennsylvania Juvenile Probation Office

“Delaware previously sent a Juvenile Justice team to CJJR to share lessons learned from our experience with the Juvenile Justice Reform and Reinvestment Initiative (JJRRI). Sharing our experiences with other participating states was an empowering process that helped Delaware build upon our Blueprint for Success. The Blueprint was designed to construct our evidence-based platform and use structured decision-making tools to improve efficiency and outcomes for youth. CJJR Certificate Programs are enriching and engaging to all child-serving leaders, and I encourage any practitioners to take advantage of the Evidence-Based Decision-Making Certificate Program.”

Nancy Dietz, Director, State of Delaware, Division of Youth Rehabilitative Services

“Reform through the Juvenile Justice System Improvement Project (JJSIP) in Florida has led to demonstrated improved outcomes and enhanced Department-Provider collaboration. The core components of JJSIP’s Comprehensive Strategy for Serious, Violent, and Chronic Juvenile Offenders and requisite graduated sanctions service array, coupled with a validated structured decision-making disposition recommendation matrix, ensures the right youth are placed in services matching criminogenic need and risk profiles to intervention intensity. The Standardized Program Evaluation Protocol (SPEP) ensures youth within programs across the continuum receive ideal dosage of quality intervention, delivered with optimum fidelity, to maximize program effectiveness and improve outcomes. The SPEP implementation has led to a process of continuous quality improvement and collaboration between the Department of Juvenile Justice and its contracted providers, rather than a relationship with adversarial undertones, which has further strengthened our capacity to effectively serve our youth.

The Evidence-based Decision-Making Certificate Program provides an exclusive opportunity to drive juvenile justice reform to measurable outcome improvement through strategic implementation of the core elements of JJSIP.”

Michael Baglivio, Director of Research and Program Development, G4s Youth Services, LLC

Download the Application Packet