CAPP Evaluation Partnership

ACT for Youth Center of Excellence (COE) is partnering with Comprehensive Adolescent Pregnancy Prevention (CAPP) providers in New York State to improve adolescent sexual health in communities throughout the state. Through evaluation, this statewide initiative will teach all of us about the benefits and challenges communities experience with regard to evidence-based programming.

This section is specifically for CAPP providers. Here you will find information about the evaluation process and tools, which are designed to help you improve your programs and reach your goals.

Why evaluate?

To be useful, evaluation must go beyond simply tallying up numbers of youth served. The goal of the evaluation process is to help us answer the questions "Are we making a difference? Can we improve on what we are doing?"

Evaluation should be useful: it should inform decisions to make us more effective and enhance the quality of our work. The role of the COE is to work in partnership with CAPP providers to help you improve program implementation, strengthen your work, and ultimately benefit the young people whom you serve. At the same time, your experience will add to the knowledge base, helping all of us understand what works when it comes to adolescent sexual health and evidence-based programming in diverse settings throughout New York State. If this evaluation process succeeds, what we learn together will influence future approaches to adolescent sexual health in our state and beyond.

What are we evaluating?

A core strategy of CAPP is the implementation of evidence-based programs (EBP), and this is currently where we are focusing evaluation efforts. Two types of evaluation are needed to improve program effectiveness and document the impact of programs.

Process evaluation tells us what happened in the program, and whether the program was implemented as planned. We know that implementation matters: implementing a program with fidelity and quality increases the chances that program goals will be achieved. We also know that it is not at all easy to replicate an EBP exactly as it was conducted in a research setting. Many factors -- at the community, organization, facilitator, and participant level -- will affect implementation; the program will never be offered under conditions that exactly match those under which it was originally tested. We have much to learn to help us close the gap between the demands of fidelity to a particular program and the demands of real world conditions. Evaluation of program implementation can help us close that gap and improve the quality of program delivery. Examples of process evaluation questions include:

• Which youth participated in our program? (Neighborhood, runaway/homeless, LGBTQ, foster children, etc.)
• How many sessions were offered?
• What percentage of participants attended all of the sessions?
• What program activities were conducted?
• What adaptations were made to the EBP?

Outcome evaluation tells us whether the program made a difference. Examples of outcome evaluation questions include:

• Have attitudes toward condom use changed?
• Has the intention to be sexually active in the near future changed?
• Has the intention to use birth control in the near future changed?
• Have participants gained confidence in their ability to negotiate condom use?
• Do participants feel more confident in their ability to find sexual health information and services?

COE/CAPP Partnership for Evaluation

The COE has designed CAPP evaluation tools that will help you document the implementation of your programs. Using these tools, CAPP providers will supply data to the COE, and in turn the COE will compile information from across the initiative and report back to providers.

These results will help you:

• Improve program implementation over the life of the grant
• Document that improvement for stakeholders, including youth
• Communicate with your community coalitions about program successes and challenges
• Make the case for future funding
• Contribute to the knowledge base about the realities of program implementation

From all of us at ACT for Youth Center of Excellence, thank you to all of the CAPP providers for your partnership in this effort!