Dr. Susan Cramer
College of Education and Human Services
University of Wisconsin Oshkosh
Meeting with our Advisory Board is always informative. At our spring meeting one person started asking about evaluating programs and community impact. That prompted me to explore this topic in more depth and bring you a short list of sites on these topics. If you have additional sites you use, send them to me cramer@uwosh.edu and I’ll list them in our next issue.
Program evaluation is a critical activity for agencies. It can tell you if your programs are making a difference in the lives of your clients as well as adding to the quality of life in your community. It can help you make decisions in times of scarce resources as well as trumpet successes when you are conducting fund raising campaigns. Funders typically want to know that your program has achieved the goals set forth in the proposal. And, you can use evaluation to determine if the program was delivered as designed.
The type of evaluation you conduct depends upon what questions you have and what answers are needed. Typically, one selects from the four major types of evaluation: impact, performance monitoring, process, and cost. Resources listed below will explain each of these types of evaluations and more.
Performance Measurement and Evaluation: Definitions and Relationships
http://www.gao.gov/special.pubs/gg98026.pdf
This five page document, written by the United States General Accounting Office (GAO), provides concise, easy to understand definitions for each of the four major types of program evaluation. It then explains the relationship between performance measures and program evaluations. This is a great introduction to program evaluation terminology.
Evaluation Strategies for Human Services Programs: A Guide for Policymakers and Providers
http://www.ojp.usdoj.gov/BJA/evaluation/guide/documents/evaluation_strategies.html
This paper provides a thorough discussion of evaluation principles, designs, and issues. It not only discusses the four major types of evaluation but also discusses experimental, quasi-experimental and non-experimental impact evaluations. Read this paper when you are serious about program evaluation.
Evaluating Juvenile Justice Programs: A Design Monograph for State Planners
http://www.ojp.usdoj.gov/BJA/evaluation/guide/documents/documentg.html
Also published by the United States Department of Justice, this monograph goes into more detail than the previous resource. Use the two together to get a relatively complete handle on program evaluation designs, issues, and concerns.
Cost-Benefit Analysis for Juvenile Justice Programs
http://www.jrsa.org/jjec/about/briefing_cost-benefit.html
Cost-benefit-analysis is not a complete program evaluation strategy. It does not even answer the fundamental program evaluation question of “does this program work?” It can however begin to provide a “’bottom line’ economic estimate for the program.” Since many programs struggle with determining how they are positively impacting their communities, this resource may provide insight into how to arrive at a dollar figure that represents your impact. It’s an easy read, check it out.
Connecting Program Outcome Measurement to Community Impact
United Way Outcome Measurement Resource Network
http://national.unitedway.org/outcomes/resources/
Agencies and programs do not exist in a vacuum. A myriad of competitors and collaborators exist within communities and regions providing services that may be similar to yours. This monograph discusses how local United Way chapters can examine the programs it funds within the context of the larger community for the purpose of calculating the impact these programs are making. Individual agency directors can take these same ideas and apply them to their agency and programs to gain greater insight into the impact they are making in their community. Read this article when you are ready to do strategic planning, a major reorganization, or need to better articulate how your agency contributes to your community.