Program evaluation is the systematic collection of information about the activities, characteristics, and outcomes of programs. Programs are evaluated in order to (a) make evidence-based judgments about the program, (b) improve program effectiveness, and/or (c) inform decisions about future programming (Patton, 1997). Because usefulness to our client is our first criterion of quality, we maximize the usefulness of evaluations by always beginning with the question, «Who is going to use the evaluation findings and how will they use them?» Collaboration with program staff and managers is a key principle of our approach. The evaluation is planned so that our clients have a say in every stage: They set the objectives, discuss the findings, and help develop the recommendations. We balance this collaborative approach with independence and thoughtful adherence to professional standards.
Incorporation of Monitoring and Evaluation in Your Organization’s Activities
To incorporate monitoring and evaluation in your organization’s activities we help you develop and implement Monitoring and Evaluation:
- Policy—ground rules, framework, and principles for distributing the M&E function throughout your organization
- Guiding Principles—included in the policy or developed as a separate document. Some organizations even implement monitoring and evaluation standards
- Procedures—aligned with the new policy and principles
- Job Descriptions—for personnel in subdivisions that include M&E functions
- Manual—detailed practical recommendations, instruments, and case studies
- Glossary of Terms—definitions of core terms for internal use to compensate for the lack of common terminology in this field
Monitoring System Design
One of every manager’s goals is timely supervision of the process of program implementation—the ongoing activity called monitoring. Useful program monitoring requires a management system. While the system’s complexity depends on the complexity of the activity to be monitored, the outcome of any monitoring system is accurate and timely information on how well program activities are proceeding and whether there are any digressions from the intended plan. If a monitoring system is working well, it helps a manager appropriately guide the project by making timely decisions and adjustments. We help to design program and project monitoring systems. Together with program managers and staff (a) we work out the system, (b) prepare all necessary papers, (c) train people who will carry out the monitoring, (d) participate in the system implementation, and (e) improve it if needed. Creating a monitoring system includes identifying indicators of program or project implementation. We pay special attention to a convenient format for presenting program implementation information to decision makers.
While it is always informative to carry out an analysis of program outcomes and lessons learned, it is not always necessary or possible to use external evaluators. In consequence, we offer assistance in program self-evaluation in which our specialists help clients plan and organize their own internal evaluations. Our experience shows that professionally facilitated self-evaluation helps an organization quickly achieve better results.
In one- to two-hour formats, our specialists introduce you to the main principles of program evaluation, illustrating their presentation with examples of best practices in Russia and the NIS.
«Introduction to Program Evaluation»
This two- to three-day training gives a basic idea of all stages of evaluation and self-evaluation: purpose identification, planning, information gathering, information analysis, report writing, and preparation for the final discussion of findings. Each stage is accompanied by practical tasks based on real situations. Three-day seminars include more practical exercises.
«Program Monitoring and Evaluation. Basic Course»
This five-day seminar allows participants to engage in evaluation. The seminar’s program is designed to develop a detailed understanding of all stages of evaluation and to practically develop the most frequently used tools. We pay special attention to professional evaluation standards.
«Program Monitoring and Evaluation. Advanced Course»
This program is designed for those who decide to seriously engage in program evaluation. The training includes (a) a five-day “Basic Course”, (b) a 1.5–2 month evaluation exercise (a real project with distance consultations), and (c) a five-day final seminar, when we analyze the work done and provide additional training in the skills that proved insufficient for independent work.
«Monitoring and Evaluation for Donors»
This two- to three-day seminar is designed for managers and staff of organizations that fund projects and programs. In addition to the basic principles and techniques of evaluation and self-evaluation, training participants review possibilities of involving grantees in monitoring and evaluation carried out by donors. One of the central challenges the seminar addresses is how to improve a donor organization’s policies and procedures with the help of monitoring and evaluation. We offer participants practical examples and exercises that demonstrate the use of monitoring and evaluation in organizational development.
It is common knowledge that no matter how effective the training, certain skills and knowledge remain that cannot be transferred without working together. We offer our clients the option of carrying out their own project evaluations under our specialist’s supervision. Our experience has practically demonstrated the effectiveness of this approach for organizations that wish to prepare their own evaluators and to conduct training in this field. The duration of this kind of work depends on the scale of the project to be evaluated, with one compulsory requirement: All evaluation team members must attend our course «Introduction to Program Evaluation» or a similar training by another organization.