|
Critique of a Program Evaluation (Group) |
|
Even if you never do a program evaluation, it is important for public
managers and policy analysts to be a good consumer of evaluations and be
able to communicate complicated evaluation findings to a lesser informed
audience of policymakers or the public. For this assignment, you
will be assigned to a group early in the semester. Your group
must identify a major evaluation of a policy or program implemented in
the public or nonprofit sector. The evaluation should be from a
reputable governmental or nongovernment organization (e.g., Brookings,
Urban Institute, etc.) and be available for download on the internet. It
should be methodologically interesting and be data rich. In other
words, it should be a substantive, well funded effort to evaluate a
program of some social significance. You will provide the class with a
link (or series of links) to the evaluation report and or executive
summary so that they can review it before your presentation.
|
| |
| Guidance for Critiquing Research
Designs/Evaluations |
| The objective of this assignment is to critique
an actual program evaluation and get you to think about its strengths
and weaknesses. Since nearly every evaluation has a flaw, it often
helps to think critically when reviewing evaluations and their research
design. That said, it is often impossible to do a perfect
evaluation so it is important to recognize that while the study may have
flaws, sometimes it is the best information that you have to work from.
The following websites provide some guidance for critiquing research
studies and evaluations: |
|
|
| |
| Written Assignment |
|
Your group should prepare a short concise critique of the evaluation in
the form of a 4 - 5 page memorandum. The critique should describe
the treatment (i.e., program), its intended targets, the intended
benefits, the program’s causal model/program logic/theory, the research
design, methods, and conclusions. You should then critique the
evaluation’s major strengths and weaknesses with a critical eye and
highlight any major sources of potential bias or problems related to the
study’s validity. |
| |
| Presentation |
|
You will present your findings during the last few
weeks of the semester when we are discussing measuring program impacts
and discussing randomized and quasi-experimental designs. At least
a week before your presentation, you
should circulate a copy of your draft critique
to the rest of class, a hyperlink to the final report, and if available
a link to an executive summary or website for the
evaluation project. Your presentation is limited to 12
minutes and should be prepared as if you were briefing an audience of
policymakers and the public. |
| |
| Grading |
| This is a group assignment that will be graded
based upon the quality of the written analysis and your presentation.
|
