Public Policy Evaluation 2100-PP-M-D3EWPP
The course aims to equip students with the knowledge and skills necessary to plan, conduct, and utilize evaluations in the public sector within a dynamically changing reality. The course demonstrates the evolution of evaluation: from classic research, through strategic knowledge management, to the intelligent augmentation of analytical processes.
The workshops will cover the following thematic blocks:
From Analysis to Evaluation – Introduction: Establishing the place of evaluation in the policy cycle. Defining the differences and links between analysis (ex-ante) and evaluation (ex-post). Discussing key evaluation types and standard assessment criteria (OECD DAC).
Intervention Logic, Theory of Change, and the Behavioral Perspective: A workshop where "projecting outcomes" (from Bardach's analysis) will be transformed into a formalized Theory of Change. The module will be enhanced with a behavioral perspective (based on Chapter I of "(R)ewaluacja 2"), showing why interventions must account for cognitive biases and heuristics, not just the rational actor model.
Designing an Evaluation Study with GenAI: Formulating evaluation questions and selecting criteria. Workshop: Using Generative AI to design the scope of the study (based on Chapter 5 of "Generative AI in research"). Students will learn to use language models to generate and critically assess initial proposals for evaluation questions and criteria.
The Evaluator's Workshop – AI-Assisted Research Methods: A module dedicated to modern data collection and analysis methods, integrated with the practical application of AI.
Fundamentals of Working with AI: Prompting, data security, and ethics (Chapters 2 & 3 of the GenAI book).
AI in Literature Reviews: Using AI tools for rapid review and synthesis of existing knowledge (Chapter 7).
AI in Qualitative and Quantitative Data Analysis: Practical examples of using AI for coding transcripts, sentiment analysis, basic statistical analysis, and data visualization (Chapters 6 & 8).
From Results to Recommendations – The Evaluator as a Knowledge Broker: This module focuses on the utility of evaluation. Discussing the process of creating recommendations that are genuinely useful to decision-makers. Introducing the concept of the evaluator as a knowledge broker (Chapter II of "(R)ewaluacja 2"). Workshop: Communicating results with GenAI (Chapter 10) – students will learn how to use AI to personalize and create various communication products (e.g., policy briefs, press releases) for different target audiences.
Type of course
Mode
Prerequisites (description)
Course coordinators
Learning outcomes
KNOWLEDGE: The student knows and understands:
The place and role of evaluation in the policy cycle and its relationship with policy analysis.
Key concepts, types, and standard criteria of evaluation.
Fundamental concepts of behavioral economics and their implications for public policy design and evaluation.
The role and tasks of an evaluation unit as a knowledge broker in the public system.
The principles, capabilities, and limitations of generative AI in the context of evaluation research.
SKILLS: The student is able to:
Reconstruct a Theory of Change for a selected policy, incorporating behavioral mechanisms.
Formulate relevant and measurable evaluation questions.
Design a basic evaluation study.
Use generative AI tools to support the research process, including literature review, preliminary data analysis, and creating communication products.
Critically assess AI-generated outputs and identify associated risks.
SOCIAL COMPETENCES: The student is prepared to:
Work in a team to prepare a complex analytical project.
Adopt an evidence-based approach in assessing public sector activities.
Critically reflect on the ethical aspects of using behavioral insights and AI tools in shaping public policies.
Assessment criteria
Assessment Method: Graded course.
Methods of Verifying Learning Outcomes:
Completion of preparatory assignments for classes.
Active participation in class discussions and workshop exercises.
Preparation and presentation of a group-based final project.
Assessment Criteria:
The final grade is a sum of scores from three components:
Final Project (1-2 persons) - 60% of the final grade.
Task and structure remain as in the previous proposal, including designing a full evaluation plan for a selected policy, with a mandatory section on the use of GenAI.
Preparation for and Participation in Class - 40% of the final grade.
This component consists of two parts:
Preparatory Assignments (20% of the final grade):
Before most workshops, students will be required to complete a short preparatory task (e.g., reading a text, reviewing an interactive material or video).
Completion will be verified by filling out a short online form (e.g., Google Forms) with a few questions checking comprehension of the material.
Scoring for preparatory assignments will be as follows:
+1 point: for completing the form before the relevant class begins.
+0.5 points: for completing it during or up to 24 hours after the class.
0 points: for not completing the form or for late submission (later than 24h), which negatively impacts the final grade through loss of points.
In-class Participation (20% of the final grade):
Includes substantive contributions to discussions, practical exercises, and group work. The quality of input (based on preparation), argumentation skills, and constructive collaboration will be assessed.
Conditions for Passing the Course:
To pass the course, all of the following conditions must be met:
Receiving a passing grade for the final project.
Completing and submitting on time at least half (50%) of the preparatory assignments. Failure to meet this condition will result in failing the course, regardless of the grade for the final project.
Attendance in class (in accordance with the university's study regulations).
Practical placement
-
Bibliography
Olejniczak, K., Batorski, D., Pokorski, J. (Eds.). (2025). Generative AI in research. Practical applications in public policy evaluation. Warsaw: Polish Agency for Enterprise Development.
Haber, A., & Olejniczak, K. (Eds.). (2014). (R)ewaluacja 2: Wiedza w działaniu [ (R)evaluation 2: Knowledge in Action]. Warsaw: Polish Agency for Enterprise Development.
Varone, F., Jacob, S., & Bundi, P. (Eds.). (2023). Handbook of public policy evaluation. Edward Elgar Publishing.
Rossi, P. H., Lipsey, M. W., & Henry, G. T. (2019). Evaluation: A Systematic Approach. SAGE Publications.
Bardach, E., & Patashnik, E. M. (2020). A Practical Guide for Policy Analysis: The Eightfold Path to More Effective Problem Solving. CQ Press. (as a reference point).
Additional information
Additional information (registration calendar, class conductors, localization and schedules of classes), might be available in the USOSweb system: