Evaluation of Labour Market Projects 2103-ORP-L-D5EPRP

1. Introduction. Organisational activities.
2. Discussion of basic concepts related to the evaluation process.

Basic reading:
K. Olejniczak, Introduction to issues of evaluation, in: K. Olejniczak, M. Kozak, B. Ledzion (ed.), Teoria i praktyka ewaluacji interwencji publicznych. Podręcznik akademicki, Wydawnictwa Akademickie i Profesjonalne, Warszawa 2008, p. 17-39.
http://rszarf.ips.uw.edu.pl/ewalps/teksty/olejniczak_podrecznik.pdf

Additional reading:
R. Szarfenberg, Ewaluacja i analiza polityki społecznej, in: G. Firlit-Fesnak, M. Szylko-Skoczny (ed.), Polityka społeczna. Podręcznik akademicki, Wydawnictwo Naukowe PWN, Warszawa 2007, p. 127-141, http://rszarf.ips.uw.edu.pl/pdf/ewaluacja_analiza.pdf.
L. Korporowicz, The changing fate of Polish evaluation. Between hope, animation and confusion, in: Ewaluacja w edukacji, B. Niemierko (ed.), PTDE, Kraków 2011,

http://pte.org.pl/wp-content/uploads/2018/05/ZMIENNE-LOSY-POLSKIEJ-EWALUACJI-Krak%C3%B3w-22-wrze%C5%9Bnia-2011-OKE.pdf

3 Evaluation criteria. Formulation of evaluation questions according to the adopted criteria.

4. division into research teams. Discussing the rules of group work. For the duration of the next class, the workshop group turns into an Evaluation Research Centre consisting of several independent research teams. The students' task is to form a research team (2-3 persons) and to pre-define the evaluation report that will be evaluated during the next class and to indicate the project/activity that the evaluation research plan will concern.
The teams present information on the project they will evaluate and a preliminary list of evaluation questions according to the adopted criteria.

The evaluation report and the subject of the evaluation may be defined as a result of the research team's contact with a public institution/non-governmental organisation/company which is willing to cooperate in a mutually beneficial way in the evaluation of its selected activities (labour market projects).

Alternatively, the subject of the evaluation may be elements of the labour market policy of the state (e.g. specific statutory solutions, strategic documents, programmes, activities) or local government institutions.

Basic literature:
Ch. Frankfort-Nachmias, D. Nachmias, Research methods in social sciences, Zysk -i Ska Wydawnictwo, Poznań 2001, Appendix B. Principles of writing a research report, pp. 568-578;
E. Babbie, Social research in practice, Wydawnictwo Naukowe PWN, Warsaw 2005, Appendix C, Research report, pp. 563-569.
Evaluation. Guidebook for public administration employees, Ministry of Regional Development, Warsaw 2012, https://www.ewaluacja.gov.pl/media/11102/poradnik_ewaluacji.pdf.

Additional reading:
Efektywność podstawowych form aktywizacji zawodowej, realizowanych w ramach programów na rzecz promocji zatrudnienia, łagodzenia skutków bezrobocia i aktywizacji zawodowej w 2017 roku, Ministerstwo Rodziny, Pracy i Polityki Społecznej, Warszawa, September 2018.
https://www.gov.pl/documents/1048151/1060973/Efektywność_podstawowych_form_aktywizacji_zawodowej_w_2017_r.pdf/635b73c2-7aa0-d24c-399e-86cd5b2cec97

5 Evaluation as a research process.

Before class, each team conducts a search to find any three evaluation reports of labour market activities or projects. These do not have to be model reports, it is only important that they are published (if only on the Internet) evaluation reports on labour market issues. Proposals for these reports should be sent to the tutor at m.oldak@uw.edu.pl before the next class. From these three proposals, the team identifies one which will serve as the basis for the preparation of the analyses that are the subject of the class.

6-7 Qualitative and quantitative research in evaluation. Basic research techniques in evaluation
Basic reading:
S. Mandes, Qualitative methods in evaluation; Ł. Widła, Application of quantitative methods in evaluation, in: K. Olejniczak, M. Kozak, B. Ledzion (ed.), Teoria i praktyka ewaluacji interwencji publicznych. Podręcznik akademicki, Wydawnictwa Akademickie i Profesjonalne, Warszawa 2008, p. 129-174. http://rszarf.ips.uw.edu.pl/ewalps/teksty/olejniczak_podrecznik.pdf.

8 Specificity of evaluation research and indicators in evaluation.

Basic reading:
K. Olejniczak Ewaluacja jako proces badawczy, J. Górniak, K. Keler, Rola systemów wskaźników w ewaluacji, in: K. Olejniczak, M. Kozak, B. Ledzion (ed.), Teoria i praktyka ewaluacji interwencji publicznych. Podręcznik akademicki, Wydawnictwa Akademickie i Profesjonalne, Warszawa 2008, p. 101-126. http://rszarf.ips.uw.edu.pl/ewalps/teksty/olejniczak_podrecznik.pdf.

9-10. analysis and evaluation of evaluation research reports.

11. identifying system needs, target groups and participants' needs, as well as the objectives of the projects to be evaluated.
1) Teams present an initial concept of their own evaluation design for the evaluation report.
2) Teams prepare a presentation on a selected evaluation report from the labour market project - a so-called meta-evaluation. In the presentation, they include a description of the methods, the main results of the evaluation and their own methodological conclusions (what is worth including in their own study, what was missing). The presentation is intended to be a training presentation, i.e. the authors treat the other participants in the class as participants in a training course using the case study method and prepare their presentation with this in mind.

Core reading:
Evaluation. Guidebook for Public Administration Employees, Ministry of Regional Development, Warsaw 2012, https://www.ewaluacja.gov.pl/media/11102/poradnik_ewaluacji.pdf.

12 Evaluation standards.

Evaluation standards of the Polish Evaluation Society,
http://pte.org.pl/wp-content/uploads/2015/08/Standardy_ewaluacji_PTE.pdf

13-14 Teams present the concept of their own evaluation study

Teams submit concepts for their own evaluation project (in the form of a multimedia presentation and Word file).

Teams submit and present concepts for their own evaluation design (in the form of a multimedia presentation and Word file). Please note that we treat these concepts as research proposals submitted to a potential entity that has announced a competition for the implementation of an evaluation study. Therefore, care should be taken to ensure both the substantive level of the concept and the aesthetic level of the materials submitted, as well as the attractiveness of the message.
The aim of this seminar is to convey synthetically and concisely to the contracting authority the most important elements (stages) of our study, together with justification for the techniques, indicators, research questions, etc. adopted.

15 Written test.

NOTE: One topic may take more than a single class. In turn, another may be discussed along with the preceding or following issue. Details in this regard will be communicated to students well in advance.