Позитивные изменения. Том 1, №1 (2021). Positive changes. Volume 1, Issue 1 (2021)

Natalia Gladkikh,

Ph.D. in Psychology, Leading Expert of the Institute of Socio-Economic Design, Higher School of Economics

Zoya Talitskaya,

Expert, Gladway Foundation for the Development of Media Projects and Social Programs

The development of methodology and practical application of evaluation tools in the field of social projects and programs in Russia began more than 15 years ago. Since then quite a number of Russian-language manuals and methodological manuals for social projects evaluation have emerged created, digests of international publications on this topic have been published, several specialized organizations engaged in the evaluation of projects and programs have begun working, and the Association of Evaluation Specialists (AES) forming a community of specialists in the field of evaluation. In general, evaluation of projects and programs in Russia is an actively developing field of scientific and practical activity, and it has acquired a fairly long history and valuable experience.

An indicator of such experience is the thesaurus that has been formed for this subject area. It is available at least within the professional community of evaluation specialists. For example, the terms as “the Theory of Change”, “the Project Logical Model”, “the separation of categories of results and the impact of projects and programs” are established and shared by most specialists.

However, a number of difficulties is associated with defining and understanding basic terms. This is largely due to the fact that the language corpus of this branch of knowledge was originally borrowed from English and is often used as a language calque. For example, the terms “impact” and “assessment” are often used as direct transfer from English, and their definitions therefore require closer attention.

Let’s try to conduct a comparative analysis of the key terms included in the language corpus of the subject area of projects and programs evaluation.

BASIC CONCEPTS USED IN SOCIAL PROJECTS EVALUATION

The basic concept used in evaluating is expected to be the term “Evaluation" used in Russia in the direct meaning of its translation: “determination of value". One of the most popular and basic definitions of evaluation is the following definition proposed by A. I. Kuzmin [Kuzmin, 2009], based on the translation of a dictionary article by Michael Patton in 1997: “Program evaluation is a systematic collection of information about the activities of a program, its characteristics and results, which is carried out in order to make a judgment about the program, increase the effectiveness of the program and/or develop plans for the future." Note that in the context of this definition, the terms “program" and “project" could be considered synonymous, and thus it can be assigned to both categories.

Another common concept is “Monitoring", the meaning of which is also quite clearly defined in the Russian literature. Referring to one of the most classical definitions, also proposed by A. I. Kuzmin [Kuzmin, 2009], it is possible to define monitoring as “one of the types of program evaluation, which consists in constantly monitoring the progress of the program and comparing it with the plan… Monitoring of in-depth analysis does not imply, but provides operational information about the current status on the basis of comparing the actual values of a number of key figures (indicators) with their planned values."

The concept of “Assessment" is no less important, but the matter of its translation is more complicated. If you look at the second definition of this concept in the dictionary, you can designate its content as “assessment, determination of value", which does not give us grounds to determine its specificity in comparison with the previous one. However, the third definition is “definition, assessment (of viability)" — still allows you to determine this specificity. So, the main difference between this concept and the concept of “Evaluation" is the predictive nature of assessment — conditionally, you can characterize it as a “predictive" or “predictive" assessment, i.e. assessment of the present activity in the context of forecasting the development of this activity in the future, as well as forecasting the results of this activity. It is important to note here that for the second case there is an even more specific concept of “Impact Assessment", which can be translated into Russian as “прогнозирование влияния" (impact forecasting).