Позитивные изменения. Том 1, №1 (2021). Positive changes. Volume 1, Issue 1 (2021)

Kuzmin notes that “in social design, outputs are the direct consequences of a series of actions within a project. It would seem that everything is simple: if you performed an action, you got an output. However, strange as it may seem, in both foreign and Russian publications, opinions on what should be considered an immediate result (output) differ radically. And it seems that our erroneous position is becoming more and more popular. It consists in the fact that the characteristics of the activity performed are referred to the immediate results. The author gives the following example: “Suppose a project is training. Training itself is a project activity. Nobody has any questions here. Now attention! The immediate results of training are called the following: how many training courses were organized, how many hours the classes lasted, how many people took part in the training. But all of the above refers to the activity itself, describes its scale, and answers the question of how much we have done. All these are characteristics (indicators) of the activity performed, but in no way its immediate results. What is the immediate result of training, its direct consequence? What happens to the trainees: changes in their knowledge, skills, and attitudes towards something?"

It is worth noting that the above-described problem of interpreting the concept of immediate results in terms of activity characteristics is quite common. Moreover, we can say that most of the sources considered during this analysis state that immediate results are understood exactly as activity characteristics.

Thus, the recommended position on this issue is the one that takes into account, on the one hand, the need to assess the results at the level of correlation with the stated objectives of the project, but on the other hand includes a component of the scale of these results, and therefore, an indication of the characteristics of the activity itself (the number of services provided, the audience covered, etc.).

If we try to analyze the present day Russian representations of key concepts related to valuation, it is worth quoting the position of experts from one of the associations of evaluation market players actively working towards the development of systemic solutions in the field of state programs and projects — the Accounts Chamber of the Russian Federation, as well as the Center for Advanced Management Decisions. In the manual on the theory of change, prepared in partnership by experts of these organizations, the following definitions are given [A. I. Kim et al., 2020]:

• immediate results (outputs) — specific products formed as a result of direct state influence and possible for use by target audiences (charity recipients);

• response (react) — the reaction of charity recipients to the use of immediate results (including in the form of behavior change);

• final results (outcomes) — a set of significant changes that the charity recipients experience after the use of immediate results;

• final impacts — desired medium- and long-term social and economic changes.

Outputs are the results that organizations can directly measure or evaluate, while outcomes are broader changes, benefits, and knowledge that can be gained in the medium to long term.

We cannot but mention in this context the approach of the Global Impact Investing Network (GIIN) in connection with the publication by this organization in November 2020 of a new evaluation methodology, presented as the first of its kind recommendation that allows the use of unified approaches and comparison of any projects with each other (Methodology for Standardizing and Comparing Impact Performance). It is worth noting, however, that this methodology is applicable to use by social investors, in order to select investment projects. Within this methodological guide, it is proposed to distinguish the following levels of evaluation: “outputs — short term outcomes — long terms outcomes". Outcomes analysis within this methodology should be built at several levels:

• Scale: number/coverage of stakeholders who received a result (outcome).

• Depth of impact: the degree of change obtained by stakeholders.

• Exposure time: the period of time during which the stakeholder continues to experience the result.

• Volatility: the degree of variability (change) of results over time.

Table 2. Basic terms and definitions related to impact and evaluation