Позитивные изменения. Том 2, № 2 (2022). Positive changes. Volume 2, Issue 2 (2022)

Step 3 — Form the treatment and comparison groups from the units in the evaluation sample through randomized assignment.

Figure 4 shows the main steps of successfully implementing the randomized assignment method.

Once the above steps are completed, what remains is relatively simple. Once the program has run for some time, outcomes for both the treatment and comparison units will need to be measured. The impact of the program is simply the difference between the average outcome (Y) for the treatment group and the average outcome (Y) for the comparison group.

Randomized assignment is the most reliable method of evaluating counterfactual data and, to a certain extent, the gold standard in the field of impact evaluation.

ESTIMATING THE IMPACT OF HISP: RANDOMIZED ASSIGNMENT

Let us now turn back to the estimation of the HISP pilot that involves 100 treatment villages.

Having conducted two impact evaluations using potentially biased counterfactuals (as mentioned above), the project team decided to obtain a more precise estimate — using randomized assignment. It was determined that building a valid estimate of the counterfactual will require identifying a group of villages that are identical to the 100 treatment villages in all respects. Since the 100 treatment villages were selected for HISP randomly from among all of the rural villages in the country, the treatment villages had the same characteristics as the general population of rural villages. All that was left to be done was to evaluate the difference between these two groups. Thus, data was collected on another 100 villages that were left out of the program.

Table 2 shows the average health expenditures of households in the comparison and treatment groups according to the same criteria. The pre-intervention average health expenditures of households in the two groups do not statistically differ, which is what’s expected with randomized assignment. Mathematical analysis showed that the outcome of the intervention was a reduction in household expenditures by USD 10.14 over two years.

Note: Significance level: ** = 1 percent.

Note: Significance level: ** = 1 percent.

Source: Paul J. Gertler, Sebastian Martinez, Patrick Premand, Laura B. Rawlings, and Christel M. J. Vermeersch. Impact Evaluation in Practice Second Edition. — International Bank for Reconstruction and Development / The World Bank, 2016

Summing up, we would like to say that the use of rigorous evaluation methods and the regular collection and monitoring of data about a project or program represents the main set of tools that the parties involved can use to verify and improve the effectiveness and efficiency of social projects and programs at various implementation stages.

REFERENCES:

11. Paul J. Gertler, Sebastian Martinez, Patrick Premand, Laura B. Rawlings, and Christel M. J. Vermeersch. Impact Evaluation in Practice Second Edition. — International Bank for Reconstruction and Development / the World Bank, 2016

2. Imbens, Guido W., and Donald B. Rubin. Rubin Causal Model. In the New Palgrave Dictionary of Economics, second edition, edited by Steven N. Durlauf and Lawrence E. Blume. Palgrave, 2008.

3. Rubin, Donald B. Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies. Journal of Educational Psychology 66 (5): 688–701, 1974.