Insert 1: RANDOMIZED CONTROLLED TRIALS AS A VALUABLE OPERATIONAL TOOL
Randomized assignment can be a useful rule for assigning program benefits, even outside the context of an impact evaluation. The following two cases from Africa illustrate how.
In Côte d'Ivoire, following a period of crisis, the government introduced a temporary employment program that was initially targeted at former combatants and later expanded to youth more generally. The program provided youth with short-term employment opportunities, mostly to clean or rehabilitate roads through the National Roads Agency. Young people in participating municipalities were invited to register. Given the attractiveness of the benefits, many more candidates applied than there were places available. In order to come up with a transparent and fair way of allocating the benefits among applicants, program implementers put in place a public lottery process. Once registration had closed and the number of applicants (say, N) in a location was known, a public lottery was organized. All candidates were invited to a public location, and small pieces of paper with numbers from 1 to N were put in a box. Applicants were then called one by one to come to draw a number from the box in front of all other candidates. Once the number was drawn, it was read aloud. After all applicants were called, someone would check the remaining numbers in the box one by one to ensure they match the applicants who did not turn up for the draw. If N spots were available for the program, the applicants having drawn the lowest numbers were selected for the program. The draw was organized separately for men and women. The public lottery process was well accepted by participants, and helped provide an image of fairness and transparency to the program in a post-conflict environment marked by social tensions. After several years of operations, researchers used this allocation rule, already integrated in the program, to conduct impact evaluation.
In Niger, the government started to roll out a national safety net project in 2011 with support from the World Bank. Niger is one of the poorest countries in the world, and the population of poor households eligible for the program greatly exceeded the available benefits during the first years of operation. Program implementers relied on geographical targeting to identify the departments and communes where the cash transfer program would be implemented first. This was feasible, as data was available to identify relative poverty or vulnerability status of the various departments or communes. However, within communes, very limited number of people could enroll in the program based on objective criteria. For the first phase of the project, program implementers decided to use public lotteries to select beneficiary villages within targeted communes. This decision was made in part because the available data to prioritize villages objectively was limited, and in part because an impact evaluation was being embedded in the project. For the public lotteries, all the village chiefs were invited in the municipal center, and the names of their villages were written on a piece of paper and put in a box. A child would then randomly draw beneficiary villages from the box until the quotas were filled. The procedure was undertaken separately for sedentary and nomadic villages to ensure representation of each group. After villages were selected, a separate household-level targeting mechanism was implemented to identify the poorest households, which were later enrolled as beneficiaries. The transparency and fairness of the public lottery was greatly appreciated by the village authorities, as well as by program implementers — so much so that the public lottery process continued to be used in the second and third cycle of the project to select over 1,000 villages throughout the country. Even though public lottery was not necessary for an impact evaluation at that point, its value as a transparent, fair, and widely accepted operational tool to allocate benefits among equally deserving populations justified its continued use in the eyes of program implementers and local authorities.
Sources: Bertrand, Marianne, Bruno Crépon, Alicia Marguerie, and Patrick Premand. Impacts à Court et Moyen Terme sur les Jeunes des Travaux à Haute Intensité de Main d’oeuvre (THIMO): Résultats de l’évaluation d’impact de la composante THIMO du Projet Emploi Jeunes et Développement des compétence (PEJEDEC) en Côte d’Ivoire. Washington, DC: Banque Mondiale et Abidjan, BCPEmploi. 2016Premand, Patrick, Oumar Barry, and Marc Smitz. "Transferts monétaires, valeur ajoutée de mesures d’accompagnement comportemental, et développement de la petite enfance au Niger. Rapport descriptif de l’évaluation d’impact à court terme du Projet Filets Sociaux." Washington, DC: Banque Mondiale. 2016.
