Позитивные изменения, Том 3 №1, 2023. Positive changes. Volume 3, Issue 1 (2023)

If we take Canada as an example, does this country have a higher education program (such as a master’s degree), or is it a certification program run by a professional association? What is the process for obtaining a certificate?

The Canadian Evaluation Society organizes trainings. People attend the trainings, complete the curriculum, and present examples of past evaluation work. By acquiring certification from the Canadian Evaluation Society, they become certified evaluators.

There is no “gold standard” for evaluation. The main challenge is adaptability, appropriateness, also the program needs are the key.

In your opinion, when a customer decides whether a particular evaluator is the best person for their particular project, which option is better – certification or no certification?

I think having a certificate helps, because it allows customers, who may not be very proficient in evaluation, to understand they are dealing with someone who has been trained. But the field of evaluation in the United States and in Europe is so diverse that it has not been possible so far to achieve a common understanding of the basics of this profession. In Canada, we were able to agree on the basic concepts that an evaluator must be familiar with, that is why certification is possible there. It is unlikely to happen in the U. S. because the field is too diverse there.

Let’s talk about evaluation methodology. Have there been any changes in this area?

Of course. Artificial intelligence (AI), big data and its sources have emerged. The pandemic brought forth great changes: online services have enabled conducting interviews and focus groups remotely. Satellite imagery has been used in new ways.

For example, thanks to satellites, it is no longer necessary to go somewhere in the field to check whether new agricultural equipment is being used there. If we want to know how many children attend a school in a developing country, we can use satellite imagery again.

Remote data collection, AI, big data, the Internet, and new data visualization techniques allow for better communication. It has become easier to distribute evaluation results thanks to the development of the Internet and social media, to find people to interview, to follow a person on social networks. All of these technological advances are making their way into the field of evaluation.

So many changes and so many possibilities. Does every evaluator need to be familiar with all these new products? What are the most important qualities and requirements for a good evaluator, and can we talk about the “gold standard” of an evaluator?

The way I see it, we have defined the “gold standard” as the ability to evaluate a program in the proper way, which meets the needs of that specific program. Methodologically, there is no “gold standard.” In any case, I do not believe that randomized controlled trials are better than other methodologies, because the program needs are the key, and because such phenomena as the pandemic, climate change, global political instability, growing numbers of refugees, famine and drought, the speed of change in the world around us, create the need for evaluation approaches that can adapt quickly to new conditions, similar to those caused by the pandemic.

Programs need to change, and so does evaluation. There is no single method that would represent the “gold standard.” The main challenge and difficulty is adaptability and appropriateness. You need to understand what a particular program needs at a particular point in time.

If an evaluation involves, for example, the use of advanced sociological tools, which I am not familiar with as an evaluator, I would not be able to conduct an evaluation at the proper level, would I?