Tagged: Eval13

Are enabling environments in evaluation actually disabling?

This question is one that has been lingering in the weeks after AEA’s Evaluation 2013 Conference. First, let me just say that my debut experience at a national AEA conference was spectacular. I’ve been to my fair share of conferences over the years, and AEA outshined them all. The variety of sessions ensured that there truly was something for everybody (a difficult feat with 3,000+ attendees all at different stages in their evaluation careers!) There was an impressive sense of community and attendees were engaged nonstop in exchanging ideas and learning from one another.

I spent a lot of my time focusing on sessions related to evaluation capacity building and development. As an internal evaluator / M&E officer, I help program coordinators feel comfortable with M&E concepts, adopt them into their work, and advocate for their use over the long-term. My job is not to “do all the M&E,” but rather make sure that M&E “becomes a way of doing.” Evaluation capacity building and development provides great frameworks for achieving these goals within the organization. It also helps me think about how I can contribute to strengthening the practice of evaluation on a larger scale.

I started off the week with Michele Tarsilla’s full-day workshop on Evaluation Capacity Development 101. His workshop emphasized capacity as both a latent attribute and a potential state, something that already exists within individuals, organizations, and institutions, but which can be developed over time. This process requires an understanding of the level of capacity that already exists. As any qualified educator would confirm, it’s essential to recognize that learners already possess significant background knowledge. The job of the educator is to activate and build off that knowledge, which in combination with new information leads to increased levels of knowledge, skills, abilities, dispositions, etc. Tarsilla’s distinction between capacity building (short-term, activity-focused) and capacity development (long-term, process-focused) takes this concept of prior background knowledge and existing capacity into account. Although some might see it as simple semantics, language does matter, not only in how we frame conversations but also in how we execute our work.

Several other evaluation capacity building and development sessions I attended (including from the World Bank’s CLEAR Initiative), emphasized the importance of creating an enabling environment for evaluation. This terminology is common in the field, but can be a bit opaque at first glance. To put it simply, an enabling environment is a context that facilitates the practice of evaluation (it enables effective evaluation to take place). There are a lot of factors that make up such an environment, but examples might be demand by policymakers for the production of evidence, funding available for evaluations, widespread interest in and use of evaluation findings, existence of evaluation policies, etc. EvalPartners is doing great work globally to create more enabling environments for evaluation. They’ve successfully declared 2015 as the International Year of Evaluation (EvalYear) as part of this effort!

Enabling environments were on my mind when I participated in a new AEA session type, Birds of a Feather Roundtable Sessions. These lunchtime gatherings brought together diverse participants to talk about a shared area of interest. I grabbed takeout at the local deli and joined a session on international development, where we chatted about a variety of issues related to evaluation in the sector. One of the questions posed to the group dealt with the introduction of new methods that could effectively address issues of complexity (Here are some great posts that can catch you up on that debate: Complexity and the Future of Aid and Complexity 101 Part 1 and Part 2).

I immediately responded to this question with some doubt. I think the methods exist (or are being developed). There’s a lot of talk about developmental evaluation, systems thinking in evaluation, etc. It’s not uncharted territory methodologically speaking. But it is, perhaps, uncharted territory politically speaking. In other words, we are having a conversation about methods when we really need to be talking about the enabling environment. Does the evaluation environment in international development allow evaluators and practitioners to use these methodologies in their evaluation work? Or is the enabling environment actually disabling? Are evaluation practices in international development confined to certain types of evaluations? Yes, we need to continually innovate with methods. But those methods will never get used if the environment doesn’t allow for it.

I strongly believe that evaluators should not only actively engage in dialogue with other evaluators about the state of evaluation, but they should also advocate for enabling environments that allow the nuanced and specialized work that evaluation requires.  We should be active contributors both in shaping the field itself and shaping how others engage with it. It’s certainly possible (and, perhaps, common) to create an evaluation-friendly environment for certain types of evaluations, but not for others. Efforts to strengthen enabling environments should maintain a holistic perspective that allows for multiple types of evaluations, a diversity of perspectives, and innovations in the field. My hope is that, as more and more countries develop rich evaluation practices, the unique perspectives that each country brings to the table will enrich the conversation on evaluation itself. The worst thing would be to superficially limit the conversation by fostering the creation of evaluation policies, practices, etc. that operate underneath a narrow definition of evaluation. Let’s allow diversity in background knowledge to create diversity in outcomes for the field. The more minds we have thinking differently about evaluation, the more innovations will be introduced. We just need to make sure that enabling environments are built flexibly to accept and foster those innovations.

What unique contributions have you seen to the field of evaluation? Have you seen enabling environments actually “disable” the work of evaluation? How so?