Chapters of the book Transformational Evaluation for the Global Crises of Our Times can be downloaded from this site. The full and complete book can be downloaded on the main page. For individual chapters please scroll down to the chapter of your choice.
Section 1: Setting the Stage
1. Transforming Evaluation for Transformational Changes
Cristina Magro, Rob D. van den Berg and Marie-Hélène Adrien
This introductory chapter sets the stage for the discussions that are presented in the various chapters to follow, placing them in the context of the enormous challenges humanity is facing and in the UN Agenda and the Paris Agreement where evaluation is tasked to bring rigorous evidence to support transformation. The chapter introduces an operational systemic way of understanding “transformation”, offering the readers a dynamic view of the concept and stimulating them to go through the book expanding their reflections. It draws the path the book follows, presenting each chapter in the context of the four parts of the book: the first part sets the stage for the grand challenges we are facing. The second part delivers four chapters on practical experiences of evaluators with transformational evaluations. The third part raises issues of professionalization, including perspectives of young and emerging evaluators, new initiatives like the International Evaluation Academy, and deep and broad reflections on professionalization in evaluation. The fourth part goes into themes and cases, and the fifth presents approaches and methods. The book ends with a special section on the Prague Declaration of 4 October 2019.
2. Blue Marble Evaluation Perspective: How Evaluations Help Solve Global Crises
Michael Quinn Patton
Given the global climate emergency and related threats to a just and sustainable world, systems transformation is the clarion call of our times. Evaluators enter the fray to assess the fidelity and impacts of hypothesized transformational initiatives and trajectories. Doing so requires solid ethical grounding. The ethics of transformation involves the interconnection between personal ethics (transforming our own behaviours), professional ethics (actively advocating a transformational stance among professional evaluators), society (examining evaluation’s role in support of the public good and democratic processes) and the world (ensuring attention to and engagement with the global emergency by incorporating transformational criteria of equity and sustainability into all evaluations). This chapter examines the implications of transformative ethics for evaluation theory, practice and methods, concluding with Blue Marble Evaluation as a principles-focused approach to evaluating global systems transformation.
3. Evaluation in an Uncertain World: A View from the Global South
Adeline Sibanda and Zenda Ofir
The wicked and intersecting challenges facing the world, brought about by the multiple shocks of COVID-19 and the accelerating impacts of climate change and other effects of the Anthropocene, require that the global evaluation community think and work in fundamentally new ways. It is time that the potential of the Global South – in which we include minority Indigenous societies around the world – is realized, not only because it is most vulnerable to the irresponsible behaviour of societies worldwide, but also because it has important strengths that can help chart a new path for the transformative and sustainable development the world urgently needs. Evaluation will be key in supporting the drastic, much-needed changes. Although exploitation in many different forms persists around the world, the Global South must address the consequences of centuries of colonization, as well as ongoing imperialism and engineered power asymmetries in global governance, economic and financial systems that continue to favour the economically rich Global North. It has become essential that all those who shape and work in the global evaluation system collectively ask how decolonization of the mind and of practice in both the Global South and Global North can be achieved. This is necessary to respect and appropriately attend to the conceptual and methodological experiences, knowledge and wisdom that are deeply embedded in the many diverse cultures of the Global South and that can help advance evaluation in support of systems change and transformation.
Section 2: Experiences with Transformational Change Evaluation
4. Ensuring Transformational Change for Climate Action
Matthew Savage, Tim Larson, Jessica Kyle and Sam McPherson
This chapter sets out lessons learned and insights into transformational change arising from an evaluation of the Climate Investment Funds (CIF). It draws upon work undertaken during an independent evaluation of transformational change in the CIF during 2018 and 2019 (Itad 2019) and work that the evaluation team supported through the Transformational Change Learning Partnership (TCLP) after the evaluation. The CIF commissioned the evaluation to explore to what extent CIF had supported transformational change across a range of climate change areas: supporting clean energy and reducing greenhouse gas emissions (mitigation); reducing systemic risk and creating greater resilience to the impacts of climate change (adaptation); and enabling investments in sustainable forestry and strengthening the role of climate action in addressing other areas such as gender equity. We describe the baseline thinking on transformational change in the CIF that underpinned the Itad evaluation, describe findings that arose from the evaluation, provide insight into further work on transformational change that the Itad team undertook as part of the TCLP process and identify areas for further consideration and development. This article builds upon recent analysis of TCLP concepts and learning (e.g. CIF 2021, Williams, Dickman and Smurthwaite 2020).
5. Evaluation at the Nexus between Nature and Humanity for Transformational Change
Juha I. Uitto
The global environmental crises being manifested through climate change and rapid loss of biodiversity require transformational change in major systems ranging from energy and transportation to agriculture and cities. The pandemic of 2020-21 has demonstrated the interdependence of human and ecosystem health. Evaluation can contribute significantly to identifying solutions for the future but, to do so, must rise above its focus on individual interventions in isolation of their context. Evaluators must also learn to operate in the nexus between human and natural systems, where sustainable development takes place. This chapter draws upon experiences with evaluating the work of the Global Environment Facility (GEF) in supporting adaptation to climate change, an area that by necessity transcends the boundaries of human and natural systems. The chapter also introduces a framework for evaluating the GEF’s additionality in six specific areas: environmental, legal and regulatory, institutional and governance, financial, socioeconomic and innovation.
6. Contradictions and Complementarities between South and North on Transformation in the Anthropocene
Mehjabeen Abidi-Habib, Jane Burt, John Colvin, Chimwemwe Msukwa, and Mutizwa Mukute
Emerald Network is an emerging community of evaluation and learning praxis working in the field of international cooperation and development and with Global South consultancy partners. Our evaluation and learning praxis draws on our combined experience in policymaking, design, strategy, finance, implementation and research. Recognizing that we are living through the early Anthropocene – or Capitalocene to be more precise – we seek to contribute to transformative development pathways in service to a just, regenerative, low-carbon, resilient world. In this chapter, we reflect on how our praxis has evolved over the past eight years, sharing stories of success and failure and what we have learned in the service of transformational work. The purpose of this chapter is to explore the role of evaluation praxis in transformational design for sustainable development, focusing on a number of themes that have come to play a central role in our praxis. These include navigating and learning through contradictions and complementarities between Global South and Global North, the centrality of navigating power in these contradictions and complementarities, the value of understanding history and context, the importance of internal praxis and the design and facilitation of adaptive and potentially transformational learning processes.
7. Improved Governance Pathways for the Greater Caribbean: Transformative Evaluation Principles
Lennise J.C. Baptiste
This chapter was developed from a thematic review of publicized statements of stakeholders of the Association of Caribbean States (2017) to understand their priorities for post-COVID-19 recovery in the region. The analysis showed that regional priorities were to improve health systems, ensure food security, improve transportation channels to access supply chains for medicine and food, develop partnerships to leverage economies of scale, preserve the environment and develop the economy. The COVID-19 pandemic has illuminated the gaps in governance systems that were designed to keep citizens safe and provide relief in times of crisis. In this chapter, the governance systems of countries in the Greater Caribbean were examined to identify how transformation change practices could help in the crisis management and recovery phases. Transformative evaluation practices and establishment of internal monitoring and evaluation systems were proposed to increase demand for evaluation to support decision-making and build an evaluation culture. Capacity building, strategic planning, policy development and use of information and communications technology were identified as transformation pathways for the region.
Section 3: Professionalisation
8. To Be or Not to Be an Evaluator for Transformational Change: Perspectives from the Global South
Pablo Rodriguez-Bilella, Silvia Salinas-Mulder, and Sonal Zaveri
In the current global neoliberal context, evaluation runs the risk of becoming another service that gives answers wanted by those who pay for it. Being a transformative evaluator entails extending the focus of action to contribute to public good, broadening its interest towards medium- and long-term results, and investigating the root causes of those social problems that programmes and policies aim to deal with. This chapter introduces a theoretical framework on transformative evaluation based on theory and practice from the Global South. For that, it discusses a competencies profile for gender- transformative, context-relevant evaluations, a comprehensive approach built in Latin America. Then, selected cases are presented to identify the factors and evaluator competencies that facilitate usable evaluation and evaluations aimed at social betterment. The last section discusses the complexities underlying frequently invisible power issues and relations and the need to fine-tune one’s ability to identify and address them in evaluations. The chapter stresses the importance of redefining the role and competencies needed to enhance the transformative potential of evaluators, ensuring gender responsiveness and power awareness under culturally diverse and complex realities, identifying evidence-based strategies and actions to conduct evaluations that have a positive impact on people’s lives.
9. Avenues for Young and Emerging Evaluators’ Journeys to 2030
Kenza Bennani, Marie-Hélène Adrien and Gerardo Sánchez-Romero
The Decade of Action offers young and emerging evaluators (YEEs) an opportunity to embark on transformative professional journeys to 2030. In this chapter, we share concrete avenues to help them conceptualize their career trajectories. We rely on the concept of ‘professional identity work’ to present a framework that defines and differentiates the various types of evaluator identities that YEEs could explore – through formal employment engagement or involvement with voluntary organizations for professional evaluation (VOPEs) and YEE networks. We posit that, in the coming decade, VOPEs and YEE networks could be considered as ‘identity workspaces’ that support YEEs in discovering, understanding and shaping who they are and can become in the era of the Sustainable Development Goals – as transformational evaluators in the making.
Chapter 9 in French
10. Towards an International Evaluation Academy as an Agent of Transformational Change
Linda G. Morra Imas
For decades, the value of evaluation professionalization has been debated. A prolific evaluation literature is now available. This chapter puts forward a transformational concept designed in part to promote evaluation professionalization: an international evaluation academy (IEAc). The 2019 International Development Evaluation Association Global Assembly, held in Prague, culminated in approval of a declaration that supported exploration of an IEAc initiative to act as a platform for innovation, creativity and collaboration in pursuit of evaluation professionalism and influence. This chapter summarizes the pros and cons of professionalization; examines responses to an international survey that confirm broad-based support for the IEAc concept, including a focus on professionalization and outlines what the IEAc is about and some ways it will address evaluation professionalization.
11. Evaluation for Transformation: What Will It Take?
Robert Picciotto
Given recurrent health emergencies, rapid environmental degradation, pervasive insecurities and the rising popular anger that the unmet promises of modernity in liberal and authoritarian regimes alike have triggered, populism is on the rise, the knowledge professions are threatened and social transformation is imperative. Thus, evaluation faces its own transformation challenge. New policy directions will be required to transform the enabling environment of evaluation practice. Specifically, the process that the neoliberal and evidence-based waves of evaluation diffusion induced, which transformed evaluation into a private good, must be reversed. For evaluation to restore its public good character, it must break the chains of the market-based governance model currently in place and increase its influence by moving up the occupational ladder. This implies acquiring all the interrelated features of professionalism: an ethical charter, expert knowledge, proven competencies and self-management. There is no shortcut.
Section 4: Themes and Cases
12. Multiple Dimensions of Evaluation in Fragility, Conflict and Violence
Inga-Lill Aronsson and Hur Hassnain
Evaluation in contexts affected by conflict and fragility is political and complex and can exacerbate violence. In such unpredictable environments, understanding how change happens is challenging because different actors at local, national and international levels have varied interests and definitions of what change is or may be. Reports on progress on the Millennium Development Goals (MDGs) show that these contexts lag in establishing robust monitoring and evaluation systems. Most examples of tools and resources available are from ‘normal’ contexts and do not fully support identification of the multiple biases at all levels during an evaluation exercise, including possible bias of the evaluator. In these contexts, collaboration with people on the ground is paramount to contextualize scenarios, tools and values in order to visualize what works or not for evaluation in a particular setting. The authors combine in this chapter background research completed, a reflection of their own personal experience, and the rich discussions with global evaluation practitioners who have worked in some of the worst conflict-affected contexts. During the IDEAS’s Global Assembly in Prague, the authors conducted a one-day workshop on evaluation in fragile, conflict and violent contexts was attended by participants from many countries representing a range of leading actors and organizations that enriched this urgent topic.
13. The Potential of Value Chain Development for Transformative Change: the Experiences of AfDB and IFAD
Fabrizio Felloni and Girma Kumbi
This chapter examines the contribution of the African Development Bank and International Fund for Agricultural Development to agriculture-related value chain development, based on evaluations that these organizations conducted. The chapter offers a systemic perspective from which to conceptualize value chains and value chain development for poverty reduction. If well designed and implemented, value chain support can lead to transformative changes for smallholder farmers and rural small-scale producers, but both evaluations conclude that working on value chains requires major changes in the organizational culture. This chapter emphasizes the importance of corporate-level strategies in creating consistency and guidance on value chains and thereby assisting with project design and implementation. Evaluation findings indicate that reaching impoverished rural farm households through value chain approaches requires specific attention. Having approached the topic of value chains from a system perspective, this chapter identifies five key fundamentals and enablers that characterize successful agricultural value chain development, highlights policy implications and makes key recommendations. It provides some lessons that will be relevant to future evaluations on this topic.
14. Evaluating under Fragility: Lessons from the Palestinian Context
Khaled Rajab
Like that of other countries, the Palestinian government committed itself to realizing a number of Sustainable Development Goals (SDGs) and has put in place a national mechanism for tracking and monitoring progress towards achieving them, including establishing institutional frameworks; engaging different stakeholders, including civil society, donors and the private sector; identifying indicators; collecting data and producing the Voluntary National Review Report. Nevertheless, the unique context of Palestine as a fragile, conflict-affected country poses a number of challenges, as it affects the ability of the national government and other societal actors to monitor achievement of the SDGs, to say nothing of actually achieving these goals. This chapter contributes to the discussion of the challenges of monitoring and evaluating SDGs in fragile contexts by focusing on the experience of Palestine. It builds on findings from literature review and interviews with relevant stakeholders, including government, civil society and international development partners that support Palestine in this effort, in particular the various United Nations agencies. It also presents and discusses key lessons from the Palestinian experience, especially for other countries under similar circumstances, including in the Middle East and North Africa.
Section 5: Approaches and Methods
15. Complex Systems, Development Trajectories and Theories of Change
Aaron E. Zazueta, Nima Bahramalian, Thuy Thu Le, Johannes Dobinger, Eko Ruddy Cahyadi
This chapter presents a case study of the application of Complex Adaptive Systems thinking to the planning, monitoring and evaluation of transformational interventions. The chapter presents a methodology to develop a robust understanding of the dynamics of the system targeted by a development intervention and to understand the ways in and extent to which a development intervention interacts to modify the development trajectory of such a system. It also describes the lessons learned in the ongoing learning process seeking to develop transformational theories of change. It draws on mixed methods that include different conceptual frameworks, analytical tools and information-gathering techniques. The approach we adopted is different from other systems thinking-inspired theories-of-change approaches in two ways. First, instead of focusing on the transformation of a system, our approach focuses on how to steer a system development trajectory that is consistent with a set of long-term objectives that are typically broadly articulated. The second important difference is that, unlike other systems-based theories of change, which often focus on transformation pathways that identify likely sequences of developmental stages (or conditions), our approach focuses on affecting the most influential conditions to steer the system development trajectory in the direction of the stated objectives. Our approach also focuses on monitoring the most effectual conditions to continually assess the extent and direction to which change takes place.
16. A Complexity-Based Metatheory of Change for Transformation toward Green Energy
Jonny Morell
This chapter draws from complexity science to present a meta-theory of transformation that can be applied to discrete theories of change constructed to guide model building, methodology and data interpretation for evaluation of change efforts. The focus is on six specific behaviours of complex systems – stigmergy, attractors, emergence, phase transition, self-organization and path dependence. These can be invoked singly or in combination to understand pattern, predictability and how change happens. The importance of both ‘explanation’ and ‘prediction’ is woven into the discussion. A definition of ‘transformation’ is offered in which a qualitatively new reality becomes the default choice that constitutes a new normal. Indicators of transformation include measurable ranges (as opposed to specific values) for level of energy use and the time over which the change endures. Because complex systems behave as they do, the recommended theory of change is sparse; it has few well-defined elements or relationships among those elements. There is already good progress in the application of complexity to the evaluation of transformation. An argument is made that these efforts should be strengthened by deliberately incorporating what is known about complex system behaviour, and that, by so doing, both prediction and explanation would better serve the purpose of practical decision-making.
17. Of Portals and Paradigms: Evaluation, Systems Thinking and the Pandemic
Cristina Magro and Rob D. van den Berg
This chapter proposes an exercise with systems thinking, taking the COVID-19 pandemic as a platform for learning, to illustrate the kind of reasoning, language and narrative that will help evaluators focus on key questions and approaches that are adequate. With this, the authors hope to help strengthen and spread the paradigm of systems thinking in evaluation. The authors argue that all social, economic, environmental, cultural and cognitive contexts are here to support evaluators dealing with systems thinking. After a presentation of systems thinking phenomena relevant for the exercise, the chapter takes readers on a journey through a broad, interrelated view of the experience with the pandemic and presents quick takeaways and consequences for evaluators and evaluation. Throughout the journey, the habits of a systems thinker are followed to gain insights and a natural flow of reasoning in systems terms.
Section 6: Building on the Prague Declaration
18. The Prague Declaration: Meaning and Testimonials
Rob D. van den Berg, Daniel Svoboda, Ada Ocampo, Juha I. Uitto, Silvia Salinas Mulder, Rashmi Agrawal and Josephine Watera
The International Development Evaluation Association (IDEAS) has a long history of discussing global and international issues in development. From 2015 onwards this focused on how evaluators should take up sustainability issues in their work (Bangkok, October 2015), to how evaluators could support progress towards the Sustainable Development Goals (Guanajuato, December 2017), to how evaluation could contribute to transformational change to solve the global crises of our times (Prague, October 2019). In Prague, the IDEAS Global Assembly ran in parallel the Third International Conference on Evaluating Environment and Development, organized by the Community of Practice EarthEval and the Independent Evaluation Office of the Global Environment Facility. This led to a ‘perfect storm’ of ideas on how evaluation could support and strengthen transformational change, from economic, social to environmental issues, considering equity and equality as well as working in contexts of fragility, conflict and violence. This special section of the book consists of reflections on the Prague Declaration and what it means, with a substantial contribution from Daniel Svoboda, and testimonies from Silvia Salinas Mulder, Rashmi Agrawal and Josephine Water, as well as statements from Ada Ocampo, Juha I. Uitto and Rob D. van den Berg.
Comments 2
Pingback: EvalSDGs Week: Evaluation and the Transformational Imperative by Joyce Mukoma and Scott Chaplowe – AEA365
Pingback: IDEAS Book: Transformational Evaluation for the Global Crises of Our Times | IDEAS