Photo: EP Banner

Evaluating Our Work

The Laura R. Chasin Project for Monitoring, Evaluation, and Learning (MEL) encompasses a variety of approaches to measure and understand the impact of a dialogue project and to learn from that data to deepen our impact as we move forward.

In 2016, Essential Partners made an organizational commitment to comprehensively evaluate the impact of our work. EP today boasts the industry’s leading evaluation system. We use that data to learn as an organization, respond to growing calls for robust evaluation in the field, and tell powerful stories about dialogue.

Our Impact: What We Measure and How

As people engage one another in dialogue, communication habits shift, trust is restored, understanding is deepened. The group becomes more resilient and cohesive.

Not only that, dialogue participants feel more respected and understood even in the midst of deep disagreement. They move beyond stereotypes, assumptions, and implicit biases. When that happens, people feel a greater sense of belonging and begin to trust that other people—even those of different opinions, backgrounds, or ideologies—are committed to helping their community thrive.

To achieve this shift, EP trains local stakeholders to foster healthier conversations about the issues that drive polarization and conflict or silence and deadlock. We co-design systems that support these conversations and co-develop plans to address local challenges.

In our evaluation program, we look at impact over time. The first set of interviews take place three months after the project ends and occur at intervals for as long as two years. We measure the impact of our work in four categories (or domains):

  1. Understanding Others and Being Understood
    Impact in this area is felt immediately in a conversation and can be measured with quantitative questions in pre- and post-project surveys.
  2. Willingness to Engage Across Differences
    In the short term, we measure this by asking how people’s willingness to engage across difference has changed since working with us. Over time, we measure personal transformation: How have people’s habits shifted?
  3. Equipped to Engage Across Differences
    Immediately after a project, we begin measuring how equipped people feel to engage across difference. We ask people to set goals and articulate clear next steps to apply what they’ve learned. In the long term, we measure perceived shifts in communication patterns within the community and resilience, defined as the capacity of a community to remain invested in one another in moments that threaten the identities of one or more groups.
  4. Healthy Relationships and Social Cohesion
    We begin to see improvement in trust and belonging immediately after dialogue and continue to measure the growth and healthiness of relationships over time.

The full downloadable PDF goes into more depth on each of these categories. Information provided includes details on how we measure each category of impact and the questions we ask to measure each category (or domain).

High-Level Findings

Several years of evaluations have produced an initial set of high-level findings related to the impact and outcomes of Essential Partners projects.

Immediately after a project is completed, more than 90% of participants report one or more of the following shifts in their own outlooks and attitudes:

  • Increased understanding of one another
  • Increased respect and appreciation for one another
  • New skills and knowledge for communicating across differences
  • Realization that constructive communication is possible
  • Commitment to engaging differently, especially in difficult moments.

Stronger relationships and new possibilities emerge. Communities and teams that we work with have leaders and systems in place to support public conversations on divisive topics. They develop a plan suited to their own unique context that anticipates upcoming challenges and works toward community resilience. Specifically, our evaluations find reports of:

  • Personal transformation
  • Increased social cohesion (including both new relationships and improved relationships)
  • Increased sense of belonging and inclusion in communities
  • Use and retention of skills, leading to increased resilience within the community

Over time, as communication habits shift, people begin to trust each other enough to come together in exceptionally difficult moments as well. When that happens, people feel a greater sense of belonging to their community, and trust that others—even those of different opinions, backgrounds or ideologies—are committed to helping their community thrive.

With a foundation of healthy communication and cohesion, these communities are able to cooperate, plan, collaborate, and innovate to address challenges.

MEL and You

We are making EP’s MEL system available for two purposes. First, we want to help people understand how we measure success and hold ourselves accountable. Second, we want to provide a resource for institutions, partners, funders, and supporters interested in making their own investment in monitoring, evaluation, and learning.

Data-driven change is a powerful method to ensure quality and promote best practices for any type of organization.

For large and small organizations alike, having a comprehensive measurement system that reports the impact of your work can turn valuable data into system-wide improvements, expanded partnership agreements, and ongoing customer engagement.

MEL is intended to address the challenges EP faces in understanding the impact of our work across diverse projects, populations, and contexts.

Shifting how people communicate in their communities and how they navigate differences in identity or values:

  • Takes time. The impact of this work is not seen immediately.
  • Is hard to define and quantify. The power of our data is in the stories people tell us.
  • Looks different in every group or community. Context matters—each community faces different challenges and measures success differently.
  • Will have anticipated and unanticipated consequences. It is important to measure both.

If you face similar challenges in evaluating your work, we hope our system can act as a resource for you.

If you’re familiar with MEL and you’d like to see the full details of our system, you can download the PDF of our full system for free.
 
To use or adapt this work, please provide attribution according to the guidelines of Creative Commons. This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/4.0/.

For more information and questions:

Katie Hyten
Co-Executive Director
617.923.1216 x27
katie@whatisessential.org