Data and Evaluation: Going from Zero to One
Introduction
This blog is a recap of Coeffect’s third virtual workshop of 2023, "Data and Evaluation: Going from 0 to 1," hosted on Wednesday, October 11th. This session featured nonprofit leaders (and former Coeffect clients) Betsey Mercado and Ryan Hazelton, both of whom have effectively incorporated data and evaluation into their small-but-mighty organizations.
Why This Workshop?
Over the course of 90 minutes, this workshop aimed to:
Highlight how data and evaluation can contributed to the success of small nonprofits.
Share Mariposa Kids and Objective Zero’s evaluation journey.
Address common challenges to data collection and analysis.
Share authentic advice from Ryan and Betsey on fostering a data-centric culture.
In our introductions, Betsey, the Executive Director and co-founder of the Objective Zero Foundation, highlighted their organization’s mission to connect the military and veteran community to through a mobile app that facilitates peer-to-peer connections. Ryan of Mariposa Kids shared his organization’s approach to providing play-based, after-school programming to K-5 students in San Francisco.
Coeffect’s Vision & The Data Journey
At Coeffect, we've dedicated seven years to helping nonprofits establish the foundations of their data practice, including setting up their impact measurement strategy, habits for data collection and analysis, and technology systems for working with data. Ensuring data literacy and fostering a culture of learning among teams are two other important enablers we address in all of our projects with clients like Objective Zero and Mariposa Kids.
In the introduction to our discussion, we introduced attendees to the "Evidence Continuum," a concept representing the varying levels of evidence for impact that can exist within social mission organizations. Progress on this continuum isn’t necessarily linear – in our experience, iterative cycles of planning to collect data, doing the program, learning from it, and implementing those learnings in the following year help organizations build up their evidence over time.
Betsey & Ryan's Data & Evaluation Experience
Betsey spoke about their growth and the challenges of evaluating the Objective Zero Foundation’s work. In particular, Objective Zero’s collaboration with the Centers for Disease Control Foundation and Visible Connect accelerator helped them build their data practice. Objective Zero has internally evaluated their programs for the last four years, and data from those evaluations have been published on their website and have also been shared with their Board of Directors and funders.
Ryan shared how he led Mariposa Kids' transition from virtual to in-person learning in the wake of the COVID pandemic, while simultaneously launching an organizational development process (their P.L.A.Y. initiative) to clarify their mission and prepare the organization for future growth. Ryan launched his first organization-wide evaluation project in 2022, and concluded this in the spring of 2023. This data proved to be an important aspect of the overall P.L.A.Y. initiative, by helping Mariposa Kids become more transparent and underscore their commitment to child learning through play.
The Role of Data & Evaluation
Both Ryan and Betsey shared how data and evaluation contributed to their organizations. Ryan mentioned their data became a tool to help outsiders understand their work more deeply, instill community trust, and facilitate a culture of feedback that has been an asset to their fundraising. Betsey underlined how her data helped her team identify ways they can improve the Objective Zero app user experience, and noted the rising emphasis she’s seen on data in grant applications for organizations working towards health and wellbeing outcomes.
Creating A Theory of Change
Both speakers emphasized the need for a well-defined theory of change as a starting point for the data journey. Objective Zero and Mariposa Kids both evolved their respective models over time, and shared that having a clear and concise visual representation that shows the interconnectedness between their program activities was critical in helping outsiders understand exactly what their organizations do. For example, you can see Mariposa Kids’ Theory of Change here.
User-Friendly Data Collection
Betsey and Ryan shared their methods and tools for data collection. For Objective Zero, data collection occurred within their mobile app. They leveraged in-app surveys to collect data to augment information on app user behavior, and leveraged MetaBase to report this data. In contrast, Mariposa Kids collected data directly from kids using a simple interview protocol, and collected data from parents using Google Forms. While the tools and techniques differed, both Betsey and Ryan shared that embedding data collection into normally scheduled activities, and proactively addressing questions around privacy was crucial to getting engagement.
Crafting Impactful Data Products
We discussed two example of impactful data products: Objective Zero’s 2020 impact report and Mariposa Kids’ 2023 learning report. These reports included thoughtful data visualization that allows a reader without significant knowledge of the organization’s mission area to grasp the key results and what next steps the organizations intend to take to make an even greater impact in the future.
Committing Time & Money
Finally, we discussed the financial and time commitment each team needed to bring their data and evaluation work to life. Betsey received support from a Centers for Disease Control Foundation grant, and designing and executing the evaluation for this grant turned into a part-time (~20 hour / week) job for the organization during the busiest points of the project. Ryan utilized organizational reserves to begin his evaluation effort, and was then able to solicit external funding once he had an interim product (a Theory of Change) that demonstrated his organization’s progress. Ultimately, he spent up to 10 hours a week on this work during the data collection phases of his project, but less at other times. While the time expenditure for these projects was significant, both agreed that they have seen their investment in evaluation come back to them in the form of better feedback to improve their services, increased community trust, and additional money raised from sharing their evaluation products.
How to get in touch
If you’d like to watch the recording of this webinar, you can find it on Coeffect’s YouTube Channel. In addition, you’re welcome to get in touch with Betsey, Ryan, or Paul on LinkedIn. Betsey and Ryan are always interested in collaborating with similar organizations. In addition, Coeffect provides individualized data coaching and training for nonprofit leaders – if you’d like to discuss this more, schedule a meeting with Paul here.