The importance of measuring what matters

Early one Wednesday morning, I found myself faced with the daunting task of snaking my way through Nairobi traffic. Kenya’s capital is notorious for its traffic jams, sometimes stretching well outside the city limits.

I was on my way to meet Jenny Sia, the senior manager of Social Investments for Pfizer’s Corporate Responsibility unit. On the agenda that morning: the true purpose of measurement for an organization like PATH.

As the senior director of PATH’s Results Management, Measurement and Learning Department, I’m always interested to learn how our funders view both the art and science of measuring the social impact of their investments. Jenny and I had a very thoughtful discussion about how data, as a learning tool, could be used to transform project results into information that could deliver future successes more efficiently and effectively.

I have had several of these meetings lately. And every person—whether a funder, corporate donor, or private philanthropist—has had similar questions: “How does PATH measure its impact?” and “How do you know you’re doing good work?”  These questions are not meant to be theoretical.

Funders and implementers alike struggle with how their investment is helping the fight against disease. They want to know, in very real terms, how that looks. I think PATH fields this question from a large variety of donors because of the kind of work we take on.

This is where the art and the science of measurement meet. It can be complex work that requires a dedicated investment. For forty years, learning and adaptation have been the driving forces behind PATH’s innovative culture and our influence with so many organizations and country partners. For us, measurement must always be a means to that end.

Working across a large number of innovation platforms, PATH’s comparative advantage is our ability to move innovations past the idea or prototype stage. There are numerous obstacles and barriers that prevent funding, introduction into markets, manufacturing, and the ability to scale a solution widely to meet global needs. Some say PATH resembles a research and development–focused company, still others say an innovation incubator or accelerator, rather than a typical global health NGO.

To that end, we have adopted an adage that is becoming popular in the development and global health space: “Measure what matters.”

While measuring impact matters to us as an organization, trying to roll up the measurement at an institutional level cannot fully express the true influence and impact of our work. So instead of developing just one measurement methodology and showing how our work ties to it—such as number of lives saved, deaths averted, cases averted—we have chosen to focus on a diverse set of approaches and methods to measure our work across the lifecycle; what PATH calls the journey of innovation.

For example, our work in the beginning of this journey relies on measurement methods that help us estimate and model the impact of our work. For work toward the end, we focus on more robust surveillance and program evaluation to tell us if we’re reaching our intended target populations.

We spend time developing and applying practical methods so we can be nimble when we work with a variety of donors, country counterparts, and other implementers. For those unfamiliar with the nuances of our work, we still have institutional measures that show both the detail and scale of what we do. These indicators—such as the number of lives touched, number of doses delivered, and number of products making it to the next development stage on time and within budget—provide us with important benchmarks that help keep us focused and on track.

We have learned that if we only measure our work to comply with donor rules or regulations to show that we’re meeting performance targets, or to only show our collective value, we fall short of achieving our mission.  We have learned that measurement cannot be a monolithic discipline that is driven by standards or top-down measures only. It has to be robust, rely on a diversity of methods, and connect back to the established institutional and programmatic learning agendas.

In the end, measurement must inspire us and provide a road map to the new global health interventions and approaches that are on the horizon.