Do social impact metrics matter?

“Social ventures can use logic models to communicate their effectiveness to investors and funders, and to identify how to improve their impact, all without collecting a single measure.”
-MaRS white paper: Social Entrepreneurship (Social Impact Metrics), 2010

“Arguably the biggest obstacle to the creation of social capital markets is the lack of a common measure of how much good has been done: there is no agreed unit of social impact that mirrors profit in traditional capital markets.”
- The Economist, September 2009

Impact investing and social enterprise are the current darlings of the business world. Differentiated from Darwinian capitalism by their focus on making the world a better place while generating a financial return, these emerging industries have enormous potential to create dramatic change - if they can believably, quantitatively prove themselves.

But social impact metrics aren’t cut and dry like financial metrics. They’re messy. They’re difficult to define: how do you quantify increased quality of life? They’re difficult to collect: who do you talk to, what numbers do you need? And they’re difficult to interpret: do higher test scores really mean better schools? Do better schools lead to increased income, equality, and happiness? Are people telling you the truth, or telling you what they assume you want to hear? Social impact metrics require substantial investment of time, energy, and resources. But without them, how can you be sure you’re making a difference?

Completing summer internships with a social enterprise and an impact investor, Mary Fritz (Erb Institute for Global Sustainable Enterprise, MBA/MS ‘13) and Patrick Huang (Ross School of Business MBA ‘13) have experienced the complexity of this question firsthand. They discuss their thoughts in the conversation below.

Patrick: Could you briefly share what you're doing with Wello in India?

Mary: Wello addresses clean water access problems in rural areas by improving personal transportation methods through our innovative tool, the WaterWheel. We are structured as a hybrid social enterprise, comprised of a for-profit and non-profit arm. Most activities to this point have been conducted under the non-profit umbrella; my overarching goal is preparng Wello for capital investment to engage the for-profit side.This has involved creating a pricing structure, financial modeling and scenario planning, and developing our market entry strategy (including identifying robust, meaningful impact metrics and a means of collecting the necessary data). 

And what are you doing for Village Capital in India?

Patrick: My main mandate is on pipeline generation, i.e. identifying and shortlisting enterprises for two upcoming Village Capital programs in India. In brief, VilCap organizes 12-week business accelerator programs for early-stage social enterprises. VilCap’s main differentiator is our intensive peer-review approach in which the cohort not only provides honest feedback to each other but also decides which two receives pre-committed financing. In the past 3 years, we have organized 14 programs by partnering with organizations such as the Hub, Dasra Social Impact, and the Unreasonable Institute.

To start off, I wanted to quickly draw a contrast between traditional, commercial enterprises and “social enterprises.” Commercial start-ups collect standardized financial measures like revenue and costs to understand their own financial situation and success, to plan for the future, and to share with external users like investors. For-profit social enterprises do the same (by committing the time and resources), but they also grapple with the additional quantifying their social impact. Is that what a social start-up should focus on right away?

Mary: It’s tough for a start-up to devote the resources necessary to collect and interpret meaningful data around impact. You need people on the ground, in the field, to constantly monitor change, and you also need people to respond to those inputs from a strategic level. It’s been my experience that most investors would rather see those resources put toward building and scaling the core business. So it’s a matter of comparing the value of metrics with the value and opportunity costs of spending your resources there. What do you think, Patrick?

Patrick: I agree, especially for early-stage enterprises like the ones that I have spoken with. The main priority for entrepreneurs in the early stages should focus on defining a financially sustainable and viable business model. Without this, any social returns generated will be short-lived.

Since these enterprises are usually started by one or two founders with seed funding from friends and family, entrepreneurs need to allocate any capital and resources to the core business and operations. With this initial capital, entrepreneurs can develop a prototype or proof-of-concept and test this out through a small-scale trial run or pilot. Through this process, entrepreneurs start to define and refine their business model. All of these activities are critical for a business to build a business, social-oriented or otherwise.

Mary: Exactly. At Wello, we’re building our impact data collection into our business model pilot. People “get” our intended outcome, but the type of customers we hope to attract will need us to prove it. So essentially, the business model relies on accurate social impact data.

Patrick: I get that. Any true social entrepreneur (whether he or she self-identifies as one) will always remain cognizant of the enterprise’s social impact and ideally, that impact would be “baked into” the model itself. As another example, I’ve spoken to a couple of entrepreneurs that design and implement renewable-powered micro-grids that provide access to energy to households in remote, off-grid areas in India. These entrepreneurs have a business model that provides a key service (energy) to households and generates revenue for the enterprise while creating social impact through energy access. At the same time, these entrepreneurs track the number of households served as a necessary metric for both their operations and social impact.

Mary: So does tracking the number of households served equate to social impact?

Patrick: Not necessarily, although in the case of access to energy, I would argue that renewable energy has a strong correlation to positive social returns such as better health (by eliminating the use of kerosene) to increased income and better education (by providing light to shop owners to sell their wares and children to continue studying). However, these social returns are much more difficult to define, collect, track, and manage for an early-stage entrepreneur who is still struggling with keeping her business running while staying true to her initial mission for social change. I still maintain that entrepreneurs at the early stages have much higher priorities to focus on.

On the other hand, I would argue that tracking and managing social metrics is not only viable but necessary for growing social enterprises that have the resources and capital to scale, especially those that are financed by impact investors. These enterprises have already received financing from traditional commercial investors or impact investors and both the entrepreneurs and investors should support the allocation of resources to do so. Based on my conversations with people from Grassroots Business Fund and Acumen Fund, both impact investors that focus on supporting social enterprises in the growth stages, they have developed initial approaches to track the social impact of their invested enterprises but neither of them are perfect.

Mary: I worked with Acumen Fund for MAP. We were tasked with researching successful exits in some target sectors and evaluating potential pipeline deals. Trying to expand our limited selection of examples, we stretched our definition of social impact (e.g., does PayPal have a social impact because it gives people access to digital transactions without credit?). Because the social enterprise space is so young and holding times are lengthening, it was tough to find relevant example deals. But they pushed us hard to prove a legitimate social mission and impact. If Acumen Fund can’t collect metrics from its portfolio companies, how can it measure its own impact? Without metrics, how do you evaluate potential investments?

Patrick: I can relate to your and Acumen’s struggle with identifying “social enterprises”. Identifying a social enterprise requires a commitment to a market-based business model and social impact. During my internship, some questions that I may ask entrepreneurs include “Who are your target customers or Who do you serve?,” "What area(s) do you serve - urban, rural, something in between?,” "Who are your major clients to date (large corporations or individual households)?,” “Who do you employ [for enterprises that work in rural areas)?" Based on the answers to these questions, I can better determine whether an enterprise is socially oriented or not. In addition, as a SvF Fellow at Ross during my first year, I tackled these same issues as a member of the Health Investment Circle. There was an assumption that any enterprise in health would be socially-oriented. At a high level, this is probably true but the same type of questioning is still relevant. For example, if an enterprise targets wealthier customers or those with more means than other customers, that enterprise would probably not pass our social impact screen.

Mary: That’s largely what I’ve seen, and I wonder if Acumen Fund and Grassroots Business Fund are exceptions to the rule? Several investors I’ve spoken with this summer have explicitly told me that impact metrics aren’t necessary; they just want to see the financial return. Financial return is necessary to build a sustainable business (and thus maximize impact), but this approach bleeds pretty heavily into traditional VC.

The other problem with that more general approach is that it doesn’t force you to critically evaluate your true impact on the populations you aim to serve. Are you accomplishing your mission? Do your achievements reflect your intentions? Are there negative impacts to your activities? There’s a difference between a socially oriented organization and one that produces a true social return on investment.

From the entrepreneur’s perspective, there are also some pretty compelling reasons to collect metrics early on. As I mentioned, sometimes business depends on it. For example, Wello expects larger orders to come from institutional customers such as government, NGOs, and CSR departments at MNCs. These customers are all impact-focused, and to attract them, Wello will need to demonstrate quantitative proof of concept. We will have a much easier time appealing to a wider range of stakeholders when we can quantify our impact. And we want to ensure that we’re adhering to our mission and actually accomplishing what we’ve set out to do - before scaling.

Patrick, how credible do you think metrics can be? How much insight do investors have into the meaning behind the numbers? How do you ensure your cash is really making an impact?

Patrick: Credibility for social impact metrics is largely determined by how accurately we can quantify the impact at an absolute and relative level. The time when we can quantify social impact metrics at the same level as we do for financial metrics, however, is probably a ways off. For example, on education, how does selling books to children compare to building schools? On health, how does providing telemedicine services to remote, rural communities compare to building low-cost primary health clinics in low-income areas? If one is better than the other, how much better? Now, what if we were talking about a rural community in New York and a low-income area in the war-stricken Congo or vice versa? The answer to these questions are difficult as they tend to define the value on a person’s life. Despite the adage “a person’s life is priceless,” people are already quantifying life. If we don’t quantify a person’s life whether that person is a teenage girl in rural Afghanistan or a wealthy person in upstate New York, then someone else will do it and that number may ultimately be zero.

Mary: There has been some pretty public criticism of impact investing recently - complaints that SVC is ill-defined and has failed to distinguish itself from traditional VC, etc. Do you think social metrics could help legitimize the industry?

Patrick: Definitely. There are many critical next steps for the industry to take such as supporting enterprises to reach scale that would inevitably demonstrate a viable track record for the impact investing industry as a whole. Defining reliable and standardized impact metrics is another important first step to start an open and transparent discussion as to whether a commercial business model that seeks both financial and social returns is even viable in the first place.

Mary: I agree. People naturally want to believe in social enterprise and impact investing, especially as traditional business and hardcore capitalism are increasingly vilified. They're such nice stories - do good and make money. But meaningful metrics are the only way of solidifying that theoretical link between social and financial returns. And those metrics have to be standardized and applicable across social businesses, so collaboration is necessary. It will be really interesting to watch all of this unfold over the next few years.

Patrick: Do the needful, kindly revert.

Mary: Great answer. I think we’re done here.