The world is awash in data. Every day data is generated from sensors measuring our climate, energy usage, geo location, and physical and emotional well-being. Some people have said that data is the new oil. But this vast amount of diverse information is—unlike oil—not in-and-of-itself highly valuable. In the oil metaphor, big data is more like the Earth’s crust; while it covers every square inch of our planet, only a few select areas contain anything valuable under the surface.
In every sector of society, organizations are increasingly turning to those that can help them gather, analyze, and translate their data into meaningful statements about their customers, competitors, and the market. IBM is perhaps the best known of these data oracles, companies that help translate the vast amounts of information into something useful while aiding in the cultivation of that which is important for each distinct application.
While for-profit companies and governments are able to engage in "building a smarter planet" with the likes of IBM, nonprofits and the organizations that make up the social sector lack the means to engage such sophisticated talent. And yet money is not the major factor keeping the social sector from embracing the data age. It is instead the sector’s lack of understanding—both implementing organizations and the donor community alike—with regard to how data can impact effectiveness, which keeps data-driven decision-making on the sidelines.
Herein lies the great, unexplored data field just waiting to be tapped.
This is not to say that the social sector is unconcerned with data. For decades everyone from the U.S. government, the World Bank, and the Gates Foundation, to small nonprofits across the world and boutique firms specializing in program monitoring and evaluation (M&E) have attempted to objectively demonstrate and understand the impact of social change interventions. Despite spending tens of millions of dollars on these efforts, it remains unsolved. There is still no consensus on what makes for quality M&E.
What is more, the effort to understand effectiveness is largely an accountability measure put in place by donors to ensure that the funding they disburse is used appropriately. While that is a perfectly understandable and warranted expectation, it limits the landscape of data discovery and the potential to gain greater understanding of how to effect social change. Consider what we could learn and how much more effective our efforts could be if the push to collect data were part of a pure science study instead of an exercise in responsible accounting.
Social change is brought about through the construction of complex systems. These systems include social entrepreneurs, funders and investors, partner organizations, policy makers, academics, community groups, and media outlets. These actors play a powerful role in the shaping of a given intervention and its subsequent uptake in society. None, however, is more influential that the social entrepreneur who crystalizes the insight and instigates the change. This entrepreneur (or group of entrepreneurs) is largely responsible for building the network of other actors around their insight, and yet we know little about the when, why, and how of that experience.
For instance, entrepreneurs in the social sector turn to mentors and early stage funders just as their counterparts in the private sector do. But while we have considerable research and insight into a best practice guide for the for-profit startup, we lack the same understanding for those budding organizations striving to change the world for the better. But one person’s missed opportunity is another market potential, and opportunities abound in the data desert that is the social sector.
Through ubiquitous smartphone technology and the relatively inexpensive cost of creating apps, data capture is easier than ever. Cloud computing offers organizations of nearly any size the ability to aggregate vast amounts of data and create distributed networks that share information resources. What is more, new platforms are helping all of us tame the big data monster with Apache’s Hadoop leading the way.
The average nonprofit or social enterprise doesn’t have the skills or infrastructure to gather and analyze the data for an entire sector, so this will take a concerted and collaborative effort. That said, convening and organizing a broad-based consortium of nonprofits that should instead be focused on program delivery probably isn’t the answer. This, therefore, is a huge opportunity for the same foundations that pride themselves on being sector leaders in transparency, accountability, monitoring, and evaluation.
Institutional investors conduct extensive research to better understand the markets into which they are investing, allowing them to make more informed and, arguably, more profitable investments. Similarly, foundations—the primary investors in the social sector—have an opportunity to increase their effectiveness and that of the entire sector’s.
Best of all there is no real potential for failure, so the bar is pretty low.