Skip
Current Issue
This Month's Print Issue

Follow Fast Company

We’ll come to you.

7 minute read

The Future Of Big Data Is Bigger Than You Can Possibly Imagine

What happens when everything you do is collected as data? It's either really helpful or really scary (or both).

The Future Of Big Data Is Bigger Than You Can Possibly Imagine

Illustrations: Login via Shutterstock

Imagine a world without government, schools, a legal system, law enforcement, or companies. It’s a world unlike the one we currently live in—but based on the evolution of technology and how we use it—representative of what the world may become.

Imagine a computer infrastructure that could—with global knowledge and the ability to enact precise tweaks to the social and economic structure—drive the evolution of society. This is the idea behind the Universal Graph.

In mathematics, this is a graph (or network) in which a piece of information can be connected with other pieces of information until all finite information is integrated. In fact, these graphs already exist—albeit in the disconnected data silos of large tech companies such as Netflix, Facebook, Google, and Amazon. More on that later.

Currently, this information is distributed amongst all of us. You know a set of facts and I know another set of facts. The Universal Graph is about taking all the data we generate and creating an interconnected fabric—a sort of computational mirror image of our world. This mirror image can be subjected to algorithms able to realize large-scale patterns in our world and leverage that knowledge to make society work better.

What can be included in a graph? Anything and everything: A gene, a particle, a book, a test, a company, a person. All of these "data points" can be interlocked with other data points. For example, information in a book could be the basis for a test, which could be taken by a person with a gene that makes him or her gifted in foreign-language acquisition. Each of these data points is both separate and connected.



This Universal Graph doesn’t exist—yet. Right now, most information is siloed. Information about you—from your date of birth and college transcript to your TV-viewing and shopping habits—is spread out over many databases that work in isolation, disconnected from each other.

But we’re getting closer to a more connected world. So think of the Universal Graph as a universal mind: datum that link to each other ad infinitum and that speak to each other via algorithms to solve problems.

In other words, the Universal Graph can better solve problems—with a higher fidelity of knowledge of our current existence—than our traditional institutions.

Crowdsourcing solutions

Let’s look at how we could apply the Universal Graph to government. The purpose of any modern democratic government is to protect the rights of its citizens, to create, sustain, and protect markets, and to produce and maintain education systems and infrastructures that allow society to function.

To that end, we elect representatives that act on the needs of their constituents. But why do we need representatives? In fact, why do we need a president?

The data that we already gather about people tells a pretty compelling story about what they consider good or bad and what they want for the future. Let’s use it to crowdsource solutions: Pairing data from everyone with analysis and the ability to route resources (financial or otherwise) to people who can solve societal problems can be arguably more effective than the systems in place now.

Let’s look at education. Is it necessary that teachers and professors teach students in physical buildings? Is it necessary for students and their families to pay tens to hundreds of thousands of dollars for a college education?

Drilling down a bit further, let’s look at testing. Algorithms can already parse classroom discussion boards (modeled as a graph!) and predict student grades and test scores with a high level of accuracy. In the not too far distant future, the Universal Graph could possibly do away with testing. Think about it. If computers are aware of every nuance of an individual, their "grade" can be computed, much like my vote can be computed.

The current movement to online education is a small step towards the future. Expand that to include automated "teachers" that can identify your knowledge gaps and the cost of education may be more accessible to more students.

The Individual’s Role in The Universal Graph

The Universal Graph leverages all known information about a person to help them better make life’s big (and small) decisions. It would know how people similar to you have succeeded or failed, as well as what opportunities or pitfalls lie in wait.

The Universal Graph could analyze information about a student (such as academic performance, extracurricular and sports interests, social media activity and family educational history) to figure out what career path he’s best suited for.

Who should he marry? How many children should he have? Where should he go for vacation? What should he do to maximize his health and prolong his life?

These questions could be answered by the Universal Graph. This is a brave new world—not Aldous Huxley’s dystopia, but one with social and economic systems optimized via integrated knowledge and algorithms.

A Friction-Free World

Jobs that require data integration (such as education) can be done more efficiently by the Universal Graph than by a human. That doesn’t mean that people won’t have meaningful careers. The Universal Graph would point you to a field that’s more suited to your skills. Humans could reach a new level organization—a friction-free existence—thanks in part to the organizing algorithms of the Universal Graph.

Broadening the scope to a company, the Universal Graph can help figure out what new products or services it should offer, what departments to cut or expand, and which employees should be tapped for management and which should receive more training and support to help them succeed.

Taken to the level of a country, the Universal Graph can help guide a nation’s stances on geopolitical issues and create more effective programs to solve heretofore "unsolvable" problems such as poverty, gun violence, and social inequality.

Taken to the global level, the Universal Graph could help people fix many of the world’s social ills so structures such as government or the judicial system can scale down or be done with altogether.

Computer World + Real World = New World

"Wait," you may say. "This sounds like science fiction." But we already live in an era in which the "computer world" and the "real world" are becoming more coupled.

Let’s look at a tiny sampling of the databases that analyze everything from your TV-viewing habits to your love life.

Netflix collects information on the movies and television shows you watch as well as when you watch them and how long you spend in front of your flat-screen TV. Dating sites such as Match.com and eHarmony log the attributes you seek in a mate and education sites such as Udemy and Coursera note what you want to learn.

Social media sites such as Facebook and Twitter learn about your personality by parsing the vacation and baby photos you post along with the news articles you link to and your thoughts on the presidential campaign. Meanwhile, Amazon builds a sense of who you are based on what you buy, everything from books and digital music to ski gear and disposable razors, and even organic apples.

What happens when the information in all these systems is in a Universal Graph? What it knows about you grows more robust with each click. The Universal Graph can help you reduce lots of trial and error so you can better find a life mate, for example, or better choose a college or university to attend.

In other words, the past and the present is encoded in the Universal Graph so you can better understand and make future actions.

There’s already a growing need by enterprises to consume and process massive-scale graph data. Software companies such as DataStax are creating and providing new graph-computing technologies to fill that need.

Graph technology is now helping data-heavy companies such as Netflix and Spotify with improved relationship analytics, for example. Graph technology can also help utility companies better predict when they will have peaks in usage or equipment failures, and banks can detect more instances of fraud or insider trading.

The Universal Graph isn’t here yet, but its promise is compelling. Current graph technology can help companies and organizations run more smoothly—think of how it can help people reach their potential and lead more fulfilling lives.

Dr. Marko A. Rodriguez is a graph computing expert and has been in the field for more than 15 years. He was previously the founder and CEO of Aurelius and is now the director of engineering for DataStax.

loading