Dataheads: Making Sense of Big Data

by Lenny Laurier |  Mar 12, 2012  

We live in a world where information is obtainable and accessible. In the Information Age, big data is the name of the game and right now information is prevalent, however it is also scattered and under-utilized. No, I’m not crazy, there is so much more we could be doing with the data we’ve collected. You’ve seen what they’ve done with data already and it’s pretty amazing. Sites can get a sense of what you are interested in and advertise relevant offers. Facebook has a good idea of who your closest friends are and can suggest that you “friend” one another. Well, truth of the matter is, you haven’t seen anything yet. For the most part, data is siloed in its respective properties. Once you start layering data from different sources, the story starts to change. The question is, how can we use intelligence to make sense out of massive amounts of data? Here are just two companies who are set to take on this challenge.

It’s Elementary My Dear Watson!

Watson is not only a super computer, but also a champion contestant on the TV game show Jeopardy. Created by the folks at IBM, Watson is able to take in massive amounts of data, understand it, and give you back an answer with a level of confidence. The key to Watson is that it is able to understand natural language, something that most of us often take for granted. Watson is able to understand natural language and review massive amounts of data in order to give you the right answer. As Jay Dweck, a financial services executive, puts it:

“Consider a human who can read essentially an unlimited number of documents, and understand those documents, and completely retain all the information in those documents. Now imagine you could ask that person a question. That’s essentially what [Watson] gives you.”

Watson is being utilized in various disciplines from medicine to financial services. Citibank recently announced it will be using Watson to help filter and understand it’s data. The hopes is that Watson will help Citi better serve their clients, and analyze financial risk and investments. Watson is also helping choose the best treatment for cancer, piloting this project with Cedars-Sinai research hospital in Los Angeles. Watson’s goal is to aid doctors by providing suggestions for cancer treatment based on the patient’s symptoms.

Viewing Into A Magic Crystal Ball

The war on terror has kept the US government working around the clock to ensure that the people of America are kept safe. Many agencies, ranging from the CIA to Homeland Security, all work on different aspects of security, and all own various databases storing different pieces of information. Combing through and pairing information from these databases has the potential to ward off criminal efforts. The challenge is searching these separate databases for information, and this is where Palantir enters. Palantir is a Silicon Valley start-up who’s software has the ability to search through all available databases, find and pair related pieces of information, and wrap it up neatly in one place. Palantir’s advanced algorithms can piece relevant data from a multitude of sources and make sense of it. It has the ability to identify a set of unsuspicious actions by an individual, analyze them, and determine that these actions could lead to a terrorist attack. It’s like the system has the ability to sense criminal activity, before it happens. This sounds like something straight out of the movie Minority Report.

Wrap Up

90% of the data in the world today has been created in the last two years alone. Now that we have access to this massive amount of data, we are left to tackle the challenge of making sense of it. Filtering, pairing, and understand information is a big undertaking, but the benefits definitely out-weigh the costs. Hey, if a super computer can go head-to-head with some of the smartest contestants on Jeopardy, then finding a cure for cancer can’t be a stretch. Especially if you’re a 13 year old child prodigy.