Are we ‘drowning in data’? Davide Hanan, MD of QlikView SA explains how all humans are naturally data analysts if we’re presented with information in the right way.
I’m often asked if we are now faced with too much data to analyse; a ‘data deluge’ if you will. People panic when confronted with large amounts of information, unable to see how it can be useful or feel incapable of trying to figure out how it can be. To understand how we can overcome this problem, we need look no further than the common housefly. It was recently discovered that the size and metabolic rate of flies allows them to process nearly seven times more information in a second than humans can, which allows them to see a human hand coming towards them in slow motion. However, it’s interesting how it’s their natural state which gives them the advantage to process so much information so quickly. Likewise, I’d argue that, without even realising it, humans are all naturally masters of data as well. However, unlike the fly, we have lost touch with our natural sensibilities which allow us to process masses of information easily and quickly.
Our main problem with handling data is that the modern world presents us with information in an un-natural way. Outside of technology, humans are naturally capable of automatically processing thousands of pieces of information every second. From the moment we wake up we’re constantly processing images and ideas and making subconscious decisions to help us survive. And these aren’t new behaviours; our ancestors also grew up in environments rich with data where our hunter/gatherer instincts made us analyse the natural world so we learnt what we could eat to survive.
For example, a hungry hunter foraging for mushrooms in a forest can make a multitude of observations and decisions to find what they need. They will instantly distinguish between different trees, leaves, the best conditions where mushrooms can grow and, of course, the mushrooms which aren’t poisonous. Although a simple analogy, this acknowledges three effective natural processes our brain uses to digest information and which to a certain extent are not accounted for in modern data analysis and analytics; the processes of association, comparison and anticipation.
The human brain strives to make associations. The hunter knows to look under oak trees for mushrooms based on previous experience that they have been found there before. Our mind constantly categorises and connects, searching out the important features together, and the warning outliers. Furthermore, we don’t just stop there once we’ve made these associations. The hunter will not settle with the first batch of mushrooms they find, but will find others and compare them against what they have found already; deciding if they are bigger or smaller for example. Also, they may draw on past experience to know which are the ripest or indeed which kinds of mushroom are poisonous.
Finally, just as we make sense of the present based on our past experiences, we constantly anticipate the future. The hunter knows to eat certain types of food because they know from past and present experience that eating it will aid survival.
Each and every one of us uses these three natural tools on a daily basis in our everyday lives without even noticing them. And we also use them in the business world to help us understand complex problems. However, most of the technology we have access to doesn’t complement and extend these innate skills. People need to use IT solutions which can make this complex data accessible and allow us to naturally discover patterns and make projections.
The Google search engine is a great example of a kind of technology which appeals to our natural sense. It uses a simple, intuitive interface which can be used by practically anyone, but has a series of complex algorithms behind it to make it insightful.
Moreover, technology should be made available on devices which appeals to our natural touch sensibilities. Smartphones and tablets are great enablers for making IT more accessible for everyone because it’s another step to analysing data in the natural world, with gestures on screen that feel more instinctual since interaction is more direct than through the mouse and the monitor.
Already we’re seeing how anyone can effectively be a data analyst. In Peru, fishermen use analytics to find out where the best location is to take their catch to for the maximum profit. Elsewhere, in Sweden, police forces use data analysis to fight crime; in one instance Malmö Police used its data to catch a serial killer on the loose. These people aren’t formally employed as data scientists, but they use their natural instincts paired with intuitive technology to improve their decisions.
Overall there’s a huge opportunity for finding insights based on our natural ability to analyse the world around us. However, IT and business technology tends to get in the way of using these natural skills because it presents information in a technical manner which discourages the natural human experience.
Businesses need to have systems in place that appeal to an employee’s natural associative, comparative and anticipative thought processes. Until then, many of us are going to be lost in the data wilderness.