How to avoid data mistakes
Presently there is a huge shortage of people with analytical experience. We don’t know what this shortage looks like in the UK, but in the US there is a shortage of 140,000 to 190,000 people with analytical expertise and 1.5 million managers and analysts with the skills to understand and make decisions based on data. In short, data sets are growing much faster than the number of people with expertise to understand them.
The rate of data growth is not going to slow down, what this means in a practical sense is that people who have traditionally never worked with data in their job, will start having to. Reading data incorrectly is often worse than having no data at all, so to help with this process here are some tips on how to avoid basic mistakes with analysing data:
Recognise the data you don’t have
A common mistake is to assume that your data tells you everything you need to know. A great example of this happened in WWII. The mathematician Abraham Wald was tasked with adding extra armour to WWII bombers to stop them from getting shot down. The data Wald had to work with was the anatomy of the WWII planes that returned from bombing raids. Most of these planes were peppered with bullet holes on the wings, nose, and tail. See the below:
Undamaged plane (left). A plane shaded where bullets struck (right).
From this the air force generals proposed adding extra armour to the wing, nose and tail. This seems logical, after all these were areas that were frequently getting hit. Wald, however, believed they needed to reinforce the areas that didn’t have bullet holes, the engines and the cockpit. Why? Because the planes that were hit here were not returning home at all. They weren’t part of Wald’s data set, but for this exact reason, they provided him with the correct answer.
This is a classic example of how easily data can lead you astray. Before looking at any data it is important to ask “how much don’t we know?”
Avoid closed loops
If you are not careful, acting on data can cause some potentially damaging feedback loops. For example, if you were looking at demand figures for the country and you identified that Warwickshire had a higher demand than anywhere else, you may want to reroute resources to that area. Perhaps even hire more staff. As a result of this action you are able to offer a better service in Warwickshire giving you more satisfied customers, so demand keeps growing.
From this, it would seem that your original assumption was correct. You rerouted more resources to Warwickshire and your data tells you that this has increased demand. The issue is, however, that your service hasn’t improved anywhere else. It may have even gotten worse because your focus has been on Warwickshire. You might have improved in one area, but overall your change has damaged the business.
This is a difficult mistake to recognise. With closed loops like this one, it is important to be aware that data created as a by-product of your actions can lead you believe something is working better than it actually is.
Beware personal biases
Data is often held up as being infallible, but this is not true. We have a tendency to naturally want to support our own ideas, and data often provides a great platform to do so. If you are selective about what you use (and ignore) you can usually find data to back-up your arguments.
Trying to approach data without a preconceived position is often the best way forward. It is better to use data to shape your ideas rather than prove a theory you already have.
Data is very powerful tool for businesses, but it also has the potential to be damaging if used incorrectly. The tips above should help you avoid some of the common mistakes of data analytics, but the best advice is to continually assess your strategy. Be aware that sometimes you will read data incorrectly and be quick to react when you do, but also be willing to change the course of your strategy because of what your data is telling you.
Click here to contact us, or subscribe to our mailing list by entering your email address at the bottom of this page.