Dont be Deluded by Data

August 15, 2018
One of my college professors was a man named Ed Rothman. He was the head of the statistics department at the University of Michigan and asked: “You’re heard that statistic about how 50 percent of marriages end in divorce, right?” Everyone raised their hands. Then he added: “So what’s wrong with that statistic?” We all knew that something was clearly wrong. He said: “Think about the question that got that answer. The problem is the question didn’t specify first marriages. It was all marriages. It turns out that once a person gets divorced, there’s a high probability of multiple divorces.” Elizabeth Taylor gave us eight! She’s in the stat. So, Prof. Rothman concluded by saying that all the young people in his class didn’t have to fret because first marriages only had a 20 percent fail rate. So, you’re not watching this video to get marriage advice from me. I’m talking to you about the role of data in decision-making. You see, the point that professor was making – and one that has stuck with me through the years – is that you’ve got to ask questions about the data you’re receiving. You have to understand the question that was asked and you have to explore what the answer is really telling us. As business becomes more data-driven, do you know what was and wasn’t asked? Data is a great tool. I love data. But data that implies it can predict the future is fairly suspect. In fact, it’s usually wrong. There’s a reason why every investment fund has a disclaimer that says: “Past performance is not an indicator of future outcomes.” That’s because it’s hard to predict the future. That doesn’t mean you ignore the past. There are great ways to use data to improve and to optimize. But data doesn’t show us opportunities to break the mold very frequently. People often see those things faster than the data. We use our instincts to make decisions. That’s what we mean when we talk about people who can see around corners. Data can be a stimulus to help people do that. But you have to consider the questions that drove the data you’re using to generate the insights. Let me give you a simple example If you were to focus on data to drive more page views to your company intranet, your team might suggest the idea of creating a contest to get employees to visit. Yes, you will get more page views. But is that really what you wanted to focus on? Or did you want to focus on ensuring that a particular outcome is being realized? I mean, who really cares about a page view? But I do care that everybody knows the date of our upcoming Open Enrollment for benefits. To get what you really want to know and achieve your goals, you have to be asking the right questions and then digging deeper into the answers that you get. Here’s another example. Let’s say that I’m getting an 80 percent open rate on my emails to employees. For anyone who has ever done any digital marketing, that is a mind-blowingly high number. It’s an unreal number. In fact, it’s so high, it’s like that stat about 50 percent of marriages ending in divorce. I mean, how much email do you open? The question you have to ask is if it’s a valid metric or is a suspect number? Could it be that the email is not being read and instead a pixel is firing in a preview screen? You don’t actually know what you have. You have to ask the right question. You don’t care about the open rate. You care that employees read the content. That’s what you really need to measure. So, when you look at data, ask yourself and ask your data providers: what questions were asked to get these results? Then ask yourself something else that’s more important. Are those the right questions? Do they deeply align with my company and where we’re trying to go?”
Previous Video
The End of the Workplace
The End of the Workplace

“Can anyone tell me what a workplace – THE place where the work gets done – is these days?  The kitchen tab...

Next Video
4 Ways to Better Communicate with Millennials
4 Ways to Better Communicate with Millennials

As parents, we raised them to pursue their passions. Millennials have been told, ever since they were young...