Was it the night that data failed?

I hear you asking: ‘How on earth could the polls be só wrong?’ According to the New York Times last night was ‘the night that data failed’. Assuming that the pollsters worked accurately - using representative and robust samples - how could we not be able to capture the boiling anger of a large population of the American electorate? Coincidence or not, this was also the case for the Brexit polls.

Is it still possible to forecast behavior? My answer is yes. As long as we finally step away from the outdated assumption that humans are rational beings. Because we are not! The main drivers of behavior are non-rational motivations. We base decisions on emotions: Make shortcuts based on biases and heuristics, weight ‘pros’ and ‘cons’ off based on available information at that moment. And don’t forget about the context that we are in; the pressure that people can feel. Or the other way around, the lack of feeling a certain need. Weather conditions affect our mood and can make you very grumpy. And we all feel at least a little pressure to demonstrate socially accepted behavior or adopt socially accepted opinions when we are accompanied with other people.

So, how does that work with data? Apparently, just asking a simple and straight forward question is not enough. We have to be aware of the fact that we then only ask for the tip of the ice berg and that we don’t get any insights into the underlying structure of the ice. And that’s exactly the information we need to know in order to calculate the risk of changing opinions and thus changing behavior.

It starts with the first question ‘Are you going to vote?’ Some people are very congruent in their behavior and they will go, no matter what. But for a lot of people it doesn’t work this way. They make a fast calculation of gains and losses and estimate the impact of their single vote. Which - on a large scale - is indeed close to zero. But what if a million people think this way? How would that eventually affect the outcomes of the election? And what does the information of a poll - given at the night before Election Day - do with people who are right in that mindset? If the polls inform you that Clinton lies ahead with more that 10%, to what extent do you feel the urgency to give her your vote? Vice versa; if you see that Trump could really need your vote, would that affect your willingness to put more effort in going to the polling station? So I’m asking you now; how representative is the sample that pollsters used without applying a correction based on this fact?

Another reason why pollsters undermine their own predictions is the fact that they give the electorates a reference point. And that’s great, we humans love reference points; can speculate chances based on this information. So, what if you are rationally against a Brexit, but also feeling a little bit of protest in your body? And thén the polls are showing you that it will very probably (almost definitely) result in a Britain? Again, knowing about the conflict in people’s minds would set the polls in a different perspective. And even more important gives politicians the opportunity to change this.

To get back to the question; did the data fail? No, the data did its work, in all the ways it can work. We underestimated the complexity of the decision making process and the impact that only some information has, that’s true. Once again for market research, it’s proven that we need to know about all facets that affect behavior. Using innovative and intuitive tools who dive to that deeper layer and give you insights in this is crucial when you want to connect to your citizens, patients or customers.

I look forward to discuss how Living STAT’s approach to research can benefit your organization as well.

Rachel