How Big Data Affects Social (and CI) Polarization

“Everyone is entitled to his own opinion, but not to his own facts.” – Daniel Patrick Moynihan, Former US Senator

“There are three kinds of lies: lies, damned lies, and statistics.” – Benjamin Disraeli, Former UK Prime Minister  

Long before the advent of big data, controversy over the interpretation of data has been a cause of social polarization. In fact, social polarization has been prevalent within the United States since its inception, when key questions had to be answered: What rights do states have? Where should the US capital be? Should banks be centralized? Data was limited in the emerging republic and the differing parties selected narrow datasets to support their positions.

Fast forward 240+ years later and selective fact use facts continues to produce polarization. What has changed is the amount of data and the tools available to access and analyze data. The volume of data that existed when our company was founded in 1988 has increased dramatically and as we move into the era of Big Data and Artificial Intelligence (AI), analysis capabilities have also grown. One of our key observations around data analysis and its role in decision making is that those who most successfully harness accurate information and make actionable results are at an advantage.

Of course, harnessing accurate data requires skill and the ability to filter out bias. Organizations use data to their advantage, but many draw from the same limited vendor set creating the potential group think data bias or selection bias to support a single point of view. Easy access to opposing data sets and analysis quickly forms opposing sides and polarization. What is needed is well vetted and unbiased information.

So, what is the best way to vet information? Unfortunately, most people are drawn to information that confirms their current belief system. We have seen this “homogeneity bias” perpetuated by Facebook, Twitter, and YouTube – which all capitalize upon consumers’ data to provide news stories, websites, and video stories that pique their curiosity. It is also present in the “confirmation bias”, by which people are more likely to believe information that supports their conclusions. These biases are more likely to produce opposing camps, each with their own “facts” that support their position. The Competitive Intelligence professional’s role is to vet information, overcome bias, and product balanced intelligence.

Unintentional homogeneity and confirmation bias leads to greater levels of polarization about issues ranging from global warming to immigration reform and from interpreting a competitor’s intent to launching a new product. AI technology is becoming available to identify false information, false accounts, and even some biases, but our nation and most businesses have not reached a consensus regarding the implementation of AI; rather, these technologies have sparked strong political debate on free speech, privacy, and access to data. Further, the AI developers can inadvertently insert their own bias into the algorithm, leading to biased results.

Defeating polarization and effective decision making requires a balance of high-quality information and analysis without censorship. Human judgement is needed to identify what is true and what is not. At the end of the day, Big Data and AI tools cannot replace the human’s ability to analyze data from all possible angles.

Our role at Fletcher/CSI is to cut to the truth. To do this, we rely on methodical, sound, and robust research to identify issues and formulate solutions. We scour secondary data to build broad foundations, find the top experts in the field, and educate ourselves on a topic. We interview informed people to triangulate our data findings. Then, we formulate conclusions and deliver simple, powerful, insightful, intelligence.

To find out more about how Fletcher/CSI can help you to cut through the noise and generate actionable insights, please contact us at [email protected] or 802-660-9636.


-Author: Naomi Warren, Project Manager