The State of Competitive Intelligence, 2014/2015

(For a post on the State of CI at the beginning of 2014 see below for “Thoughts on CI at the Start of 2014,” from January 06, 2014.)

I’ve been in the CI business for almost 30 years. In that time there have been several major waves of strategy and approach in how CI is used and integrated into the company culture. Historically, these waves have been tied to advances in the supporting technology behind CI and the capacities those technologies enable. We seem to be in a new wave now, but one that feels less technologically driven and more socially driven.

The first wave of CI really came about with the start of CI as a practice distinct from market research. During this wave, the emphasis was on getting as much information as possible about the competitors and answering the question of what the competitor is doing now. Successful CI units were judged based on their ability to search out obscure information tidbits that could answer specific questions or shared in newsletters with staff members. Linking CI to strategy was a part of this wave, and the CI was used to fill in blank spaces and inform the strategy development. Most of the time, the CI units had their own subscriptions to proprietary databases, but also drew on third-party providers to complete secondary and primary research.

The second wave coincided with the rise of the internet. When first established, the internet was a unique and unparalleled source of unfiltered CI. It was not uncommon to find inadvertently posted sensitive company information on the company’s website or through open sources. During this wave, companies moved a significant portion of their CI efforts to in-house secondary research, augmented by outsourced primary research. Analysis really did not advance much past the analysis tools used in the first wave, and CI was still focused on addressing questions about what competitors are doing in the present. CI vendors continued to offer primary research, while the secondary providers began offering search tools that would ease the process of finding hidden information on the web.

As the amount of information available on the internet grew, it fostered the third wave of CI. This wave focused on finding and filtering information from multiple sources into a more manageable format. Secondary search and data tools allowed CI professionals to spend less time looking for the information and more time on analyzing and aggregating it. Budgets were spent building internal CI systems that could take data feeds from a variety of secondary sources and combine them into a single readable report. With the newly filtered data, CI professionals could spend more time on analysis and developing recommendations. During this wave, a number of CI software applications came on the market that could filter and categorize information for easy retrieval and storage or, in some case, provide a way to integrate secondary findings with primary research. CI questions began to move into the arena of determining what a competitor would do next, but were still much more focused on the current state of affairs.

Now we are entering into a fourth wave, this one built on big data and social media. New developments in CI combine the secondary search developments of the third wave and add in larger data analytics. These larger data analytics have made more information available to the CI professional than ever before and allow the CI analyst to make comparisons and find correlations among data sets in ways that were not possible in earlier times. Proponents of big data as a new source of CI insights regularly share stories of how analysis can predict what an individual will do based on past actions and the ability of big data to find connections between companies using social media. In this fourth wave, CI questions are really starting to look at what a competitor will do in the future, based on past actions.

One of the more interesting observations about the four waves is the consistent role of primary research. Regardless of how we get secondary data, and the increasingly powerful capabilities we have to filter and match that data, it has not replaced primary research in its ability to get the unfiltered story from the knowledge-holder in a timely fashion. Big data and social media each require large data sets, while primary research requires having the right conversations with the right people.

A second interesting observation is that CI has often been focused on addressing what a competitor or market has done or is doing. In a few cases, questions are asked with the intent of building a model of what a competitor will do, but that tends to be mostly linked with scenario planning or war games. Even in today’s big data and social media world, the tendency is still to look at what a person or company did in the past and extrapolate into the future. Only primary research has the ability to actually find out what decision makers will likely do in the future, drawing on direct interviews and targeted questions.

A final observation is that the analysis tools have remained very constant across these four waves. CI professionals still rely very heavily on a small tool box for analysis, consisting mostly of SWOT, Four Corners, Five Forces, Gap, and trend analysis.

CI has always had a degree of hype in offering promises that one new tool or another will be the one that makes it possible to know what a competitor has done, is doing, and will do. But at its nature, CI is about choices and decisions based on the best available information. The human factor still outweighs the machine, and while the machine can present data in new forms, it is still up to the human to make the final analysis.

– Erik Glitman, CEO