8 trends in the area of Competitive Intelligence

As we enter the age of social networking and social IT, 8 Competitive Intelligence Trends were identified as a result from a recent study sponsored by Comintelli, by Södertörn University. The study was based on both a literature survey and an exploratory study with in-depth interviews of several Competitive Intelligence (CI) experts.  The Competitive Intelligence Trends are:

…

I will discuss these one at a time, starting with trend 1, namely “Internet changes the business model for Competitive Intelligence (CI)”.

Intelligence Trend 1: Internet changes the business model for Competitive Intelligence

One theme discussed by several experts was how the business situation for the Intelligence industry is changing, similar to how the business models of the media industry in general is changing. A recurring theme in the interviews was a concern with how new competition from Internet services with a strong end-consumer orientation, such as Google, Twitter and Facebook would affect the CI industry.

The Internet services have features and functionalities that are partly overlapping with services from the CI industry, as well as those of traditional media. In contrast to traditional media, Internet services have business models that can be said to be context-oriented rather than content-oriented, i.e. their primary value lie in structuring and accessing information that already exists, rather than creating new content, which is similar to how many CI firms operate as well.

One observation was that the CI industry, therefore, needs to look more at how to connect and refine knowledge generated from general Internet services instead of traditional media. Since CI services are relatively expensive they need to add substantial value “on top” of the Internet-oriented information services to be able to motivate their value for their customers. For example, new CI services could add value by offering different mixtures of more extensive service solutions, adding more analytical power, offering more advanced forms of filtering of information or by making the collaborative and social dimensions of the tools more advanced.

The challenge in this new situation is how to reach out and connect to the new CI users and customers. The CI providers must find ways to explain to their future customers what added value their solutions give and how they are intended to use their products on this new market. For example, the use of many different terms does not make it easier for a non-CI expert to understand, e.g. the confusing terminology of competitive intelligence, business intelligence, knowledge management and market intelligence, and so forth. When users of the intelligence services are no longer “CI specialists”, it is crucial that they are easy to understand the benefits and very simple to use.

Intelligence Trend 2: Competitive Intelligence’s role in the modern networked organizations

Several of the interviewed CI experts noted that the need for handling information flows is infinitely large today due to the increased availability of information (which is similar to the view taken in e.g. Manyika et al. (2011)). This development was observed to be driven by a combination of increased market-orientation and technological innovation that offer both opportunities and challenges for the CI services. One expert observed that traditionally the CI analysts have often worked as single self-governed experts or in a small group of specialists. They worked exclusively with CI sources and other related database and newsbased services for expert usage. Typically, they have either delivered tailored analysis for management decision-support, or competence support for the whole firm in the forms of information portals or pamphlets. The question is how that work role will change in the networked organization. When the company is no longer divided into clear-cut function but works more in interdisciplinary teams, then the CI services for that environment must also become more general-purpose to fit that situation. At the same time it was noted that the worker in a decentralized knowledge-intensive organization is accustomed to manage large flows of information.

Moreover, it was noted that information about the surrounding business environment of an organization is useful in many different places, roles and situations in the organization. Today, it seems that competitive intelligence as a specialist profession is mostly self-taught, at least in Sweden, according to one of the experts. There are some minor courses or education, but the initiatives lack a larger clear professional context and clear academic identity. According to the expert, this reflects the fact that CI is largely a work behavior that all professionals should have in a knowledge-intensive organization. The CI industry and earlier CI scholars made the distinction between spontaneous and organized CI, cf. Hamrefors (1999). The point made by several CI service providers has been that they focus on organized CI, only. This seems to contradict the fact that most companies focus on spontaneous, “self-taught” CI according to one expert. It was suggested that perhaps the distinction spontaneous vs organized needs to be revisited, in the light of the networked organization, and, thus, any tool or service that is strictly specialized in nature will not fully fit the new needs. At the same time, according to several of the interviewed experts, the use of networked work methods is still distant for many larger organizations today.

Well-established larger industrial enterprises have close ties between their traditional way of working and their core business idea. For these organizations, it seem unclear how they can become networked without challenging their core business values at the same time, noted by two of the interviewed experts. Interestingly, it was also pointed out in the interviews that contracts with major IT enterprise service-providers was thought to be an impeding factor in the transformation to networks. This goes against the idea that IT in general is a progressive force in the context of organizational development. In this case, it seems that the Internet-centered information providers are considered progressive, but traditional enterprise IT providers are considered impeding. An interesting question here is how more “progressive” alternatives of CI services would look like, if this is true. Can CI solutions and services be a key driving force of growth and innovation that transforms the way organizations work as well? Another discussion centered on how to help large companies that have realized that they are “stuck” in an industrial way of working, and provides CI solutions, perhaps in combination with other organizational development solutions, that would help these companies transform into more networked ways of working. CI solutions are typically a mixture of automatic tools and the services of human CI analysts.

Several of the interviewed CI experts noted how the increased automation was a driving force that “pushed” the human experts towards more advanced forms of analysis work. According to some of the interviewed experts, it is unclear exactly what will be the professional role of the CI analyst of the future, depending on which way the technological development will go. For example, will automatic text summarization become good enough so there is little need for humans to intervene at all, or will automatic tools only be used to empower the CI analyst when interpreting and analyzing a text? In other words, understanding of how the boundaries between technology and human expert work will develop will be an important part of the competence of the CI professional. In that sense, the CI professional needs to understand the socio-technical nature of CI, together with content creation and communication.

Several of the interviewed CI experts noted how the increased automation was a driving force that “pushed” the human experts towards more advanced forms of analysis work. According to some of the interviewed experts, it is unclear exactly what will be the professional role of the CI analyst of the future, depending on which way the technological development will go. For example, will automatic text summarization become good enough so there is little need for humans to intervene at all, or will automatic tools only be used to empower the CI analyst when interpreting and analyzing a text? In other words, understanding of how the boundaries between technology and human expert work will develop will be an important part of the competence of the CI professional. In that sense, the CI professional needs to understand the socio-technical nature of CI, together with content creation and communication.

Intelligence Trend 3: Increasing complexity of analysis

The CI process can vary in details which external experts may not always have insight into, according to several of the interviewed experts. However, the CI process was described by several of the interviewees as a chain of information refinement steps where the initial step is usually starting from public sources, such as daily press and trade journals. Intermediate steps are typically done in specialized CI service organizations that aggregate and refine information relevant for different industries or sectors. The final steps are taken within the user-organization that will also use the final information. Larger user-organizations often have their own specialized analysts that further aggregate and refine the information, were one of the observations. The final analysis, that turns knowledge into action, is typically done by the end-receivers of the information in the business processes. Another observation was that the CI analysis chain is mainly motivated by efficiency, but another important factor is to guarantee high quality.

An interviewed expert noted that when the automatic information seeking tools become more powerful the CI analysis chain will be affected in several ways. One suggestion was that the chain may be shortened, where some intermediate steps in the chain can be skipped. For example, the need for internal expert analysts in the user-organization may not always be needed anymore. Instead, information may go more directly from external sources to an end-receiver in the core business process, the interviewed expert noted. Similarly, studies in social networks of research and development also suggest that the role of a single “gatekeeper” is transformed into a network of specialists (Whelan et al. 2013).  One interviewed expert noted that the role of the CI analyst may have to evolve when automatic solutions become more advanced. One suggested way on the human side of CI is to improve the quality of the analysis by adding more insight into it. For this to be possible the analyst must broaden or deepen the analysis somehow. The interviewed expert suggested that the CI analyst must become more of a domain expert as well.

Another suggested alternative was to increase the complexity of the analysis and for example look at more variables and larger data sets. A third suggested alternative by an interviewed expert was to use more advanced forms of collaboration during analysis, in order to make the analysis richer and more multidisciplinary. At some point, migrating to a networked work model is probably the way to handle the increasing complexity of the analysis work, which is also what is indicated in Whelan et al. (2013).

…

Intelligence Trend 4: Quality assurance of content

One way to add value to the CI process is to work with information quality (Eppler 2006) in order to systematically raise the level of insight in the analysis and also make the level explicit to the receiving party. This type of work seems to be at an early phase, at least in Sweden, according to one interviewed expert. Content analysis of CI is analysis of texts and other media, which is related to methodology from social sciences and humanities. …

However, the quality of CI should be determined based on its quality for business analysis purposes, similar to business intelligence (BI). For BI it is natural to use the notion of data quality systems since data is normally numerical, where the quality measures can be easily automated. CI is different from BI since it deals mainly with text and media, i.e. with so-called “unstructured” information, or information in free form.

It deals with information, in the sense that it is contextual coherent message of “potential knowledge” (Eppler 2006, p 22). But even though the content is in free form and its interpretation requires human thought, the analysis includes both qualitative and quantitative approaches, similar to other kinds of methods for media analysis and media evaluation. One of the interviewed experts raised an open-ended question how exactly this kind of quality assurance should be done, and how it could be communicated in a transparent and understandable way to the receiving party (that may not be a specialized CI analyst). It can also be noted here that to use more rigid quality management systems in the domain of CI and knowledge management “is a dangerous undertaking” due to the unpredictability of knowledge work (Eppler 2006, p. 13).

Intelligence Trend 5: Integration of different content

The typical knowledge worker that uses CI has many information processing systems they work with. To define and redefine the position and role of a CI service in such an environment is an important question, according to several interviewed experts. For the user of information it is important to understand the basic function, or added value, of the CI service and how can it be connected with other streams of information. The needs and requirements for tools that can handle information integration is highly dependent on the level of IT sophistication in the organization.

Today this level can vary substantially depending on industry and what kind of organizational model that is used. However, several of the interviewed experts pointed out that these issues of integration of services are needed and important. In particular, there is a demand for CI services to be able to connect to general-purpose information systems in the enterprise, such as intranets and Microsoft Sharepoint. Even though this is possible on a technical level, the solution is often not satisfactory. The general-purpose platforms often lack important functionality that is required to really take advantage of CI content, such as advanced search functions and metadata filtering mechanisms.

Information integration has increased in importance for a more networked organization, cf. Grey (2012). The division in a more decentralized organization is more self-organized, continuously changing and informal. Therefore, there is no way of knowing in advance who will need what information. However, the usages of social media services are still also poorly integrated in many organizations today, according to several of the interviewed experts. There was a belief of these interviewees that the integration will continue, but the exact way is still unclear.

One tested alternative has been to introduce social enterprise software with similar functionality found online, but that has not worked well according to several experts. On the other hand, if employees start groups on external services, such as Facebook, the information becomes even more scattered for the organization, which was another observed problem.

Intelligence Trend 6: Competitive Intelligence beyond enterprise 2.0

The basic principles of web 2 and social media are not really enough anymore, according to several interviewed experts. Something beyond the vision of enterprise 2.0 (Mcafee 2006) is needed, but exactly what was not clear to them. Early attempts of Enterprise 2.0 that simply introduced social software in organizations have not worked well in the experiences of these experts, which is supported also by e.g. Li & Bernoff (2011); Bradley & McDonald (2011). The problem is not new, earlier attempts with so-called groupware as well as earlier attempts of knowledge management systems show even more problems in their approach (Koch 2008; Levy 2009). It seems that solutions from enterprise 2.0 solve some of the problems of earlier methods, but perhaps not all. There seems to be a gap between technical feasibility and the social requirements that may simply be too large for certain organizations (Ackerman 2000).

Organizations are on different levels of maturity with regards to both CI and the usage of advanced social technology, according to several interviewed experts. It seems that some organizations may be advanced in one of two ways, either in their usage of CI analysis in their work (cf. Hedin et al. (2011)), or in their use of social technology (cf. Li & Bernoff (2011)). However, it still seems uncommon that an organization is advanced in both ways at the same time, at least from the experience of some of the interviewed experts. This indicates that ways to combine advanced CI methods and enterprise 2.0 is still an open question.

Another phenomenon that was noted by the interviewed experts was that organizations that are not so technically advanced are in a similar situation today that for example telecommunication companies were in the 1990s. But the difference is that the technological tools they require are more mature today, whereas the tools in the 1990s were tailored by the organizations themselves. To guide these organizations forward, more support is needed on the technical side and the solutions must be made simpler and more attractive. On the one hand, the clients cannot be assumed to be that visionary concerning technological choices, here they need finished solutions. On the other hand, these same organizations may be mature when it comes to knowledge work and CI competence, either organized or spontaneous, compared to the technologically advanced industries.

Intelligence Trend 7: The human experience of CI services and tools

The fact that CI services and tools simply “function well” does not give it a competitive edge anymore, according to several of the interviewed experts. The basic technological problem is in a sense solved according to the experts, and most providers build their solutions on these solutions. What is still not solved is how to design the experience for CI, cf. Forlizzi & Battarbee (2004). What is a scarce resource for the CI professionals today is attention, an interviewed expert pointed out. The way to require minimal effort is to have an experience design that gives instant and non-intrusive access to information in a way that is attractive. In a similar way, the value a CI service gives to an organization must be quickly understandable, for it to get any attention at all in the first place.

It is a daunting task to make productivity tools such as CI tools that demonstrates direct value. Tools that give the organization as a whole value, rather than the individual, can have values that are not instant but pay off in the long run. Typical long term assets can be better reuse of knowledge, better collaboration, better use of experts in the organization and so forth. However, neither of these organizational assets are “instant” in nature. It will be crucial to bridge this and make these values explicit somehow, according to one interviewed expert. The expected experience of the users of CI services is often influenced by their usage of consumer services such as Google and Facebook, according to several of the interviewed experts. An observation was that this places the bar fairly high for experience design of specialized CI tools such as knowledge portals. In general for all knowledge work, this is problematic because it is expensive and solutions risk being specific for a particular organization, cf. (Davenport 2005).

Furthermore, it can be hard to get permission to study CI processes at all, due to their often sensitive strategic nature according to some of the experts. Users also need to understand that the consumer services online and tools within an organization have different purposes and functionality, something that is not obvious to the nontechnical user. Organizational systems also have a hard time to keep up with updates of systems and hardware in the same way as the individual consumer. This limits the technical possibilities to use cutting-edge technology such as the latest graphical code libraries for web browsers, according to some of the interviewed experts.

Younger persons also tend to come with new behavior and are less patient with poor design experience, according to several of the interviewed experts. No matter what the order from the superior has been, they tend to use their own consumer services to solve problems instantly instead of using the organizational solutions. Exactly what this change stands for and its universality is a question for debate, but in practice it seems to be a problem that needs to be dealt with somehow. On a positive note, the same interviewed experts said that they learn a lot from looking at how younger persons use technology, both in companies and from their private life.

In that sense, the consumer market seems to lead the way when it comes to experience design, and productivity tools follow, whereas at an earlier stage when the focus was on technical issues, the roles were reversed. This seems to fundamentally change the situation for development of specialized tools such as for the CI industry.

Intelligence Trend 8: Increased volume of information

The amount of information that the CI professional needs to handle seems to continue to increase, according to several interviewed experts. In general, this increase of information is “unstructured” in the sense that it comes from many different sources, formats and has different types of content. However, from a human and social perspective it is rather that the new formats are more natural, a perspective we prefer (Ackerman 2000). This naturalness is of particular importance in relation to collaborative work, as pointed out by Kock (2004).

Today, many organizations have to use substantial effort to handle the increase of information volumes (Manyika et al. 2011). For the CI professional, increased text volumes means less time to spend on each information item, on average. So, there is an increasing need for succinct material in “small chunks”, according to one interviewed expert. Another way is to rely more on advanced forms of metadata or other structures that classify and filter material for the CI professional.

A general question is how the value of information can be improved on the level of the individual, as one interviewed expert noted. This relates to questions of how to avoid information overload (Eppler & Mengis 2003). The increase of information is also a consequence of increase digitalization in general, cf. Castells (2010). This means that more information is easily accessible as a basis for decisions. The goal of CI is to understand the surrounding world of the organization as well as possible. With more information available in digital form, the level of predictive accuracy in the CI analysis should be possible to increase further. Due to the amount of information, new solutions will definitely have to rely on advanced forms of automatic data analysis combined with expertise in data science (Davenport 2014).

Written by: Jesper Martell & Gabriel Anderbjörk