Capturing and collecting competitor knowledge

Competitor knowledge comes into an organisation from a wide range of data sources, such as contacts with analysts, ad hoc websites, social media, sales reports, customer service histories and so it sits in many places in your organisation. Market research will have some information, finance and accounts some more, the sales team and customer service team yet more, and CRM systems and web-tracking even more.

Ideally, the business wants to have an organised view of what each individual competitor is doing, what their potential business objectives and strategy are and other useful information in one place so that it can be shared and cross-analysed to spot trends, or to flag alerts.

The sales team will want details about the way the competitors they face are likely to respond and their strengths and weaknesses to help tailor the pitch to the customer's needs.

The ideal solution is to capture information to a database. However, the form of the database needs to be sufficiently flexible to allow for a wide range of ad hoc and qualitative information to be captured along with indexes and references to a other documents and materials. In collecting market knowledge for use and analysis you face a number of issues which go beyond traditional databases and data storage and hybrid relational and unstructured database tools are required.

 

1. The tight-loose problem

Standard transactional databases require that information come in standardised discreet chunks - what we call tight data where information is strongly constrained. So a typical transactional database will capture an order with a date, quantities, product codes and prices. However, this tight-data can't represent that the customer felt that prices were high, or that an additional item would have been bought if it had been in stock, or that the customer has £100k to spend before year end. This is loose data (some people might refer to this as qualitative data, but in fact qualitative data can also be stored in a tight format - a quote, a feeling, an association). You could capture this loose data in documents and reports, but then it's not usable for broader market analysis or comparison with other customers.

An example of the tight-loose problem is measurements of business earnings (profit). There are many ways to define earnings - before tax, after tax, before dividends, like-for-like, global, US only, including or excluding exceptional costs for restructuring. A tight data approach says that only one of these can go into the database. But competitors and customers may not provide the precise measure the database needs. The database needs to be looser in its definition to allow a wider variety of data to be captured, but probably with comments and notes useful to the analyst and reader.

In our experience where companies have defined their customer or competitor database information too tightly, the data isn't available or isn't usable as they expect. However, define it too loosely and no comparisons can be drawn either. The data becomes too specific to one customer to too unspecific to be helpful. In capturing market knowledge you need a system which is both tight and loose.

 

2. The organisational complexity problem

Conventionally, databases will normally view an organisation as a company name and an address, or two if you're lucky. In reality, companies and even individuals are more complicated than this, and consequently the way each customer relates to your business is complex. To give you an example, Pizza Hut UK is a subsidiary of Yum Restaurants and Whitbread. Yum owns KFC among others. Whitbread owns Costa Coffee. The restaurants run by Pizza Hut might be sit-down, delivery-only or sit-down and delivery. They might be owned by Pizza Hut, or Whitbread, or by a franchisee. And the franchisee might own more than one franchise, potentially in different regions of the country.

Anywhere you have distribution as part of your offer you have similar levels of complexity, but this time involving distributors, channel, service agents, area managers. And of course it can all change tomorrow with a buy-out or take-over. So complexity changes over time and there is a high need for a flexible system

Most traditional databases simplify this complexity by ignoring it. But in understanding markets understanding and using the complexity can identify entry-points and opportunities.

 

3. Structured vs unstructured data

Related to the tight-loose problem is that of structured data. If a database is too tightly defined you can't get what needs to be captured, but it it is too loose you have no analytical power.

What you are really looking for is to build structured data out of the unstructured information collected. To add structure you need to be able to classify and determine what information we want in a structured format.

But there is a problem. The more deeply you analyse the data, the more classification you want to include. Yesterday the CEO said that our delivery performance needed monitoring so we've started collecting delivery times. Today, having looked at the delivery times we realise that it's actually about delivery completeness - on-time in-full. A different piece of data is needed today to that captured yesterday.

In conventional database designs this means adding another field and increasing redundancy in the data, not to mention the complexity of rebuilding the database (again) and the problem of maintaining historic data in this new setting. Any system for marketing knowledge needs to be able to capture data, classify it into a structured framework, but be able to do this on-demand, flexibly and easily. Fortunately there are tools like Map-Reduce, and batch analytics and structured coding methods for improving data consistency.

 

4. Incomplete data

By its nature competitor intelligence is also incomplete. A typical relational database will be full of empty cells, or it adds to complexity with another table for less frequently used data, so noSQL type tools are best for storing this information so that data can be stored and catalogued without defining fixed fields, and allowing factors like commenting and qualification to help define the data. Not only that, we may be able to get some information for some competitors, but not for others as we are limited in what data can be found at a point of time. Ideally we will also be able to flag data we would like to have as in competitor intelligence systems, data is as-and-when and rarely complete.

 

5. Ambiguity

Prices, product codes, order numbers, invoices are all designed to a fixed format to avoid ambiguity. For competitor data, ambiguity is almost unavoidable. Take a simple example: Annual turnover. If you're comparing companies annual turnover you want to be able to compare like-with-like. But HP has a sales channel, Dell sells direct. IBM services form a large part of its services and Siemens bills in Euros. Seemingly simple terms can lead to ambiguous results. In capturing and classifying information care is need to define precisely what the data is that you are trying to capture, and to a certain extent, the quality of that data. This means that within the structure and complexity you need ways to control to reduce ambiguity without making the database too tight again.

 

6. Who captures, who sees, who uses?

The final challenge to be mentioned here is the problem of data collection and data sharing. Competitor data can be captured centrally, but there is also information from those in the field who know what competitor activity is happening in their area. So while it may be that someone centrally is also monitoring and collecting data, within the system there needs to be spoke and mechanisms to allow different people to capture information to the database, whilst still ensuring that the quality of the data is maintained.

Then having captured the information, those who would find it most useful are, again those people closest to the customer. But they want a micro-picture specific to their customers. Centrally, the company wants to draw all the information together: a macro-picture. And it may be that the central macro picture is only available to a small team of planners and managers. The system in use needs to be able to control who captures, who sees and who uses the data.

---

For help and advice on building customer, competitor or marketing knowledge systems such as our Cxoice Insights Platform contact info@dobney.com