Market research basics

Types of market research

Market research surveys can mostly be divided in two - quantitative research and qualitative research. A third category is observation studies, which was less common, but is increasingly use for social media monitoring and tracking online behaviour.

Quantitative research (quant) is about getting the hard measures of a market - market share, how many people think... , how many people saw our advertising, how many people would buy... . Qualitative research (qual) is about the softer issues exploring why people do things or think the way they do. These types of survey are usually complementary - you might explore the reasons why people buy in qual, then measure how important these reasons are in quant.

Quantitative surveys mean getting people to answer fixed questions in questionnaires. These questionnaires can be completed online, on mobile devices, over the telephone, face-to-face, through the post, in-house, on the street or 'in-hall' or any combination. Because the objective of quant research is measurement, it is important that the questions and questionnaire is standardised. Changes to wording, or order of questions can dramatically change the measurements. For self-complete style online surveys, where individuals read the questions themselves, this is not a problem. But where there are interviewers on the phone, or face-to-face, good quantitative interviewers have to be trained not to stray from the script.

Since what you ask is what you measure, good questionnaire design is extremely important. For online and self-complete surveys wording and method are even more important because there is no interviewer to help or support someone through difficult, confusing or unclear questions. A survey will go to a cross-section of the population, some of whom are experts and some who will know nothing about the topic. The questionnaire has to work for all types so it has to be clear and jargon-free, trying to reflect the respondent's view of the world and not just the clients.

Qualitative surveys are about exploring an issue with people. There are no fixed questionnaires and interviewer use their interviewing skill to draw views and opinions from people using a discussion guide (there are some differences in the US and UK views of how qualitative research is conducted). There are still two main approaches to qualitative research - depth interviews (one-on-one interviews) and the focus group (aka discussion group) where the group dynamics means that individuals spark ideas and discussion off each other to explore a topic. It is important that the qualitative interviewer doesn't bias the discussion and lets the interviewee(s) describe things in their own terms - how something is said can be as important as what is said. Qualitative work is typically carried out in person, but it can be conducted over the telephone or over the Internet. Although qualitative research is done online, face-to-face remains extremely important to study how people react, not just to observe their words.

Both quantitative and qualitative research are based on the notion of sampling to identify who to talk to and the idea of the interviewer as an unbiased observer or collector of information.

---

Samples

At the core of market research is the idea of a sample. A sample is a group of people drawn at random from all those people who are in your chosen market. By "drawn at random" we mean that each person in the market has the same probability of being questioned as any other. For instance if you had a database of customers and wanted to select customers "at random", you might put this list in alphabetical order and take every nth person.

Some care is needed in choosing how to select a sample. If you do not make the sample random, you risk introducing bias and so get misleading results. For instance if you choose 1 in n people from a telephone book, you miss out all the ex-directory phone users, who are more likely to be the wealthier individuals. Typically a researcher will use their judgement and experience to trade off the quality of the sample against the cost or difficulty of contacting the right people.

Many people now use online panels to get access to consumers. Though this makes life easy, anyone carrying out research still needs to ensure that the panel reflects the market to an appropriate level of accuracy.

Samples are crucial to the potential accuracy of the research. Where the sample is poor, for instance a simple convenience sample - that is a group of people that were easy to get hold of and collect data from, for instance from an online discussion forum - there are likely to be hidden biases in the data. These type of samples may still be sufficient for generalised feedback and able to give good insight, without necessarily being representative of a particular market or group of customers.

---

Sample size

If you took a random sample of UK voters and 25% said they would vote for the Blue party you would want to know how accurate this was. Random samples are subject to known statistical laws which describe how accurate each sample is likely to be.

If your sample was 10,000 people you would be 95% confident that this reflected the views of all voters 0.85 percentage points. In other words there is a 19/20 chance that the "real" share of the vote for the Blues would be 24-26%. (95% is the usual level of confidence quoted, but you can estimate accuracy to any confidence level - 99 in 100, 999 in 1000 etc).

Because contacting people and making interviews costs money, there is a trade-off between sample size and required accuracy. If in the example above the sample size was different you would have different levels of confidence.

If you look at the table above, there is practically no benefit in going from a sample of 5,000 to a sample of 10,000 yet your costs would have doubled. In fact, accuracy doubles every time you quadruple your sample so accuracy can be very expensive.

Nonetheless, the sample size you use depends on the type of survey you are trying to carry out. If you are trying to assess the impact of small price rises or changes to attitudes after an advertising campaign, or changes to satisfaction then you may want real fine accuracy (to 2%). But if you just want a "yes/no" decision you can take a broader view of results and accuracy of 10% may be entirely suitable. For most quantitative market research studies a common rule of thumb is that you aim to talk to 100 people for each significant subgroup in your market.

Note that the sample size is pretty much independent of the size of the market you are looking to research (unless there are very very few customers - under 500). So the error in researching a market of 100m customers is almost exactly the same as in researching a market of 100,000. You can't cut the sample for a smaller market.

For qualitative research, the aim is not measurement but exploration. Consequently sample size is less of an issue. Of more importance is to get a complete picture of how a market works. However, clearly if you only talk to 5-6 people the usefulness of the research will be less than if you talked to 12-16 people.

---

Quantitative questionnaire design

Good questionnaire design makes it easy for the person answering to answer and is clear, unambiguous and doesn't lead the person answering in one direction or another.

Common problems are to ask questions that are too difficult to answer (eg "Would you buy X in five years time?"), or unclear in their meaning ("Do you think relationships will change the world?"), or have double meanings ("How hot was your spicy pizza when you ate it?"), or contain two questions ("Do you think corrupt politicians and charity workers should be eradicated?"), or lead in some way ("Do you use your air-polluting gas-guzzling car less often than you use to?"), or fail to address the problem from the interviewees point of view (ignoring key service elements is common).

The order in which questions is asked can also be important. The adverts that can be remembered without prompting (ie spontaneously - Which ads can you remember seeing) are more likely to be top-of-mind than those remembered with prompting (Which of these ads can you remember seeing). Questionnaires, therefore typically run from the very general to the very specific as this also helps the flow which reduces the time taken to complete the interview. Consequently, interviewers are trained not to lead the interviewees and to probe to get fuller answers (Why is that? Could you clarify that for me? Could you explain), without prompting (Did you say Mustang or Mazda?).

A survey that is too long will bore the respondent and reduce your response rate, so what do you really need to know to answer the business question? And what is the best question to find this out? A customer, for instance, would struggle to answer a question about what they would buy long into the future, but they could say what they would consider buying now.

Ideally all questionnaires should be piloted. To find that questions don't work after collecting responses from 1000 people is not very helpful. A good initial test, even before a pilot is to ask a member of your family to complete the questionnaire as this will typically highlight the most glaring of problems.

---

Qualitative research designs

The design of qualitative research depends on the subject and the audience and the skills of the interviewer/moderator. If the subject is personal or private then depth interviews will be better than group discussions, if the subject is more widely considered and discussed then focus groups may be best as the stimulation for discussion comes from the group members. For certain types of respondent (eg business people), a face-to-face group discussion may be not be feasible although telephone conferences (possibly with video) or online groups may be alternatives.

Interviewers themselves are also important in the qualitative process as the level and quality of probing and questions will depend on the interviewers understanding of the subject at hand, in order to identify areas that need clarification or further exploration and to provide a point of reassurance to draw more out of the interviewee. It is difficult to discuss the issue of hedging with a financial director unless you recognise that this might have something to do with exchange rate risks and to have some understanding of what the implications of hedging could be.

Often in qualitative research, it can be difficult for the interviewee to express themselves adequately in words particularly where you are looking at the emotional appeal of something as opposed to its practical features and benefits. Researchers use a lot of games and prompts to draw out different views. For instance in projective techniques researchers might ask someone to say "If X were an animal, what type of animal would it be?"and then explore the reasons behind these views. Most researchers have a kitbag of tools and techniques that they find useful to help capture views and responses. Typically the closer you are to the creative design arena, the more exotic these techniques become. But for most work, plain vanilla qual with an interviewer who knows the subject area (but doesn't lead the respondent) are to be advised.

Group discussions add another twist as the interviewer in the guise of a moderator has to ensure that the group captures and includes the views of all the participants. There are dangers that one or two voices pre-dominate negating the benefits of the group environment.

A key element of qual research is that you should record the discussion and then transcribe notes later. Relying on memory is often not good enough and listening to the discussion guide a second time can change your opinion of what the interviewee really meant. Typically qualitative research is reported as finding supported by quotes.

---

Doing it yourself

Typically the main problem with DIY is not so much not being able to do it, but not having the experience of what works and what doesn't. For more advanced subject areas, design expertise can be crucial. Having said that if the first run is not crucial, then the experience will be really helpful subsequently whether you do-it-yourself later or buy in expertise.

The key steps are:

1. Select the type of research you need (Qual/Quant)
2. Choose how you want to get in contact/recruit the relevant people
3. Design a questionnaire/discussion guide
4. Recruit the relevant people/interview them/send out the questionnaire
5. Amalgamate and analyse the results.

The recruiting/interviewing task is the biggest. There are many companies that will just do fieldwork for you or arrange interviews if you provide them with lists. For internet surveys these usually come with analysis tools, but there are also many DP/transcription companies that will process your completed questionnaires or transcribe your interviews for analysis. All these steps can be carried out by hand, but the biggest issue is normally time which is where sharing the workload will help.

In terms of cost, most market research is charged on a time basis plus a management/design fee. Surveys online are obviously much cheaper than phone or face-to-face surveys. Online, it's just time for the person answering - for which they usually get some reward or incentive. There are no interviewer costs, just the costs to access a panel, and host the survey online. The data will mostly be processed automatically except for advanced techniques that require specialist analysis and the cost of design and programming.

Face-to-face and telephone studies are more expensive. For instance for a population study of 1000 people,  might require 15 minutes for the interview, but 20 minutes finding/contact time, with an additional 10 minutes for cleaning and processing each questionnaire - so 45 minutes per interview or 750 hours of time (100 days), on top of which would be added time for questionnaire design, production and dispatch, interviewer briefing and management, creation of tables, analysis and presentation. For physical surveys these elements should add about 15-20 days, although at a higher daily rate.

For more complex studies into difficult to reach groups, incidence rate or contact time can have a far larger impact on the cost than the actual questionnaire length. Studies with businesses for instance can be extremely hard to get people to take part. In addition, questionnaires that are too long put respondents off research or demotivate interviewers and typically need high levels of incentives to encourage completion. For on-line surveys, sample cost is much lower, so the proportion spent on design and analysis is usually a bigger part of the final bill.

Qualitative research is charged on a similar basis to quantitative research, but as sample sizes are smaller it is typically less expensive. However although recruitment is far less expensive, the interviewer is more expert and interview and analysis time and costs are typically higher reducing the difference in cost between small quant and larger qualitative work.

For help and advice on carrying out market research contact info@dobney.com