Advertising testing

Advertising basics and direct response measurement

Advertising has a dual role. It creates sales, and it is used to increase consideration of the brand or product. Measuring sales and response is the simplest method of advertising testing and is very easy to do with online marketing looking at click-thru, sign up, revenue generation etc. For off-line brands these measures often are not available.

The aim of advertising testing is to improve advertising performance. For online ads, different concepts or executions can be tested using what is know as A/B testing. Version A of an ad is shown to one audience and Version B shown to another audience.  By measuring which leads to the best response the ads can be refined and improved, though responses may be low - under 1% at times. For advertising with more variables, statistical experimental designs can be used.

 

Mass media advertising measurement

While online-marketing allows direct tracking of an advert to an individual, mass-marketing advertising such as posters, television commercials or radio advertising (or PR or sponsorship) works in aggregate and it is more difficult to separate out practical day-to-day changes in response.

As the costs of television and other mass media advertising are high, getting the advertising right before launch is extremely important.

The first stage is to test the advertising using qualitative research to check what consumers think of the advert content before launch. Research often uses focus groups to get feedback about what consumers think of the advertising, what they like or don't like about it and how they think it would affect their behaviour.

The test ad is normally shown as part of a pack to help limit yea-saying and to try to get a fair understanding of what consumers think. This feedback is then used to refine the advertising. In agile research, this process of test and refine can be carried out over several iterations.

 

Pre-testing

Once the advertising is defined, the second type of research is quantitative pre-testing. Research is used to show the advert to a sample of consumers and the advert measured for factors such as likeability and recall, in order to develop a forecast for advertising effectiveness before spending money on a big launch.

An additional method of pre-testing, again to prevent wasting money, is to test the advertising in a smaller region or area prior to rolling out more widely. Research would measure actual live response, and the advertising is only rolled out if it meets certain performance metrics.

 

Pre- Post- Test and Control testing

Post-launch, the main testing of advertising effectiveness is measured using a pre- and post- statistical trial, using matched samples. A pre- measurement is taken before the advertising and sets a benchmark. The post- measurement then re-measures these details. Changes are then assumed to be attributable directly to the advertising campaign.

In practice this still might not be sufficient to measure a true effect. External events can also cause the post- measurement to change. So for full measurement, a full pre-, post-, test and control trial is used.

The pre- and post- measures are divided into two areas (typically geographic, such as different cities) - one larger area, the test area where people get to see or hear the advertising and a smaller area - the control - where the advertising is not shown. From this it becomes possible to isolate out the advertising effectiveness from other factors by looking at how measurements changed in the control area compared to how they changed in the test area.

For mixed-media campaigns (eg TV, poster and radio) multiple test and control areas for different media can be used - eg some with radio, some with radio plus poster and so on. With appropriate design and statistical analysis the impact of each media element can be estimated (generally media has a cumulative effect - that is combined has a bigger effect than either one thing or another separately).

 

Social media metrics

An addition to formal statistical testing are social media metrics to measure mentions, likes, sentiment and other forms of interaction that occur during the advertising period. Social media is monitored automatically and then systems like artificial intelligence used to estimate factors like sentiment (whether the mention is favourable or unfavourable). We do not do social media monitoring, but can advise on the most appropriate method and build this in to a complete advertising effectiveness research programme.

 

We design research to improve advertising effectiveness and performance, using appropriate statistical and sampling design to test which messages and which media works best for your brand.

For help and advice on ad testing and measuring advertising and marketing effectiveness with research contact info@dobney.com