Principles of Qualitative Ad Testing

1. People want advertising and accept ad testing!

In our culture people’s relationship to advertising has changed: Advertising is no longer generally considered to be ‘indecent’. If typical consumer statements are to be believed, then advertising is still annoying, intrusive, boring and excessive. However, many people have changed their attitude to advertising: People want advertising! Not only do they accept advertising, they like it and are seriously interested in it – something that is down to much more than pure habit. People actively seek out advertising, e.g. by downloading ads, by turning on the television prior to the show they want to watch, etc. 

This very positive attitude towards advertising is also revealed by advertising research: Growing numbers of companies sense that people are prepared to actively participate in the mutual development of better advertising. They voluntarily evaluate advertising in Internet forums - but are also perfectly willing to get involved in lengthy research processes. When people are given a voice in advertising development they frequently enjoy attributing advertising with a ‘manipulative’ character - and making it more effective. 

Many providers – and nowadays even business consultants have got in on the act - have discovered a lucrative ‘sub-branch’ to their businesses in the form of qualitative research. As before, their principal business consists of quantitative ad testing. But questioning a few people and capturing a couple sound bites is seen to represent an uncommonly attractive add-on – hey presto the research is done. As an agency manager once said: "If all else fails, I’ll open up a market research institute, anyone can do that!" It’s a common misconception: often people who have conducted a group discussion or perhaps even only observed one, believe themselves to be qualitative market researchers on the basis of this ‘experience’.

2. Consumers want advertising and advertising research - but this does not always impact on quality!

Just because people want better advertising, does not automatically mean it is going to get better - and that is exactly what people are experiencing. Advertising is unpredictable, intrusive and assailing at those points where consumers voluntarily take it in. Flooded mailboxes, screaming radio ads, hard to read print-ads, pointless sequences in TV-/radio blocks etc: People do not know when, nor what is being offered, nor whether what is being offering fits the current context or whether the contents possess a meaningful context. But people won’t have their advertising spoiled for them - but actively seek it out when it suits the mind state they are in. The still frequent ‘criticism of advertising’ primarily refers to the way advertising is placed and how it ‘assails’ consumers. Consumers evaluate advertising content in a differentiated way: They sense exactly whether advertising really affects them, whether brands convey authenticity and whether they address relevant triggers to usage: These ‘quality criteria’ for advertising can be finalized using good qualitative advertising research.

Just as the desirability of advertising does not automatically lead to better advertising, the increased acceptance of advertising research does not necessarily lead to improved quality of research. For a long time now not all qualitative advertising research has possessed real qualifications – neither in terms of explorative methods nor regards analysis of findings. Those that think qualitative research is ‘easier’ than quantitative are vastly mistaken. Good qualitative research is just as difficult to learn – and just as hard to find as really good advertising. Unfortunately, in the recent past increasing numbers of ‘biotopes’ have sprung up in qualitative advertising research that can jeopardise the quality of research.

The surfeit of advertising is paralleled on the research side mushrooming numbers of techniques and theories: More and more, providers are embellishing themselves with a plethora of qualitative methods: there is hardly anyone who does not offer depth psychological groups and interviews coupled with psychological and ethnographic interviews – and if speed is of the essence they also provide half-open interviews or telephone interviews, not forgetting the mega trend towards online surveys which is now happily entering qualitative terrain.

Just as aimlessly as advertising selects its channels – type of media, why flyers, below the line etc. - qualitative advertising research rarely asks why a particular procedure has been applied. Here the only thing that seems to be important is that the procedure is new or different. Not infrequently would it make as much sense to concentrate on targeting a lesser number of advertising channels that reach consumers in the right (recording) mind states; just as it would to concentrate on specific techniques that consistently arise out of the research question.

It is now possible to recognise good quality advertising research? Is it possible to simply ‘sense’ - as consumers often instinctively know which advertising touches them - or can tangible criteria for good advertising research be indentified here?

3. Advertising’s dual mechanism – and the two levels of qualitative advertising impact research

Now lets take a quick look at advertising:

Advertising works on various levels. An ad’s impact can only be fully understood when its dual mechanism is revealed. That is, the relatable open ‘story’ – the cover Story – it communicates and a secret, covert, ‘story’ or the supporting mind state it contains – the impact story. People experience these different degrees of impact very clearly - even if they are not always able to clearly articulate them, they always clearly reveal initial reactions to an advert.

"I do not know exactly what it is exactly, but something bothers me about it." Or vice versa. "They are just all clichés, but I still like it somehow": What at first glance looks the same does not necessarily have to have the same effect.

The general function of the cover story is - for good advertising – to deliver specific information about the product and brand. It is at the forefront of events and possesses a certain psychological stringency - but does not necessarily have to be ‘sympathetic’, ‘loud’ or ‘active’.

The impact story conveys the supporting mind state and underpinnings of the TVC. A good supporting mind state is clearly ‘palpable’ but tends to work in the background – i.e. it cannot necessarily be retold immediately.

An example of successful, but ‘unsympathetic’ advertising is provided by one of the first campaigns for Always sanitary pads: A uptight, very strict and almost sterile-looking lady – a strict governess type - pours a blue liquid onto two towels - one leaks while the other absorbs everything. She raises an eyebrow – insinuating: “as if I didn’t already know that” – she turns around and walks stiffly from the screen. Women do not want to indentify with this woman - but the campaign nevertheless achieved good sales. And this is mainly due to the impact story.

The impact story primarily delivers a motive relevant foundation: For women the qualities of sterility, being strict and uptight are not desirable – but in a figurative sense for a sanitary pad they mean: absolutely leak proof (= uptight), does not smell (= sterile) and is reliable (= strict). And, of course, they guarantee that nobody will notice any embarrassing red marks – something the TVC has already drummed in. The protagonist is a symbol for the perfect pad - and from a psychological point of view should not be allowed to become more open or ‘sympathetic’. She fulfils her purpose wonderfully as she is - the pads sold extremely well.

Similar to the advertising itself, two levels in advertising research can also be differentiated: in qualitative advertising analysis the ‘cover Story’ is at the survey level, while the impact story is mostly found at a concealed level. Currently qualitative research differs less via the type of survey than via the type of and quality of the analysis!

Because what is true for advertising is also true for advertising research: Things that seem the same are not always the same. Anyone can for instance offer depth interviews, even if they lack the tools to do so and even if they have never actually conducted one before. But even where researchers are qualified to conduct explorations, ultimately study findings depend on the underlying analytical theory. Researchers should always be able to state according to which theory analysis is taking place.

4. People evaluate the advertising cover story according to the impact story - good researchers evaluate the phenomena that have been revealed according to their theory.

What effect the cover story has on the people, actually depends on the story’s impact: As the foundation to a means of communication however, the impact story is composed of three dimensions: the respective trigger to product use, the secret logic of the brand and the broader overarching themes that are being targeted (thematic valence).

Beyond the triggers and product relevance, advertising especially touches people when these culturally relevant issues are touched upon in the background.

What culturally relevant topics exist at present? To understand this, you have a quick look at the yearning for personal individuality that marked the 90s. Our desire to individualise ourselves occupied us to the extent that we forgot about the world around us for a while. Although we might have repeatedly complained about the loss of values, moral decay and lack of social cohesion, we never really actively did anything about it. On the contrary, we revelled in the ‘me society’.

With 9/11, the conflicts in Afghanistan and Iraq the mood changed: We board a train – or plane – and feel slightly nervous at the sight of people who look as if they could be Muslim. Our sense of fear has changed our perspective on things. We suddenly realize that there is still something of value: life itself!

Individualization and focussing on the self gave way to reflection on the essentials of life. Currently these essentials are in the eyes of almost all our respondents: Life and – one may hardly dare to type it - love!

Of course these issues change in the course of cultural development - they need to be repeatedly reviewed and re-explored. Using the Plus Viva Vital campaign with Kai Pflaume it is possible to reveal the central influence of cultural issues. Through a successful impact story, people evaluated a relatively simple ‘cover story’ extremely positively - and bought the products to an extent not predicted by Plus.

As the impact story influences evaluation of the cover story, so analytical theory used in advertising testing influences evaluation of the collected phenomena. Only by using a theory does the link between the phenomena and the results become clear. Just as only structural analysis can lead to a clear understanding of the impact story, the cover story and the spontaneous response to an advertisement.

The problem is to be found here: Analysis often fails to relate to the explorative method. The perceptual representation of qualitative surveys provides a very simple example: The object of a qualitative survey cannot be the measurement of frequency – since the concept is completely unsuited to this. This kind of ‘counting’ lacks statistical representativeness and is thus problematic and does not enable predictions to be made about the effectiveness of the advertising.

Qualitative advertising research however always concerns itself with how advertising works - indeed it is almost indifferent about how many people like or dislike the colour, the protagonists or the tonality of the advertising medium. Instead it makes consumer responses to the advertising comprehensible and thus delivers ‘emotional representativeness’: Why is something rejected? Does this concern a structural problem that is hindering advertising effectiveness? Or is the opposite true – as in the Always campaign – and the advertising provides a symbol that drives purchase?

Basically the nature of the questioning needs to be pre-defined by the analytical theory: a depth psychological survey is decided on due to the depth psychological theory behind it.

Qualitative advertising research which pursues a strong correlation between survey and analytical theory hardly need bow to the accusation of arbitrary interpretation. But because this is rarely the case, quantitative tests tend to provide the benchmark for many, they provide the desired security and reliability - qualitative are subordinate, ‘subjective’ and in case of doubt unreliable candidates.

Different ‘interpretations’ are only obtained where a theory has not been used in the analysis or where numerous theories have been pursued simultaneously – something which an array of providers see as ‘progressive’. In the first case, the results either depend on gut feeling or the evaluator’s personal experience. In the second case, the supposedly appropriate theory is consulted.


Ad testing that is accompanied by qualitative research – the
rheingold message tuner™ for instance – reveals findings at different levels: It takes in rational as well as emotional aspects, takes spontaneous reactions seriously (Cover Story) and aids further structural analysis of initial reactions and explanation of depth psychological impact of communication (Impact Story). Such an approach also explains why initial responses to an advertisement do not always have to be positive – although the advertising may still be highly effective advertising and relevant.

Theory-driven ad testing provides yet another added value for advertisers: Qualitative and depth psychological insights can be compared with quantitative results using the rheingold messageTuner™. Thus, they become complementary and are no longer contradictory. Both approaches pick up on the cover story, while the quantitative approaches primarily measure how an approach such as rheingold messageTuner™ helps to understand these reactions: Why do some many people find the protagonists unsympathetic? Why are they buying the product anyway? What does the supporting structure behind an idea look like? How can it be improved?

Depth psychological analysis via the rheingold messageTuner™ therefore offers a real benefit over purely quantitative surveys - and helps to tie into success and to avoid failure, since it is able to identify specific criteria for successful advertising material.

This article first appeared in Planung&Analyse, edition 6/2007, and is also available from rheingold as a special edition. 

© 2015 rheingold