Become a Member
Proudly Sponsored By
articles / advertising know-how and fearless opinions
IHAVEANIDEA.ORG > articles >  Why We Hate Pre-Testing

Why We Hate Pre-Testing

Posted on November 3, 2010 and read 5,210 times

Why We Hate Pre-Testing thumbnail

chriss Why We Hate Pre TestingChris Staples
Partner, National Co-Creative Director

If you’re a client who’s spending several million dollars on an ad campaign, you want some assurances it will actually work, right?

So you do a few focus groups to put your campaign in front of some “real consumers” at the concept stage to see what they think.

It all sounds like a reasonable way to ensure success- or at least reduce the risk of outright failure.

So why do so many campaigns that go through this pre-testing process still fizzle? Why do so many pre-tested ads feel over-processed and boring?

Writers and art directors would argue that pre-testing kills their best and brightest ideas- the kind of ideas that have a chance to become bona fide home-run hits.

I’d argue that pre-testing per se isn’t the culprit, but rather the kind of pre-testing that’s become routine at virtually every agency around the world.

In my next column, I’ll talk about a kinder, gentler form of testing we believe leads to better ideas and better results. In the meantime, let’s discuss focus groups- still the dominant form of pre-testing in most agencies (quantitative concept testing is another matter- and deserves another column).

Why do groups?

Even with the advent of digital and social media, clients still rely on focus groups to test multi-media campaign ideas at the concept stage. Groups are generally used for three reasons: To test comprehension. To seek out red flags. And to try to predict magic- i.e., determine if the campaign is memorable and likeable, with break-though potential.

Focus groups, though, tend to be a very blunt instrument, and are especially hard on ideas that are truly new and different. Why?

  • Focus groups assume people care. People at groups are forced to look at your campaign- after all, that’s why you’ve paid them to attend. Unfortunately, nobody’s handing out cheques to people in their living rooms. The sad fact is that the vast majority of advertising is simply ignored. Groups can’t predict whether people will view the ad- and can actually give clients the false impression that average people actually care about their new toothpaste or mutual fund.
  • Focus groups are inherently negative. When you pay someone for their opinion, they’ll give it to you- and I can guarantee they’ll feel like they have to mention at least one thing they find wrong. It’s human nature. People have seen enough examples of groups in popular culture to know they’re being paid to be critics. This is their chance to be Siskel and Ebert and finally have their say.
  • Focus groups are subject to group-think. Despite their best efforts to control bias, groups are easily (and invariably) influenced by alpha dogs. This person is not always the loudest voice, but simply the most persuasive. People also tend to give answers that reflect how they want to be perceived by the group, rather than what they truly think. (For more on the perils of group think, Blink by Malcolm Gladwell should be required reading for every marketer.)
  • Focus groups reward the tried and true. Truly new ideas sometimes feel uncomfortable at first. It’s far easier to comment on an idea when you’ve seen something like it before. Which is why so many TV hits- from Seinfeld to All in the Family- bombed in pre-testing. As Gladwell puts it, “The problem with market research is that often it is simply too blunt an instrument to pick up the distinction between the bad and the merely different.”

As a result of all of these issues, small things can become red flags in groups that kill promising ideas. All the sharp edges can be whittled away until you end up with something that offends no-one, but also delights no-one. Which is why GM cars are so bland. And why Apple doesn’t do focus groups.

What about magic?

As far as trying to predict a truly magical idea, groups are completely ill-equipped. It’s like asking someone to look at three recipes for chocolate cake and pick the winner. To really judge, you need to taste a slice.

Many of the things that make advertising magical are intangible and come later in the process- music, performance, nuance. Without the “icing,” groups tend to pick ads that work best on paper. Which are invariably the most obvious and familiar ideas. The kind of ideas that rarely become home-run hits.

Another problem is that once a script or comp has been blessed by a group, there’s a tendency to want to follow it to the letter during production. This, too, prevents magic from happening. Most of our most memorable TV spots didn’t start out that way at the script stage- the magic came from tweaks during casting, on the day, and in editing. For some of our spots, we shoot up to a dozen different versions before finding just the right one.

Testing digital or social ideas at the concept stage is even more problematic. User experience is key here- and virtually impossible to test except in finished form.

For all of these reasons, we’ve never used focus groups to pre-test unfinished creative concepts at Rethink. Not once in 11 years.

We tell potential clients right from the start that we have a better way. A kinder, gentler way to pre-test for comprehension and red flags without killing big, ground-breaking ideas.

It’s called Peer Review- a process we didn’t invent, but have refined and perfected over the last 15 years. It’s helped us give our clients peace of mind that their advertising will actually connect with people. And it’s the subject of my next column, “Why we love pre-testing.”

Until then, what’s your perspective on focus groups? Are they the best way to pre-test ideas? Or is it a blunt instrument that kills and neuters truly original thinking?




Moving Millennials thumbnail Moving Millennials
Thoughts from a Cannes Creative Effectiveness 2013 Jury member thumbnail Thoughts from a Cannes Creative Effectiveness 2013 Jury member


Agency Profile: Advico Y&R thumbnail Agency Profile: Advico Y&R


Copyright © 2001-2017 IHAVEANIDEA inc. All rights reserved. No material contained in this site may be republished or reposted.
IHAVEANIDEA™ is a trademark of IHAVEANIDEA inc. Terms and Conditions | Privacy Policy

Copyright © 2009 ihaveanidea inc. All rights reserved.

No material contained in this site may be republished or reposted. Terms and Conditions | Privacy Policy