Mass Media Research: An Introduction - 9th Edition
Research in Advertising
A version of this article appeared in Marketing Research: State-of-the-Art Perspectives by Chuck Chakrapani (Ed.), American Marketing Association, 2000. Do not use in any form without permission from the publisher.
Roger Wimmer, Ph.D.
Advertising messages are everywhere. There are advertisements on radio, television, billboards, newspapers, magazines, the Internet, matchbook covers, gas pumps, shopping carts, clothing, and on and on. It is probably safe to assume that those who create, sponsor, and use the messages would analyze such a pervasive medium. It is also probably safe to assume that because of the numerous types of advertising approaches there are numerous research methods to analyze the messages. However, before we get to a discussion of the research in advertising, it makes sense to find out what we are planning to investigate. In other words, what is advertising?
(Note: Due to space limitations, I assumed that the reader has a basic understanding of qualitative and quantitative research methods and elementary statistics. If a term or concept is used that you do not understand, immediately consult the Internet or the references listed at the end of the chapter for more information. In addition, a countless number of sources are available on the Internet for virtually every term or concept discussed in this chapter.
A Collection of Advertising Definitions
A quick search for a definition of advertising immediately reveals that the word is not a simple one. It seems as though there are as many definitions of advertising as there are people who work in the industry. And it seems as though there are as many definitions as there are approaches to conduct research on the messages. Just look at some of the variety of definitions available for advertising available from The University of Texas at Austin, Department of Advertising web site.
First, there are definitions that equate advertising to sales:
“The simplest definition of advertising, and one that will probably meet the test of critical examination, is that advertising is selling in print.” Daniel Starch, Principles of Advertising, 1923, Chicago, IL: A.W. Shaw Company, p. 5.
“Advertising is selling Twinkies to adults.” Donald R. Vance (no source listed)
“Our job is to sell our clients’ merchandise . . . not ourselves. Our job is to kill the cleverness that makes us shine instead of the product. Our job is to simplify, to tear away the unrelated, to pluck out the weeds that are smothering the product message.” William Bernbach, quoted in Bill Bernbach said . . . (1989). DDB Needham Worldwide.
Some definitions of advertising suggest that the method is simplistic:
“Advertising is what you do when you can’t go see somebody. That’s all it is.” Fairfax Cone (1963), quoted in James B. Simpson, Contemporary Quotations. (1964). Binghamton, NY: Vail-Ballou Press, p. 84.
“Advertising is, actually, a simple phenomenon in terms of economics. It is merely a substitute for a personal sales force - an extension, if you will, of the merchant who cries aloud his wares.” Rosser Reeves, Reality in Advertising. (1986). New York: Alfred A. Knopf, Inc., p. 145.
“Advertising is salesmanship mass produced. No one would bother to use advertising if he could talk to all his prospects face-to-face. But he can’t.” Morris Hite, quoted in Adman: Morris Hite’s Methods for Winning the Ad Game, (1988). Dallas, TX: E-Heart Press, p. 203.
Some definitions claim that advertising is an art, not a science:
“I warn you against believing that advertising is a science.” - William Bernbach, quoted in Bill Bernbach said . . . (1989). DDB Needham Worldwide.
“Advertisements may be evaluated scientifically; they cannot be created scientifically.” Leo Bogart, quoted in Randall Rothenberg, Where the Suckers Moon: An Advertising Story. (1994). New York: Alfred A. Knopf, p. 110.
“Advertising is fundamentally persuasion and persuasion happens to be not a science, but an art.” William Bernbach, quoted in Randall Rothenberg, Where the Suckers Moon: An Advertising Story. (1994), New York: Alfred A. Knopf, p. 63.
On the contrary, there are definitions that stress the importance of research in advertising:
“The most important word in the vocabulary of advertising is TEST. If you pretest your product with consumers, and pretest your advertising, you will do well in the marketplace.” David Ogilvy (1963), quoted in Stephen Donadio, The New York Public Library: Book of Twentieth-Century American Quotations. (1992). New York: Stonesong Press, p. 70.
“Advertising people who ignore research are as dangerous as generals who ignore decodes of enemy signals.” David Ogilvy, Ogilvy on Advertising. (1983). New York: Crown Publishers, p. 158.
Some definitions are somewhat pessimistic:
“Advertising is legalized lying.” H.G. Wells, quoted in Michael Jackman, Crown’s Book of Political Quotations. (1982). New York: Crown Publishing Inc., p. 2.
Finally, some definitions suggest that advertising is a form of communication:
“The sole purpose of business is service. The sole purpose of advertising is explaining the service which business renders.” Leo Burnett, quoted in 100 LEO’s, Chicago, IL: Leo Burnett Company, p. 30.
This final definition will emerge as significant later in this chapter. Keep this definition in mind.
This small sample of definitions shows that advertising is defined in a variety of ways, and this suggests that research in advertising be equally as varied. Let’s take a look at some of the current approaches to research in advertising.
Approaches in Advertising Research
Because there are so many types of advertising, we should expect that there are an equal number of approaches to analyzing advertising. This expectation is verified by a review of any advertising textbook or manual. Let’s take a look at some of the methods that are used to conduct research in advertising.
Probably the most prevalent type of advertising research falls under the broad category known as copy testing. While there are numerous research approaches used in this category of research, the approaches usually focus on such things as layout, design, color, narration (voice-over), music, illustration, size, length, and more. (To get an idea of the range of copy testing or message research, conduct an Internet search for “copy testing.”)
As Wimmer & Dominick (2000: 347) note:
Copy testing refers to research that helps develop effective advertisements and then determines which of several advertisements is the most effective. Copy testing takes place at every stage of the advertising process. Before a campaign starts, copy pretesting indicates what to stress and what to avoid. Once the content of the ad has been established, tests must be performed to ascertain the most effective way to structure these ideas. For example, in studying the illustration copy of a proposed magazine spread, a researcher might show to two or more groups of subjects an illustration of the product photographed from different angles. The headline might be evaluated by having potential users rate the typefaces used in several versions of the ad. The copy might be tested for readability and recall. In all cases, the aim is to determine whether the variable tested significantly affects the liking or the recall of the ad.
One way to explain the variety of methods used in copy testing research is to consider the dimensions of persuasion discussed by Leckenby and Wedding (1982). The three dimensions, which are appropriate to copy testing research, are the cognitive dimension (knowing), the affective dimension (feeling), and the conative dimension (doing).
Advertising research in the cognitive dimension includes research about attention, awareness, exposure, recognition, comprehension, and recall (unaided and aided) of advertising. In other words, what do people (consumers) know about a product, service, concept, or phenomenon after being exposed to advertising messages? Researchers use a variety of methods to collect such information, including focus groups, physiological studies (eye movement, etc.), and consumer panels. In some cases, the research involves a pretest and posttest (measurements are taken before and after exposure to the advertising), while other research involves posttest measurements only.
Advertising research in the affective dimension typically investigates if (or how) consumers’ attitudes toward a particular product or service have changed because of exposure to an advertisement or an advertising campaign. The data are gathered in a variety of ways including focus groups, telephone studies, central location testing (large groups in an auditorium setting), and a variety of physiological measurements.
The importance of the affective dimension was emphasized by Walker and Dubitsky (1994), who noted that the degree of liking expressed by consumers toward a commercial was significantly related to awareness, recall, and greater persuasive impact. Indeed, several advertising researchers have suggested that liking an ad is one of the most important factors in determining its impact (Wimmer & Dominick, 2000: 352)
Advertising research in the conative dimension deals with actual consumer behavior, particularly buying predisposition (intent to purchase) and actual purchasing behavior. In buying predisposition research, consumers are asked about their probability of purchasing a product or service presented in an advertisement or campaign. In purchasing studies, actual sales are tracked after consumers are exposed to advertising. Sales information is available from private sector research companies such as A. C. Nielsen (see www.nielsen.com).
Research in advertising also includes a variety of other approaches including, but not limited to,
Psychographic studies to determine the attitudinal and behavioral characteristics of consumers so that advertising can be designed to target these characteristics.
Market segmentation studies to find out which types of people like and use which types of products and services. This information, often combined with psychographic data, is also used to create and develop advertising messages.
Audience size and composition studies (also known as reach and frequency) estimates how many consumers were exposed to a message, how many times they were exposed to the messages, and what type of people were exposed to the message(s).
Market share studies investigate a company’s share of a product or service before, during, and after an advertising campaign
Studies of competitor’s advertising to learn about the products and services competitors are offering and how the competitors communicate this information to their customers.
Popularity research (such as is published in USA Today) showing the television commercials consumers say they like the most (see www.usatoday.com).
Space limitations do not allow for a complete discussion of the steps involved in all of these advertising research approaches. This information is available from other sources. However, it is important to note that research in advertising must follow the same steps and tenets of scientific research that are used in any other area of research. See Wimmer & Dominick (2000) for additional information about the steps involved in scientific research.
Reliability and Validity of Advertising Research
Anyone who is interested in advertising research probably has two basic questions. First, is advertising research valid? That is, do the research studies test what they are supposed to test? Secondly, is advertising research reliable? That is, does the research consistently produce similar results? These questions are logical for any type of research and are not limited only to the area of research in advertising. As noted in Wimmer & Dominick (2000: 354-355), to test the assumption that advertising research does, in fact, identify advertising that does work well in the marketplace, the Advertising Research Foundation (ARF),
. . . sponsored a research validity project to determine which copy testing measures were effective (Haley & Baldinger, 1991). To begin, ARF selected five pairs of TV commercials. Since one of the ads in each of these pairs had already been shown to produce major sales differences in test markets, ARF researchers used the ads in a field experiment that included many common copy testing measures (see www.arf.amic.com for more information).
The experiment revealed a strong correlation between ads that copy tested well and ads that performed well in the marketplace. For example, measuring reaction to commercials on an affective scale (like/dislike) predicted the more effective ad of the pair 87% of the time. Top-of-mind awareness (unaided recall) measures correctly classified the more effective ad 73% of the time. Less effective were measures that asked respondents to recall the main point of the ad.
In addition, as Wimmer & Dominick (2000: 354-355) state:
The current trend in copy testing is multiple measures of copy research effectiveness. A technique called Advertising Response Modeling (ARM) provides a conceptual model that integrates several measurements to evaluate ad effectiveness (Mehta & Purvis, 1997). ARM differentiates between high and low involvement situations and includes measures such as recall, liking, buying interest, and brand rating. The technique highlights that different ads can be successful for a variety of reasons.
The available information about research in advertising verifies that there are numerous research approaches seeking a variety of answers. Yet, much of the current advertising research investigates a definition of advertising that may not be correct. However, before we get to that discussion, we need to look at some of the ways advertising research data are collected and analyzed.
Data Collection Methods and Statistical Analysis
Research in advertising uses the same data collection methods and statistical analysis used by all other areas of mass media research. Advertising research uses a variety of qualitative research and quantitative research approaches (Note: the difference between qualitative and quantitative research refers to how the data are collected. In essence, the difference relates to the flexibility of data collection. Qualitative research is flexible and allows researchers to modify immediately the questions respondents are asked, such as in focus groups. Quantitative research uses standardized or static types of measurement instruments where all respondents are asked the same questions. There is no flexibility in asking unplanned follow-up questions, or even questions that were not planned in the research design.)
Some of the types of data collection procedures include, but are not limited to:
Focus groups: controlled, small group discussions with dozens or hundreds of consumers selected for a specific reason, such as people who purchase a certain type of computer.
Mail surveys: respondents are sent questionnaires or other product purchasing tracking studies to their home or business.
Telephone surveys: respondents are called at their home, their work, or other location to gather data.
Online surveys: perceptions about advertising (print or electronic), evaluations, pretests and posttests can be conducted using the Internet, where respondents are recruited (or volunteer) to complete a variety of measurement instruments.
Disk-by-mail: respondents are sent a computer disk to their home or work. The disk contains various measurement instruments and respondents either sends the disk back to the research company or the responses to the instruments are forwarded via the Internet.
CD by mail or purchase: respondents answer questions contained on a CD they receive from a research company or the company that produces a product or service, or on a music CD they purchased from a retail outlet. The completed questionnaire is usually sent back to the research company or manufacturer via the Internet.
In-field surveys or observations: advertisements, or variations of the same advertisement, are tested on television or cable channels, or in different runs of print vehicles to determine if there is a correlation between awareness or sales and the version of the advertisement. In-field studies can also monitor consumer behavior on site after they are exposed to advertising messages.
Shopping center intercepts: respondents view and respond to advertising messages shown to them by a research company that recruits them in a shopping mall.
Central location testing: a large number of respondents, usually 100 or more, are recruited to a setting where they are exposed to various advertising messages at the same time. Pretest and posttest questions are possible, as well as almost any other type of data gathering procedure.
The type of data collection depends on the type of research conducted. If we want consumers to respond to new television commercials, the consumers must view the commercials. This means that a data collection method where consumers can view the commercials is necessary. However, keep in mind that new technology has expanded the ways in which consumers can view or analyze information (in this case, advertising). For example, prototype television commercials, radio commercials, billboards, or almost any type of advertising can be shown on Internet web sites, or on CDs and DVDs sent to a consumer’s home or business.
The types of statistics used to analyze advertising research are also no different that in any other area of mass media research. Space in this chapter prohibits a complete discussion of all statistics used in advertising research. Suffice to say that research in advertising uses all forms of statistics. Research studies may use simple descriptive or summary statistics such as frequency tables, or measures of central tendency such as the mean, media, mode, standard deviation, and Z-scores. Other research may use univariate statistics (single dependent measurement) such as the t-test, Chi-square, and analysis of variance, or multivariate statistics (multiple dependent measurements) such as factor analysis, cluster analysis, canonical correlation, and discriminant analysis.
Additional information about all of these statistical methods is available from the references listed at the end of this chapter.
A Problem with Current Definitions and Methods
The information up to this point is relatively straightforward: an explanation of advertising, types of research used in advertising, and how the data are gathered and analyzed. The problem is that all of the information up to this point is based on antiquated, incorrect, and/or opinion-based definitions of advertising. Most definitions of advertising attribute too much power to the procedure. A more realistic definition of advertising is needed, which means that a more realistic research approach will follow. (Remember the definition of advertising by Leo Burnett who said that the purpose of advertising is to explain a service that a business renders.)
Leo Burnett hit the nail on the head, and it is time now to conduct research in advertising that addresses what advertising really is, not what it is thought to be.
A New Approach
As mentioned earlier, research in advertising covers a wide range of activities such as copy testing, layout, composition, recall, sales increase/decrease, and more. If we look carefully at all of the types of advertising research, one fundamental concern is always present: is the advertising effective? This underlying goal seems logical. A lot of time and effort are invested in advertising, and it makes sense to find out if the message or campaign actually “does” anything. Does the advertisement sell anything? Does it increase awareness? Does it increase store traffic? All of these questions are valid, but they are missing the point of advertising. They are missing the point of what advertising actually does. This problem must be solved.
Several definitions of advertising were included in the beginning of this chapter, and it is clear from these definitions that there are a variety of definitions available—probably one for every person interested in advertising. However, two frequently used definitions of advertising that I have heard during more than 25+ years of experience in mass media research is that advertising is designed to sell something or increase store traffic. These two reasons are also the major complaints voiced by advertisers when their ads seem to have no effect. I have heard a countless number of advertisers complain that their radio (television, newspaper, magazine) advertising did not work because they did not receive more phone calls, or their sales did not increase, or that the traffic in the store was no greater than before they placed the advertising. “Advertising is a waste of money” is a typical response from these people.
All of these complaints are misdirected; the complaints do not relate to what advertising really is. Sales figures and store traffic are usually the deciding factors in determining if an advertisement or campaign was successful. The “end” is tested, but not the “means.” That’s where things need to change.
The underlying problem with most current advertising research is that there is no universally accepted definition of advertising. Advertisers, creative people, and researchers, among others, attribute too much power to advertising. They falsely attribute powers to advertising that it does not deserve. So what is advertising?
Defining Advertising - A Different Approach
The sample of advertising definitions included earlier in this chapter shows a broad range of perceptions of what advertising is supposed to be and what it is supposed to do. Some definitions say that advertising sells products, others say that advertising replaces a sales person, while others say that advertising is an art, not a science. The variety of definitions of advertising is confounding to a researcher. Which definition should be used to analyze advertising? Which definition is correct? Which definition, if investigated, will provide the most useful information to those who create and used advertising? To eliminate this confusion, it is best to search for another definition. We need to look elsewhere.
A good place to start is with an unbiased source, such as Webster’s Dictionary (1989:59), which defines advertising as:
[T]he action of calling something to attention of the public, especially by paid announcements.
We can also look at Webster’s definition (1989:59) of the word advertise:
[T]o make something known to; to make publicly and generally known; to announce publicly especially by a printed notice or a broadcast.
Notice that neither definition mentions sales, increased store traffic, art, science, or research. The definitions do not include cute analogies, metaphors, or pessimism. The definitions merely say that advertising is only communication. The only thing that advertising does is communicate a message from one source to another. If we accept this, we can proceed to how communication (advertising) should be investigated. First, we need some discussion about communication.
Again referring to Webster’s Dictionary (1989:266), communication is defined as:
An act or instance of transmitting; a verbal or written message; a process by which information is exchanged between individuals thorough a common system of symbols, signs, or behavior.
Do you see the similarity between the definitions for advertising and communication? Do you see that the words “advertising” and “communication” are interchangeable? Both terms are defined in similar ways—advertising and communication are defined as transferring a message from one source to another. In other words,
Advertising = Communication
Communication = Advertising
Accepting this definition of advertising makes the discussion about research in advertising much easier. We now have a goal for any research in advertising: analyze communication. That is, we have a goal to find out if the communication effectively transmits information from one source to another.
Is there a simple way to do this? The answer is “yes.” But before we get to the methods of research, we need to take one step backwards—a step that will help further define what it is that we are doing when we conduct advertising research.
What we need is more information about the term communication. A good place to start is with one of the masters of communication. Let’s step back in time, to the time of Aristotle, the ancient Greek philosopher. In his book, The Art of Rhetoric, Aristotle describes in great deal all of the elements of communication. One of his many conclusions about communication is that communication and persuasion are the same thing. In other words,
Communication = Persuasion
Persuasion = Communication
All communication is persuasion and all persuasion is communication. Taking the information that:
Communication = Advertising
Advertising = Communication
we can substitute a few words to produce the following:
Advertising = Persuasion
Persuasion = Advertising
We can even take this relationship one more step. In 1967, Watzlawick, Beavin, & Jackson developed a wonderful argument for the fact that we cannot not communicate. In other words, everything we do or say communicates a message. The message may be through verbal communication, nonverbal communication, or metacommunication. The bottom line is that no mater what we do we communicate a message. We always communicate—100% of the time. There are no exceptions to this rule.
Now, considering the fact that we cannot not communicate, let’s move one step further and put all of the relationships together:
Communication = Persuasion
Persuasion = Communication
Then . . .
Advertising = Persuasion/communication
Persuasion/communication = Advertising
Considering that . . .
We cannot not communicate/persuade
We conclude that . . .
We cannot not advertise
If adverting is only communication/persuasion, then how can it be analyzed from any other perspective? How can it be analyzed by investigating increased store traffic or increased sales? Quite simply, it can’t. As persuasion, advertising alone does not sell a product. Advertising alone does not increase store traffic. Yet, for years, the success of advertising has been based on the advertisement or the campaign’s ability to generate additional sales.
What happens if the advertising communicates to consumers that a manufacture/company has a bad product? What happens if the advertising communicates that the product or service is over-priced? What happens if the advertising communicates that a competitor has a better product? All these things happen. Sales do not increase and advertising gets the blame. For example, a company can hire an advertising agency to produce print and broadcast advertising that tells consumers’ that their product is over-priced. In the following weeks, sales for the product do not increase. Where does the blame fall? Not on the company that made the product, but on the advertising. The advertising did not work. The advertising did not “sell” the product. The advertising agency is then blamed for the problem, and the manufacturer might even look for another advertising agency (to then go through the same process all over again). This scenario is repeated thousands and thousands of times every year. Advertising is blamed for the failure of a product or service. The blame rarely, if ever, falls on the company that produced the product or service.1
This has been the fundamental problem with most research in advertising—it has focused on investigating things that advertising is not. It’s time to change the focus. But in order to do that, we need to know more about communication/persuasion. (Since these terms are interchangeable, only the word persuasion is used in the remainder of this chapter. Please note, however, that whenever the word persuasion is used, the word communication can be substituted.)
How does persuasion work? Since the time of Aristotle, many descriptions of how persuasion works have been presented. In fact, a discussion about the theories of persuasion is contained in almost any book on persuasion or communication. However, the simplest description is probably the Five Stages of Persuasion; the five steps that all people pass through in order to make a decision about anything.
Five Stages of Persuasion
The Five Stages of Persuasion are unawareness, awareness, comprehension, conviction, and action. As mentioned, these stages apply to literally any decision we ever make, from purchases of products and services, to decisions about dating, marriage, careers, religion, and everything else. Every human being goes through these stages before making any decision. This is a universal behavior. It is another rule without an exception.
The Five Stages of Persuasion is a simple process. A person is first unaware of a product, service, or idea, then moves to awareness after gathering (being exposed to) some amount of information. The person then passes into the comprehension and conviction stages after gathering (being exposed to) addition information. The action stage is reached when the person buys a product or service, or listens to a radio station, or watches a television program, or believes an idea or philosophy, or takes a specific action.
It’s that simple. To persuade anyone to take some kind of action or believe anything, that person must be taken through the five stages. The problem is that the process is not 100% successful.2 Many people are forever stuck in unawareness, awareness, comprehension, or conviction and never reach the action stage. In addition, some people may reach one stage but may actually go back to a previous stage. For example, a person may buy something as mundane as Hostess Cupcakes for years, then stop buying for one reason or another. This person, either from a comment by a friend or relative or exposure to an advertisement about Hostess Cupcakes may say, “I haven’t had one of those in years” and then once again enter the action stage by buying Hostess Cupcakes the next trip to the grocery store.
What do we know about the Five Stages of Persuasion? We know that:
Everyone must pass through the stages for every decision he/she makes.
Everyone passes through the stages at a speed unique to him/her. There is no universal timing to the process.
Not all people make it to the Action stage.
The only way to move people through the Five Stages is through repetition of the message. In most cases, people do not make decisions after only one exposure to a message. The process requires several exposures.
There are also several things we don’t know about the Five Stages of Persuasion. For example, we don’t know:
How many exposures are required to move a person through each stage.
How many people are in each stage of the process at any given time.
When people will move from one stage to another.
Why people move from one stage to another.
Where people are when they move from one stage to another.
We know that people must pass through the stages to make a decision, but we don’t know where, when, how, and why they will pass through the stages. Lucky for us that we understand that in order to move people through the stages, we must have repetition of the message. And that’s where advertising becomes so important. We repeatedly present advertising messages to consumers because we know they will make a decision only after repeated exposures. In addition, understanding this process changes the way decisions about advertising placement is made. If we don’t know where, when, how, or why people get to the Action stage, then it makes no sense to schedule or place advertising at a specific time or place. For example, grocery stores have historically included coupons for products in the Wednesday edition of the newspaper. Why is that? Do all people make decisions to buy groceries only on Wednesdays? The answer, of course, is no. But the practice of placing coupons in the Wednesday edition of the paper takes the position that the grocery store owner/manager knows when the store’s customers make a decision to buy. This is ludicrous.
There is no way to circumvent The Five Stages of Persuasion. There is no company or manufacturer that can slip around the process to sell products or services to consumers. There is no person or group that can slip around the process in an attempt to persuade another person or group to do something or believe in something. There is no parent who can slip around the process to get his or her child to do chores around the house. The process is universal. It is, as mentioned, a rule without exception. Therefore, the Five Stages of Persuasion should be (must be) the basis for all research in advertising. Any other approach is irrelevant to the definition, purpose, and goals of advertising. So how should we conduct research in advertising?
Advertising Research - A Different Approach
The Five Stages of Persuasion model is a simple approach to follow in communication. The Five Stages also relate very nicely with the simple three-step approach for success in any personal or business endeavor. The model for success in business as well as personal life is:
Find out what the people (or person) want(s).
Give it to them (him/her).
Tell them that you gave it to them (him/her).
Think about this three-step approach for a moment, particularly in reference to you. If I find out that you prefer seafood when you go to a restaurant, and then decide to take you to a seafood restaurant (a good one), how can I fail? I can’t. Or if you are a student and you find out what your teacher wants you to do to earn an “A” and you provide the teacher with these things, how can you fail? You can’t. Finally, consider that a company wants to make a new type of computer. If the company asks people who use computers for suggestions for a new product, and then makes the product based on those suggestions, how can the company fail? It can’t. (All of these examples assume that the third step of the approach was followed: “Tell them that you gave it to them.”)
This three-step process is included here because it also relates to research, in this case, research in advertising. There are two basic approaches we could follow to develop a research methodology for advertising studies. The first is that we could develop what we think we need to test or investigate and then administer the measurement instruments to consumers. The second approach is to pay attention to consumers describe advertising and then develop a research methodology based on what these people say (and do) when they discuss and analyze advertising. The second approach is the one I used. For more than 20 years, I have listened to consumers talk about and analyze advertising. These thousands upon thousands of comments provided the basis for a method to analyze advertising.
It would be gratifying to take personal credit for the discussion of the research methodology about to be presented, but that is not possible. It isn’t possible because I didn’t develop it. The methodology came from consumers. And in this case, the consumers provided a foundation for a three-step process in analyzing advertising messages. The method is appropriate for any type of print or electronic advertising. You should notice immediately that the consumers helped develop a research methodology based on what advertising really is: communication.
Three-Step Research Approach
The three-step research process to analyze the effectiveness of any type of advertising includes the following three steps:
An overall rating for the advertisement as compared to all similar advertising.
A rating about how successful the advertisement is in encouraging a person to try a product or service for the first time, or use the product or service more often.
An open-ended description of the message contained in the advertisement.
After thousands of advertising studies, these three points have emerged as the most successful in determining the success of an advertisement. There is no test of recall, potential for increased sales, or potential for increased store traffic. That is correct. The success of an advertisement is not based on a test of recall or a test of increase in sales. The success of an advertisement is based on its ability to communicate a message successfully and nothing more.3
Considering all of the information up to this point, we now know that the goal of advertising research in any medium should be related to finding out if the advertising communicates the correct message and how well it achieves this task.
Let’s look at each of the three steps in advertising research developed by consumers.
Discussions and debates about learning usually focus on the fact that a person learns things much easier if the new topic or idea relates in some way to something that he or she already understands. Consider, for example, the field of statistics. It is much easier to teach a person how to conduct an Analysis of Variance if the person already knows how to conduct a t-test. (If you don’t know anything about either statistic, conduct a search on the Internet for more information, or consult one of the statistics books listed at the end of this chapter.)
Learning something new by associating it with something old is clear when discussing advertising with consumers. Consumers almost always compare a new advertisement to something they have already heard, seen, or read. (Consumers also may relate the product or service advertised to something they already know). When analyzing advertising, consumers may or may not make a product-to-product comparison. However, consumers will always compare the advertising they are analyzing to all other advertising in or on that medium. When discussing television commercials, for example, respondents will says things like, “I don’t think I will ever buy the product, but the commercial is excellent. It presented the information very clearly, was easy to follow, and was entertaining.” Or respondents will say, “I already use that product, but I didn’t like the commercial because it went too fast and didn’t tell me where I could buy it.” In other words, consumers compare advertising in the same way they compare everything else in their lives—to something they already know.
The range of comments consumers have about advertising is almost limitless, but the key is that respondents compare new commercials to old commercials. I have learned over time that consumers have in their minds what constitutes a good advertisement, whether the ad is for dog food, automobiles, or anything else. Since this is true, it makes sense to have consumers rate an advertisement by having them compare it to all other advertisements they have been exposed to—a comparison to an “ideal” commercial. The rating is not across media, but within the same medium. That is, “How do you rate this television commercial against all other television commercials you have seen?” Or, “How do you rate this newspaper advertisement against all other newspaper advertisements you have ever seen?” This is important, because consumers have difficulty, for example, comparing a television commercial to a magazine advertisement.
The question “How do you rate this [insert medium] advertisement against all other [insert medium] advertisements you have ever seen [heard]?” is the first question that is asked of consumers when analyzing advertising. The consumers are asked to rate the advertisement as compared to their “ideal” advertisement. Over time, I have learned that a 10-point rating scale is most effective, where “1” means Poor, “10” means Excellent, and 2 through 9 are in-between.
To summarize, then, assume we are testing a television commercial. The first question asked in an analysis of the commercial is:
“How do you rate this commercial as compared to all television commercials you have ever seen? Please use a scale of 1 to 10, where “1” means Poor, “10” means Excellent, and 2 through 9 are in-between.”
The question can also be posed as:
“How do you rate this commercial as compared to all television commercials you have ever seen? Please use a scale of 1 to 10, where the higher the number, the more you liked the commercial.”
This rating provides an indication of how attractive the commercial is, how clearly the message is delivered, how easy it is to understand, and how much attention the consumers might pay to it. During the past several years consumers have shown that an advertisement should have a minimum average score of 7.0 to be considered successful. An advertisement with an average score under 7.0 does not meet the minimum expectation and will probably not be received well by consumers. Experience in the area has shown that advertisements that achieve average scores of 8.0 or higher are the ads that consumers like to see, hear, and watch over and over. (Recall that people move through the Five Stages of Persuasion because of message repetition.)
Amount of Encouragement
This rating was developed based on consumers saying things like, “The advertisement really convinced me that I need to try the product,” and “The advertisement didn’t give me any reason to try the product.” Listening very closely to consumers’ comments highlighted an important point: the consumers were telling the advertisers and advertising community what they want in an advertisement: “Explain to me why I need this product or service . . . give me some good reasons.” (Relate this now to the “Find out what they want and give it to them” step for success.)
Therefore, continuing with the already commonly used 10-point scale, this question asks consumers to rate an advertisement’s ability to encourage them to buy or try a new product or service for the first time, buy the product or service again, or buy the product or service more often.
Continuing our hypothetical test of a television commercial, the respondents are then asked:
“How do you rate how well this commercial encouraged you to buy the product or service? Please use a scale of 1 to 10, where “1” means It gave you no encouragement at all, “10” means It gave you a lot of encouragement, and 2 through 9 are in-between.”
The question can also be asked as:
“How do you rate how well this commercial encouraged you to buy the product or service? Please use a scale of 1 to 10, where the higher the number, the more encouragement the commercial provided.”
This rating provides an indication about the content of the advertisement. It provides and indication of how well the message is delivered, how easy it is to understand, how relevant it is to the person’s lifestyle, and how likely the consumers might be to try the product or service for the first time or use the product or service more often.
As before, experience in this area has shown that a successful advertisement must achieve a minimum average score of 7.0 on the 10-point scale. Advertising that does not achieve this minimum will probably not be successful with consumers. However, advertising that achieves average scores of 8.0 or higher will be more successful in moving consumers through the Five Stages of Persuasion more quickly.
The final step in the three-step advertising research approach is to ask consumers to answer a simple open-ended question about the message the advertisement gave them. Again using the television commercial, the consumers are asked:
“Finally, please explain in your own words what the commercial told you? What did you learn or understand from the commercial?”
This question is asked as a “cross check” for the two 10-point ratings scales. The reason for this is that an advertisement may receive high scores, but the message that is delivered may not be the message intended by the advertiser or those who created the ad. An example should make this clear. Several years ago, we tested a television commercial for a soft rock radio station. The commercial received very high scores: Overall – 8.4 and Encouragement – 8.9. The commercial looked very promising, but after reading the open-ended responses by the respondents, it was discovered that almost all of respondents thought the radio station play hard rock music, not soft rock music. In other words, the commercial was extremely successful in communicating the wrong message. Needless to say, the commercial was changed to solve this confusion.4
Once a test is completed, the data summary is quite simple. The average scores are shown along with a summary of the results for the open-ended question. For example, the following summary tables show the results of a test for a radio station’s prototype television commercials. The commercials were shown to two focus groups. Group 1 included all men; Group 2 included all women. The “Try” is used as a short-form for the “Encouragement” rating:
|Prototype I||Group 1||Group 2||Average|
Virtually all of the respondents said they like this commercial because it is presented in a very simple way. The commercial is rated highly because it included several examples of the type of music the radio station plays, it showed the station’s call letters and frequency throughout the entire commercial, and the information was presented in an easy-to-understand pace. According to most of the respondents, this commercial is “home run” for communicating a good message about the radio station.
|Prototype I||Group 1||Group 2||Average|
This commercial was not received well by either group. The men criticized the commercial for “going too fast” and “including silly pictures of the DJs.” The women did not like the “anti-female” approach of the morning show hosts. Overall, the women agreed that the commercial said, “This radio station is not designed for women, so don’t listen.” The average scores for Rating and Try indicate that this message will not be successful in communicating a positive message to the radio station’s listeners and potential listeners.
As you can see, the data summary for this process is simple. The average scores along with the comments by the respondents present an easy way to determine if the advertisement is successful. There is no subjective judgment involved. In addition, because advertising is only communication, the data summary relates only to this area. There is no mention of increasing sales, increasing store traffic, or anything else. It’s a pure analysis of communication.
Incidentally, this three-step process of analyzing communications works all types of adverting including, but not limited to, radio and television, newspapers, magazines, billboards, direct mail, and Internet banner advertising.
The Future of Advertising Research
As mentioned earlier in the chapter, high technology and the Internet have both contributed to substantial changes in mass media research. Although still in its infancy, both high technology and the Internet will eventually become commonly accepted methods of research, including research in advertising. Let’s look at advertising research in particular.
It is obvious that in order to test any type of advertising, consumers must be exposed to the messages. For example, we cannot merely ask consumers if how they would rate a television commercial or newspaper advertisement without actually seeing the ad. The consumer obviously must watch, hear, or read the advertisement before they can rate it and comment on it. Historically, focus groups have been used to test advertising because the methodology allows consumers to hear, see, and read the advertisements. But this procedure can be done via CDs, DVDs, and the Internet. Consumers can be sent a CD or DVD to their home or business and watch the advertisement (print or electronic) on their own equipment. Consumers can also be exposed to advertising via the Internet.
While there are still questions about control over the testing situation since we really don’t know who is answering the CD, DVD, or Internet questionnaire, the costs of research and the pervasiveness of computers and Internet access will force these methods to be accepted. These high tech/Internet methodologies save time, money, work, and travel. An example may help support this idea.
In 1999, Wimmer-Hudson Research & Development was contacted by a major music company to investigate how they could collect opinions from consumers who purchased their products (music CDs), or from radio station Program Directors who listen to new music and provide comments to the company. Historically, a small response card was included in the CD. Consumers who purchased the CD were asked to complete the small card and mail it back to the music company. In the case of Program Directors, the music company called these people on the telephone to retrieve their comments about new music releases. Enter the high technology approach.
Wimmer-Hudson designed a way to include a questionnaire on the music CD. The CD can be played in a normal music system, but also works in a computer. Included inside the CD container is a small card that asks consumers to listen to the CD on their normal music system, then take the CD and insert it into their computer. The computer automatically brings up a questionnaire that includes questions about the music selections included on the CD, evaluations of the CD cover, and even a test of introductions for the video planned for one of the songs on the CD. After the consumer answers the questions, he/she is instructed to sign on to the Internet. Once on the Internet, the responses to the questionnaire are automatically sent to a secure music company website where the executives can view the responses to their new product.
This same procedure can be used to test literally any type of print or electronic advertising.
1The fact that we “Cannot not communicate,” and therefore we “Cannot not advertise” is something that both individuals and companies often overlook. Let’s take a look at a real example based on my own experience. During the past several years, Sears & Roebuck has frequently reported lower than anticipated sales figures. A typical press release to stockholders attributes the soft sales to one reason or another (it doesn’t matter what the reason is). The bottom line is that fewer customers are shopping at Sears. Why is that? Well, it may be due to competition, or it may be due to the fact that consumers don’t like Sears’ prices. I’m not sure. The only thing I know is that Sears has forgotten about the fact that they “Cannot not advertise.”
Let me explain. I have been a loyal Sears’ shopper for years, the same as my parents. I will admit that my loyalty is to Sears’ tools and household appliances, but at least that is something. In the past five visits to Sears to buy tools, I have walked out of the store without making a purchase. I had the product I wanted to buy in my hand, but I could not find a salesperson to take my money. What decision did I make? I put the product back on the shelf, walked out of Sears, and went to another store to buy the product. The problem for Sears is that in my past five visits, they advertised (communicated/persuaded) to me that they do not want me to shop in their store. Businesses (and individuals) cannot not advertise. From the moment a consumer walks into a store, or visits a company web site, or reads a catalog, the store is advertising.
In the case of a retail store, the store advertises a message from the ease of parking, the entrance to the store, the lighting, the product layout, the sales people, the floor covering, prices, and literally everything else. A store that advertises positive messages through these elements allows a person to move through the Five Stage of Persuasion and the customer will spend money. If the store advertises negative messages through the elements (such as what Sears does by not having sales people available), the customer will not spend money. It’s that simple.
Sears is a big retail store in the United States (#2 behind Wal-Mart), so we would think that they would know what to do to correct the situation, and then correct it immediately. That seems logical, but the logic is wrong. Did Sears add more people to help correct the problem of sagging sales? Did they make the buying process easier for customers? The answer to both questions is “No.” Instead, Sears changed its advertising campaign. No longer do we hear or see “The softer side of Sears.” We now hear and see the new Sears’ slogan: “A good life at a great price.” As an advertising researcher and customer of Sears, I have to say, “Sorry, the new slogan does not indicate to me that I will be able to buy things when I enter the store.” What the slogan says to me is “A good life at a great price for products you can’t buy.”
The slogan is a plastic bandage for a gaping wound. The slogan says anything about hiring more sales people. The slogan says nothing about solving the problem of having to wander around the store trying to find a sales person. How do I know? I went to Sears just the other day and, you guessed it, put another product back on the shelf and walked across the street to a competitor. It may be that Sears’ management should get out of their offices in Chicago and try to buy something from one of their stores. If they would personally experience the difficulty of buying something in one of their stores, they might make a few changes. And they might decide to change their slogan from “A good like at a great price” to something like, “You can now buy things at Sears.”
2Mass communication and the mass media are often said to have a great deal of power over people. One early theory about the power of mass communication was developed during World War II when researchers investigated Adolph Hitler’s propaganda. The theory known and the “Bullet theory” or “Hypodermic-needle theory,” suggested that mass communicators need only “shoot” a message at the masses and they will react in universal and predictable ways. However, research in later years did not find evidence for the theory. Additional research found that individual behavior negates the idea that everyone reacts the same way to one specific message or campaign.
3Advertising recall research is a waste of time and money. Generally speaking, recall studies involve showing a person an advertisement and then asking the person to name the product or service or the company that sponsored the ad. Recall is investigated either immediately after exposure to the message or a day or more after the exposure. This research then concludes that an advertisement is successful if it receives high “recall” scores. This makes no sense at all. For example, an advertisement may have high recall scores because it is misleading, false, or irrelevant. Recall of advertising alone means nothing. Recall alone says nothing about the quality of the message. Recall alone says nothing about the advertisement’s ability to move people through the Five Stages of Persuasion. The money spent on advertising recall research would be better served if contributed to a favorite charity. It is, as mentioned, a complete waste of time.
4This approach to testing advertising messages does not include any specific rating of the creative aspects of the advertisement, although creativity underlies both measurements. An advertisement can be as “new” or “different” as the creators wish to make it. The consumers merely are asked to rate the advertisement on the areas that advertising is supposed to address: How does it relate to other advertising they have already seen? How much encouragement does the advertising provide? What does the advertisement tell you? It is that simple.
Aronson. B., & Zeff, R. L. (1999). Advertising on the Internet. New York: John Wiley & Sons.
Babbie, E. R. (1997). The practice of social research (8th ed.). Belmont, CA: Wadsworth.
Bissland, J. H. (1990). Accountability gap: Evaluation practices show improvement. Public Relations Review, 16(2), 24-34.
Blalock, H. M. (1972). Social statistics. New York: McGraw-Hill.
Block, M. P., & Brezen, T. S. (1990). Using database analysis to segment general media audiences. Journal of Media Planning, 5(4), 1-12.
Boyd, H. W., Westfall, R., & Stasch, S. F. (1989). Marketing research: Text and cases (7th ed.). Homewood, IL: Irwin.
Broom, G. M., Casey, S., & Ritchey, J. (1997). Toward a conceptual theory of organizational public relationships. Journal of Public Relations Research, 9(2), 83-98.
Browne, A. (1997). Does your advertising work? Colorado Business Magazine, 24(5), 72-75.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. New York: Academic Press.
Cook, W. A., & Dunn, T. F. (1996). The changing face of advertising research in the information age. Journal of Advertising Research, 36(1), 55-71.
Davis, J. J. (1996). Advertising Research: Theory and Practice. Upper Saddle River, NJ: Prentice Hall Business Publishing.
Fenwick, I., & Rice, M. D. (1991, February/March). Reliability of continuous measurement copy testing methods. Journal of Advertising Research, 31(1), 23-29.
Fletcher, A., & Bowers, T. (1991). Fundamentals of advertising research (4th ed.). Belmont, CA: Wadsworth.
Fox, S. (1984). The mirror makers: A history of American advertising and its creators. New York, NY: William Morrow and Company, Inc.
Green, H. (1998, April 27). The new web ratings game. Business Week, pp. 73-78.
Green, P. E., Tull, D. S., & Albaum, G. (1988). Research for marketing decisions. Englewood Cliffs, NJ: Prentice-Hall.
Haley, R. (1994). A rejoinder to conclusions from the ARF’s copy research validity project. Journal of Advertising Research, 34(3), 33-34.
Haley, R. I., & Baldinger, A. L. (1991, April/March). The ARF copy research validity project. Journal of Advertising Research, 31(2), 11-32.
Haskins, J. (1976). An introduction to advertising research. Knoxville, TN: Communication Research Center.
Haskins, J., & Kendrick, A. (1993). Successful advertising research methods. Lincolnwood, IL: NTC Business Books.
Hurlburt, R. T. (1998). Comprehending behavioral statistics (2nd ed.). Pacific Grove, CA: Brooks/Cole.
Jaccard, J., & Becker, M. A. (1996). Statistics for the behavioral sciences (3rd ed.). Belmont, CA: Wadsworth.
Jacoby, J., & Hofer, W. D. (1982). Viewers’ miscomprehension of televised communication. Journal of Marketing, 46(4), 12-27.
Kish, L. (1965). Survey sampling. New York: John Wiley.
Leckenby, J. (1984). Current issues in the measurement of advertising effectiveness. Paper presented to the International Advertising Association, Tokyo, Japan.
Leckenby, J., & Wedding, N. (1982). Advertising management. Columbus, OH: Grid Publishing.
Leckenby. J. D. and Plummer, J. T. (1983). Advertising stimulus measurement and assessment research: A review of advertising testing methods. Current Issues and Research in Advertising, 6 (2): 135-65.
Lehmann, E. L. (1991). Testing statistical hypotheses (2nd ed.). Belmont, CA: Wadsworth.
Lloyd, D. W., & Clancy, K. J. (1991, August/September). CPMs vs. CPMIs: Implications for media planning. Journal of Advertising Research, 34(4), 34-43.
Maleske, R. T. (1994). Foundations for gathering and interpreting behavioral data: An introduction to statistics. Pacific Grove, CA: Brooks/Cole.
Mason, R. D., Lind, D. A., & Marchal, W. G. (1998). Statistics: an introduction. Belmont, CA: Duxbury.
McQuarrie, E. F. (1996). The market research toolbox. Thousand Oaks, CA: Sage Publications.
Mehta, A., & Purvis, S. C. (1997). Evaluating advertising effectiveness through advertising response modeling. In W. D. Wells (Ed.), Measuring advertising effectiveness (pp. 325-334). Mahwah, NJ: Lawrence Erlbaum.
Merriam-Webster (1989), Webster’s Ninth New Collegiate Dictionary. Springfield, MA: Merriam-Webster.
Moore, D. S. (1998). Introduction to the practice of statistics. New York: W.H. Freeman.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
Percy, L., & Rossiter, J. R. (1997). A theory-based approach to pretesting advertising. In W. D. Wells (Ed.), Measuring advertising effectiveness (pp. 267-282). Mahwah, NJ: Lawrence Erlbaum.
Raj, D. (1972). The design of sample surveys. New York: McGraw-Hill.
Rosenthal, R., & Rosnow, R. L. (1969). Artifact in behavioral research. New York: Academic Press.
Rossi, P., & Freeman, H. (1982). Evaluation: a systematic approach. Beverly Hills, CA: Sage Publications.
Rossiter, J., & Eagleson, G. (1994). Conclusions from the ARF’s copy research validity project. Journal of Advertising Research, 34(3), 19-32.
Ryan, M., & Martinson, D. C. (1990). Social science research, professionalism and PR practitioners. Journalism Quarterly, 67(2), 377-390.
Schultz, D., & Barnes, B. (1994). Strategic advertising campaigns. Lincolnwood, IL: Business Books.
Skinner, B. F. (1953). Science and human behavior. New York: Macmillan.
Toothaker, L. E. (1996). Introductory statistics for the behavioral sciences (2nd ed.). Pacific Grove, CA: Brooks/Cole.
Tukey, J. W. (1986). The collected works of John W. Tukey. Belmont, CA: Wadsworth and Brooks/Cole.
Walker, D., & Dubitsky, T. (1994). Why liking matters. Journal of Advertising Research, 34(3), 9-18.
Watzlawick, P., Beavin, J. B., & Jackson, D. D. (1967). Pragmatics of human communication: A study of interactional patterns, pathologies, and paradoxes. New York: Norton.
Williams, F. (1992). Reasoning with statistics (2nd ed.). New York: Holt, Rinehart & Winston.
Wimmer, R. D., & Dominick, J. R. (2000). Mass media research: an introduction, (6th ed.). Belmont, CA: Wadsworth Publishing Company.
Mass Media Research: An Introduction, 9th Edition, Home Page