Mass Media Research: An Introduction - 9th Edition
Radio Station Ratings: Where Do the Numbers Come From?
Roger Wimmer, Ph.D.
A radio station’s Arbitron numbers go up and down like a stock on the NYSE. Why? What causes the changes? According to some advertising in the trade press, the changes are due to an outside influence such as a consultant or music testing company. Let’s take a look at this situation.
Read the following four Scenarios. How many are true?
Scenario 1: A music testing company lists several radio station clients that are #1 in the market or a specific demo. The implication (or outright claim) is that the music tests were the sole cause of the increased ratings.
Scenario 2: A TV spot production company lists several radio station clients that are #1 in the market or specific demo. The implication (or outright claim) is that the TV spots were the sole cause of the increased ratings.
Scenario 3: A research company lists several radio station clients that are #1 in the market or a specific demo. The implication (or outright claim) is that the research was the sole cause of the increased ratings.
Scenario 4: A programming consultant lists several radio station clients that are #1 in the market or a specific demo. The implication (or outright claim) is that the consultant was the sole cause of the increased ratings.
How many of the Scenarios are true? The answer is “none.” Why not? We need some background information to understand the answer.
First, the Scenarios make an association between two variables—research, music tests, consulting, or TV spots and Arbitron ratings. The Scenarios follow a simple premise that is used in a variety of situations: If this, then that. Using the music testing Scenario, the premise is: If you use a specific company’s music test, Then your station will have higher ratings. In research terms, this association is written as: If x, then y.
Let’s look at this concept of associations to see what’s going on.
The research formula used to state an association is: y = f(x). In this formula, x is an independent variable, or the variable(s) that is/are manipulated; y is the dependent variable, or variable(s) that is/are measured or tested, and “f” stands for “function.” For example, in the Scenarios presented above, the “x” part of the formula represents a music testing company, a TV spot production company, a research company, or a consultant. The “y” in the formula represents Arbitron ratings. Therefore, the formula reads as follows: Arbitron ratings (y) are a function (f) of the research company, music testing company, consultant, or TV spot production company (x). The associations state that a station’s Arbitron ratings are caused by one of the agents . . . If x, then y. This is the “cause-and-effect” argument often discussed in research.
However, the statement If x, then y, is only half the story. The opposite must also be included to make a complete argument. The opposite of If x, then y is . . . If not x, then not y. Both of these statements must be considered to establish whether the independent variable(s) is/are the cause of y.
Using the four Scenarios again, the two statements of association are:
If x, then y
If a radio station uses a research company/music testing company/consultant/TV spot, the result is increased Arbitron ratings.
If not x, then not y
If a radio station does not use a research company/music testing company/consultant/TV spot, the result is no increase in Arbitron ratings.
Let’s get to the main point of the associations that are made with Arbitron ratings. The question that most people would ask is, “Are the Arbitron ratings a station receives a function only of a research company’s data, a consultant’s recommendations, or a TV spot?” The answer is “no.” The ratings a station receives are a function (the result of or caused by) of numerous independent variables. Consider the following list of items that will have some affect on a station’s ratings:
The station’s format (established, fad)
Changes in a station’s programming or formatics
Changes in any other station’s programming or formatics
Management changes (GM, PD, etc.)
On-air talent changes
Time of year when the ratings are conducted (seasonality of radio listening)
Why a listener completes a diary (To vote for a favorite? To vote against a station?)
A listener’s desire to be accurate in keeping a diary
A listener’s understanding of how to complete a diary
The accuracy of a listener’s memory in recording listening
Listeners’ errors in recording listening behavior (misspelling, etc.)
Advice from friends or relatives about how to complete a diary
A station’s external advertising (TV, billboards, direct mail, telemarketing, etc.)
Other stations’ advertising (TV, billboards, direct mail, telemarketing, etc.)
Station publicity, such as a story about the morning show DJs in the local newspaper
On-air promotions and advertising by any station in the market
Contests on any station in the market during the ratings period
Weather emergencies in the area when the ratings were conducted
News events or emergencies during the ratings period
Significant events during the ratings period (sporting events, holidays, disasters)
TV specials that might attract huge audiences
Changes in listeners’ radio listening habits affected by changes in personal schedules
Affect of listeners’ purchase of CDs or other audio sources that reduces radio listening
Effect of listeners’ viewing of music channels on TV that reduces radio listening
An undeterminable number of unknown sources of error
This list demonstrates the variety of independent variables (the x in the formula) that might be a function of a radio station’s rating (the y in the formula). Is it fair, then, for one company or one person to claim that the Arbitron numbers are the result of one independent variable? Once again, the answer is “no.”
In order to determine if a research company, music testing company, consultant, or TV spot was the only function of a radio station’s rating, a controlled experiment would have to be conducted to find out if that variable, and only that variable, was the cause of the ratings. To do this, the research would have to eliminate the influences of all other variables that could affect the ratings. In research terms, the study would have to control the other independent variables. When, and only when, all of the other influences (the other functions) are controlled could the following statement be made: “This TV spot produced an increase in Arbitron ratings.”
If all potential sources of influence are not controlled, then an association between only one element (function) and Arbitron ratings is flawed.
So, what’s the answer? Where do we go from here?
In essence, we are searching for a “cause and effect” of radio ratings. What “effects” influence a station’s rating to rise, fall, or stay the same—what is the cause? Can the answer be found by using the simple formula of, If x, then y? The answer is “probably not,” and more than likely, “no.” The real formula should look something like this:
If x1 + x2 + x3 +x4 . . .+ xn, then y
where x1 to xn are independent variables
This formula accounts for the fact that a variety of functions (effects) produces an Arbitron rating (cause). In this amended formula, the rating is not assumed to be a function of only one variable; it assumes that a variety of influences produce a rating. However, even this amended formula has a problem.
While the formula looks straightforward—just plug in the various independent variables to determine a radio station’s Arbitron ratings—it assumes that all of the variables (functions) have the same importance in producing a rating. For example, the formula suggests that the decisions made based on the results of a research study have the same importance as changes in on-air personalities or playlist changes. Is this true? Probably not, because things aren’t that simple. It is unlikely that an Arbitron rating is merely a simple linear combination of items; that is, it is unlikely that each contributing variable has the same influence and each is merely added together to produce a rating.
What is needed is a weighting system to control each variable’s influence on the rating. Some variables are more important than others, and the formula needs to account for that. A new formula that includes weights for each variable, shown by the letter b (known as a beta weight) is:
If bx1 + bx2 + bx3 +bx4 . . .+ bxn, then y
where x1 to xn are independent variables and b is the influence of each variable
However, even this formula has problems. Not only do we not know which independent variables to include, we don’t know the amount of influence or weight to assign to each variable. As you can see, the problem of finding the “cause” of Arbitron ratings is an extremely complicated problem. It is not simply, “Hire this consultant and get higher ratings.” Or “Use a certain music test approach and get higher ratings.”
In order to find out why a radio station receives a specific rating, it is necessary to look at all of the contributing variables, not just one. Anyone who says that a station’s ratings are the result of one variable (one function) does not understand the industry and is making a faulty association.
However, the problem of faulty associations is not unique
to the radio industry. Such associations are common in all industries. For
example, how many companies attribute their increase or decrease in sales only
to an advertising campaign? How often have you heard a company spokesperson say
something like, “Our sales will be up this quarter because of an increase in
advertising spending.” Or, “Our sales were down last quarter because our
advertising did not accurately portray our company philosophies (or product).”
Whether it’s a radio station or a company that produces a consumer product, the
success or failure of the business cannot be attributed to only one
variable/element/effect. The situation is not . . . If this, then that. It is
really . . . If these, then that.
©2009 Roger D. Wimmer
Mass Media Research: An Introduction, 9th Edition, Home Page