Arbitron has revised our metro survey area from a single densely populated county based on one major city (where all station signals were roughly equal) to a new 3-county scenario. Not only is it a big issue for the weaker signals, but the two new counties have an overall rural make-up. Diary return from the new counties varies greatly.
My full power station will often have roughly 350 diary mentions from our core target in the original MSA, and perhaps 5-10 mentions in the same demo from the new counties. What is your understanding of diary weighting? Are these few diaries from surrounding counties of concern to me? Should I continue to focus my programming, marketing, and promotions on my “Hot Zips” or place more attention on these new counties? - Anonymous
Anon: Hot Zips? Hmm. For those who don’t know, “Hot Zips” are the Zip Codes in a market where a radio station seems to be popular (or has at least received mentions in diaries from those Zip Codes in the past). I have a problem focusing on Hot Zips because a Zip Code that may be “hot” in one book, may not be “hot” in the next book.
I understand target marketing, but using Hot Zips as a marketing approach isn’t logical because of the randomness of diary placement and diary returns. You may focus on a Zip Code that was a fluke. Each Arbitron book uses a different sample, and there is no guarantee that your respondent participation in a Zip Code will be consistent from one book to the next.
There is no way for you to predict where your diary mentions will come from in the next book. If you use Hot Zips from the past as your only advertising and promotion tactic, you could be 100% wrong. And that don’t be right.
Diary weighting? Diaries are weighted due to a shortfall in diary returns (in-tabs). For example, if 100 diaries are expected from a certain area, or from a specific age/sex cell, and only 50 are returned, the 50 in-tab diaries are weighted (multiplied times two) to get to the 100 expected diaries. (The procedure is complicated—not all diaries are counted twice.)
There is nothing wrong with diary (or respondent) weighting as long as it doesn’t get carried away. In other words, if you expect 100 diaries and only receive one, then it wouldn’t be correct to weight that one diary up to 100.
Now, you ask what you should do. Should you focus your advertising and promotion on Hot Zips or should you worry about the two new counties? With the limited information you have provided about the two new counties, this is what I suggest:
Although logic and knowledge of human communication/persuasion suggests that you should distribute your marketing money in proportion to your population distribution in the three counties, logical approaches aren’t always possible.
My guess is that you don’t have an unlimited marketing budget, so you probably have to put logic aside here and spend your money in the next most logical way. It doesn’t appear that the two new counties will have a tremendous impact on your numbers because you say that they are both rural in nature (meaning “sparsely populated” to me). In this case, therefore, it will probably be best for you to concentrate on your primary (largest) county because it will have the most significance in Arbitron (diary placement and return).
Another guess I have is that the folks in the two new counties probably frequent the larger county for shopping and work and they will be exposed to your billboards or other advertising in the larger county. (If you don’t already do this, you may want to refer to the smaller counties on-air to try to make the folks feel as though they are equally important to you.)
It would be great if you could spread your marketing dollars around to all three counties, but I don’t think that is practical in your case. In your type of situation, most radio stations concentrate on the area (county) that contains the largest number of listeners, or potential listeners.
Diary Placement - Another Question
How much does diary placement affect the book and how does Arbitron decide how many diaries to send to a specific zip code? I have always thought the excuse we didn't get good diary placement was a cop out after all are you broadcasting to the entire city or just certain zip codes - Anonymous
Anon: I sent your question to Bob Michaels, VP of Radio Programming Services. He passed the question to Diane Woodard, Arbitron's Manager of Customer Analysis and Research. I want to thank both Bob and Diane for their help. Diane said:
Arbitron Radio market samples are designed to measure the entire market area proportionately. A Metro has a 12-week survey target. Let's assume the sample size target is 1200. There are three counties in our hypothetical metro: Smith, Jones, and Adams. The population estimates tell us that 20% of the total Metro pop is in Smith, 30% is in Jones, and 50% is in Adams. Therefore the survey target for each county are 240, 360, and 600 respectively.
Arbitron's Sampling Department maintains a history of Consent and Return rates for each sampling unit, so that we can order the correct number of phone numbers in each area to reach the correct number of respondents in order to, assuming the participation rates hold true to the trends we've been measuring across previous surveys, make the sample target for that county. (In the case of split counties, each split becomes a separate sampling unit, with its own target. Generally, return rates in HDBAs (High Density Black Area) and HDHAs (High Density Hispanic Area) are lower than in Balance areas. If Adams has an HDBA that made up half of Adams county—with the resulting target of 300—and we know from experience we can expect a 40% return rate in that sampling unit, we need to mail/place 750 diaries to get back 300. While we also know there will be a 50% return rate from the balance, so we will need to place 600 diaries in Adams Balance to get back 300 diaries.)
Thus, we will order sample phone numbers from our vendor specifically for each sampling unit in each Metro. The sample frame for each sampling unit includes all listed residential phone numbers in that geographic area, plus all potential numbers in hundred blocks where there are at least 10 listed phone numbers. All of the phone numbers with addresses include a zip code. For phone numbers without an address, a zip code is assigned based on the area code, exchange, and hundred block for the listed numbers. The sample frame is then sorted by zip code. Obviously, zip codes with larger populations will have more telephone numbers than zip codes with smaller populations. A random start point is used, and based on how many sample telephone numbers are needed and how many numbers the frame includes for the sample unit, an interval is calculated, and every Nth number is selected from the top of the list to the end of the list. The result is individual zip codes are proportionately represented within each sampling unit. That final sample is then equally divided into 12 weekly samples—so that each week will also be proportionately representative of the entire sampling unit.
That’s a detailed response and I hope it answers your question. If not, let me know.
Diary Placement - Another Question - Comment
Thanks for the answer you
printed. I take from it that you could argue bad placement during the trends,
but when the book comes out you cannot, if Arbitron has obtained the number of
diaries in each county. Did I come to the right conclusion? - Anonymous
Anon: I’m not exactly sure what you’re asking. The results in your book may not be due to a bad sample, but rather a sample that may be skewed to a specific age and/or demographic. That’s why you need to check the weighting used in each book. If the sample target falls short, it is weighted. The weighting process may affect your radio station’s performance.
As I have said in many questions in the past…the results in an Arbitron ratings book are based on many, many variables, and diary placement is only one of those variables.
My market just received the latest Arbitron ratings. The market size is 130,000, and 550 books were sent out. Is this enough books for a market this size? Hey thanks, brother. - Anonymous
Anon: You’re welcome, brother. Hey, now I have three brothers! OK, on to your question.
Are you sure Arbitron sent out 550 diaries, or is that your in-tab number (the number of diary returns)? Since I don’t know, I’ll have to give you two answers.
Your market size is 130,000, which means that your market rank is about 240. The likelihood is that you’re also a condensed market, which means that your sample size is smaller, you have fewer books, and your book contains fewer demographics. You have fewer demographics because of the small sample size. Why? Because when the data are broken down into age/sex cells, your error rates would go sky high and the numbers would be essentially meaningless. Back to your question…
First, let’s look at the 550 diaries sent scenario. The typical return rate (response rate) for Arbitron diaries is about 40%, which means that if Arbitron did send 550 diaries, your in-tab diaries should be about 220. Is this good, bad, or indifferent? Is this correct or incorrect?
The answer to those questions is the same: it depends. It depends on how much sampling error you are willing to accept. If you go to my business website, you’ll see a box titled, “Sampling.” If you click on the “95% Level,” you can find the sampling error associated with any sample size.
If you input 550 as the sample size, you’ll find that the associated sampling error is 4.18%, or rounded to ±4.2% (plus or minus 4.2%). This means that if your radio station has a 5.0 share (12+), your actual share is somewhere between .8 and 9.2 (rounded). Now, if your in-tab sample is 220, then your sampling error is ±6.6. Your actual 12+ share would fall between 0 and 11.6 (you can’t have a negative share). Is this good? It depends on your willingness to accept that amount of error. If you (or other broadcasters) don’t want that much error, you (they) can always pay Arbitron to have a larger sample (the “age-old” argument).
Understand what’s going on here? Sampling error affects all research where a sample is selected from the population, but it is particularly significant to radio stations because they have such small numbers. In other words, you may not be too excited if you had a 50 share (12+) and know that your actual share is somewhere between a 45.8 and 54.2 (sample of 550 with sampling error of 4.2%). But radio stations don’t have 50 shares, and sampling error becomes very important.
Now, I think you can see why condensed markets don’t have many age/sex breakouts. Even with a sample of 550, you may have, for example, only 25 diaries from Females 18-24. Assuming for a moment that this is true, the associated sampling error is 19.6%. If the book shows a 10 share for Females 18-24, your actual share would fall somewhere between 0 and 29.6. That’s a huge range, and that’s why age/sex cells aren’t included.
I should add, though, that the huge sampling error percentages aren’t only a problem with condensed markets. If you’re reading this and you’re from a larger market, check the in-tab diaries for some of your book’s age/sex cells. You may be surprised.
Don’t interpret my comments as an indictment against Arbitron. They aren’t. Sampling error is a characteristic of all research that uses samples. I’m sure Arbitron would enjoy having miniscule sampling error percentages, but the only way to get smaller sampling error percentages is to increase the sample size, and that not possible because the clients aren’t willing to pay the additional costs.
In summary, if you have 550 in-tab diaries for your book, then you can use the ±4.2 for sampling error. If you have fewer in-tabs, then check the sampling error using the calculator on my website.
When visiting Arbitron's HQ to view diaries, what is your advice? What can we expect, and what should we hope to come away with? What is your view on the value of diary comments? - Anonymous
Anon: I have been to Arbitron many times to review diaries, but I thought it was best to get a response from a few people at Arbitron. I sent your note to Bob Michaels (VP Radio Programming Services) and Claudine Knisley (Manager, Diary Analysis & Communications). Here is what they said (thanks to both for their help):
Diary Reviews are scheduled for one day, Monday through Thursday, with the Columbia, MD office open from 9AM to 5PM. Most stations follow the steps below to conduct a comprehensive review:
Look at the diaries credited to both your station and your competitor(s), in your key demo and, if there's time, for Persons 12+.
Read the comments recorded by the diary keepers (approximately 30% of the diaries contain comments).
Look at the entries credited as Unidentified Listening. There is a misconception that a large number of entries are credited as Unidentified. Actually, less than 3% of the total un-weighted quarter hours of listening is credited as such.
Check to see if your station's primary name is in conflict with another station's name (for example, are there two or more stations in your market identifying itself by the same Station Name). If so, then ask the attending Client Service Representative (CSR) to go over the edit procedures for this type of conflict, and how credit is being assigned. About 19% of all entries contain a Station Name.
If your station does not subscribe to PD Advantage, then run reports for diaries credited to your station (and your competitor(s) station) by zip code, week, demo, etc.
What can you expect during your visit?
You will find the attending Client Service Representative (CSR) immediately available to answer your questions, providing historical data on situations and/or detailed explanations of policies and procedures. Our CSRs are Diary Credit Policy Analysts with years of experience work, closely with our Diary Processing staff.
Most if not all of your questions will be answered on the spot by the CSR.
Any outstanding questions will be researched and responded to within 10-business days (if not sooner).
What do you hope to come away with?
A better understanding of how your station is being identified and credited.
A clarification of those crediting procedures you originally misunderstood.
A realization that diary crediting is very accurate. (On average, listening level credit is 99.5% accurate.)
A higher level of confidence with the process, and an appreciation for the diary processors who credit your diaries.
What is your view on the value of Diary Comments?
Approximately 30% of the diaries contain comments.
Collecting non-quantitative information about: (1) Your station and your competitor(s); (2) Changes in the market; and (3) Changes on a station.
Keep in mind that the Comments Section of the diary is rarely used during the station credit process. The vast majority of comments are general and do not specifically identify a station.
Roger’s comment: I think that should give you a good idea of what happens at a diary review. I will add one thing: Make sure that you don’t take the written comments too seriously because in most cases you will be looking at only one response. I have seen many radio people get carried away when they see one comment…something like, “Your morning show is terrible.” That means nothing by itself. You need to verify a comment like that in an independent research study.
Embargoed Radio Markets
I really enjoy your column and your wit. Can you please explain why certain Arbitron market's ratings are embargoed? Who determines if a market is embargoed? Why would a company want to embargo their ratings? - Anonymous
Anon: I'm glad to hear that you enjoy my wit. Flattery will get you a long way. I am not in the position to speak for Arbitron, so I sent your question to Bob Michaels, VP, Radio Programming Services. I want to thank him for his time, and this is what he said:
Embargoed markets are made as a business decision for a variety of reasons. But first, some a few comments on Arbitron data.
As you know, Arbitron makes all of our revenue based on radio stations, advertisers, and advertising agencies subscribing to our service. In most markets, the majority of radio stations subscribe to our service. However, there are markets where subscription levels are low, or there have been reports from our subscribing stations that non-subscribers are getting some top-line ratings data and using them. This violates our copyrights on our data and having them readily available (even on a Persons 12+ level) doesn't help the situation. So in these cases, the market is embargoed and none of the estimates for that market are released to the trade press.
Our subscribing radio stations in low-support markets or markets with a station accused of using our data without an agreement with Arbitron typically are the ones who want the data embargoed. Why should they pay for our service and allow a non-subscriber easy access to top-line results that they can steal and use against them? It's not fair to our subscribers or to us.
So, the market is embargoed based on requests either from our subscribers in that market or from our marketing staff.
What is the most common way listeners identify a station in their Arbitron diaries? Do they most often write down call letters, station name, frequency, or something else? - Jeff
Jeff: According to Arbitron data, about 52% of listeners identify radio stations by exact frequency, 51% identify with the exact frequency, and about 19% identify stations with a station's name or moniker. (The percentages are over 100 because some listeners use a combination of identifications.)
That's your basic answer. Here are a few excellent summaries by Arbitron:
How Diarykeepers Record Their Listening - Report
How Diarykeepers Record Their Listening - Summary
Identification during Crisis
Identification during Christmas
Roger, this is one of those "everything you wanted to know but too embarrassed to ask" questions. What is the formula/procedure for extrapolating a monthly Arbitron score from the 3-month rolling averages? Thanks. - KH
KH: I’m sure there are several approaches to this, but one common way. I want to thank Paul Douglas in Atlanta for discussing this with me to make sure that things haven’t changed.
Assume that you have a 3.2 in January, a 4.0 in February, and a 3.3 in March. Then the February-March-April trend comes out and you get a 3.2. To estimate how you did in April, multiply the 3.2 by 3. This gives you a 9.6. Then subtract 4.0 (Feb) and 3.3 (March) from 9.6, which gives you an extrapolated 2.3 for the month of April. Should you jump in the bathtub and slit your wrists? No, because the April number isn’t real.
Keep in mind that monthlies are not weighted, so your extrapolation could be way off. In addition, make sure that when you do the extrapolation that you have real monthly numbers, not extrapolations. The extrapolation procedure is unstable as it is and you don’t want to confound things even further by using all extrapolated numbers.
Doc: I'm new to radio and heard a few people talking about Arbitron numbers and a radio station in the market. One person said something like, "Their target isn't a target, it's a family reunion." Would you tell me what that means? - Anonymous
Anon: I'm not sure who coined the term family reunion in reference to describing a radio station's target, but it has been used for at least 25 years, and maybe more.
In radio, Family Reunion is usually used in two ways: (1) To refer to the fact that a radio station's target is too broad ("It includes everybody"), such as 25 to 54; and (2) When discussing Arbitron's 12+ shares, which are good only to provide an indication of a radio station's success. Why? Because no radio station on the planet has a 12+ target — it's usually something like 18 to 24 or 25 to 34. However, since you're new to radio, I think I should make a few comments about "standard age cell" targets.
A radio station's target is the key to its success because the target guides the programming and also guides advertising sales. The problem is that many people who operate radio stations don't know much about their target or how to define it. In many cases, managers (PDs, GMs, owners) select a target from typical marketing age cells (18-24, 25-34, 35-44, 45-54, 55-Dead). This really isn't a major problem since the cells span 7 or 10 years, excluding the "55 to Dead" group. However, you'll find that many people (PDs, consultants, etc.) suggest that a 7-year span (such as with 18-24) is probably the best to go with because there may be too much difference among people in the low end and high end of a 10-year cell.
OK, so there is some agreement among radio people that a station's target should be somewhere between a 7 and 10-year span (although there are some exceptions). The problem is that even though a station's management may select, for example, a 25-34 target, the radio station's listeners may not actually fall into that cell. In reality, the target may be something like 21-29, 22-28, 30 to 36, or something else.
There are two primary ways to determine a radio station's exact age target — Arbitron's software like Maximi$er, or a station's own perceptual research. (The way to do find a radio station's real target in perceptual research is to crosstab the exact age of respondents with a station's cume and P1 data. By the way, it's important in perceptual research to include respondents who are a few years younger and a few years older than the radio station's assumed target. If these people aren't included, the managers may never know their actual target.)
Now, let's get back to the 12+ number so I can show you how it may distort a radio station's success. The brief table below shows Arbitron shares from a Denver radio station (Mon-Sun 6A-Mid). My guess is that the station's target is somewhere between 18 and 34. (The only data I have are for the broad age cells, so forget the idea of smaller age cells for the purpose of this discussion.)
As you can see, the radio station's 12+ share is 2.4 (2.4% of the people listening to radio were listening to this station during this time period). Some people may think that the station isn't performing well because of the low number. However, if you look at the 18-34 cell, you'll see that the radio station has a 4.4 share. That may not sound significant to you, but in the world of radio shares, it's a big difference from the 2.4 12+ number.
Do you see now why looking at the 12+ share ("family reunion") isn't a good number to look at to evaluate a radio station's performance? The 12+ share includes all people in the survey (12 to Dead) and the radio station is NOT targeted to all listeners in the market — only to a group of listeners between the ages of 18 and 34.
If you have been observant, you should have noticed that the 12+ numbers are usually the only numbers reported in the trade media and Internet websites. That's because Arbitron is willing to provide 12+ numbers because they don't mean very much — they are only an indication, and oftentimes, a very poor indication of the performance of radio stations in a market. To get more specific age and sex listening information, you need to subscribe to Arbitron's service. That's the company's business; Arbitron is not a non-profit company.
Getting Things Right for the Book
I frequently hear PDs and consultants say that want to get “things right” for the start of the Arbitron book. Would you tell me what things are involved in this process? - Jim
Jim: I have heard the same thing for a few decades and I think it’s one of the silliest things anyone in radio can say. Why? Because the elements of a radio station should “be right” every day. Listeners do not choose to listen to a radio station for the first time or decide that a specific radio station is their favorite based on when the Arbitron book starts. These decisions are made every day of the year, and, therefore, the radio station should “be right” every day.
If you’re interested, I wrote a short article about this topic a while ago—click here.
Getting Things Right for the Book - Comment
Dear Doc: You are so “right on” about sounding good all the time, not just during ratings weeks. I think PDs are victims of the “sweeps weeks” mentality, so prevalent in TV. But TV is a different ball game. Still I find it amazing that TV spot rates are determined by shows that were only on during sweeps weeks. The agencies must be very stupid to buy into this, or maybe there is something I'm overlooking. I'm always willing to learn. As ever. - Jerry Gordon
Jerry: You are correct in saying that TV stations’ rates are based on “what was,” not “what is,” but that’s the same way radio sales works. It’s also the same with the print media. However, I need to add one thing. In most situations, TV networks (and sometimes stations) sell advertising based on projected ratings. If the program doesn’t achieve the expected Nielsen numbers in the overnights, the advertiser gets a rebate, usually in the form of additional spots to be run at a later date.
Click Here for Additional Arbitron Questions
Roger D. Wimmer, Ph.D. - All Content ©2018 - Wimmer Research All Rights Reserved