Internet Research - Music
Hi Doc: Your column is very informative - thanks for doing it! My question relates to Internet music research.
I have heard it said that Internet music research is not valid because it is a self-selecting sample (respondents sign up). But, in the larger sense, isn't all research opt-in? Since participation is never compulsory (even the US Census doesn't achieve 100% participation, right?), don't all respondents participate at their discretion ("opt-in")?
So then why is a callout survey seen as valid but an Internet survey is not? If a callout company places as many demo-specific calls as needed (1000? More?) to secure 100 respondents in a target demo, how are those results different from an Internet survey with an opt-in database? And regardless of who has signed up, only the results of 100 respondents in the target demo are looked at? (Assuming the veracity of every respondent, of course; respondent fraud is as much a possibility in callout as in Internet research.)
I await your comments, insights, thoughts, ramblings, musings, etc. on this topic! Thanks! - Anonymous
Anon: I'm glad you enjoy the column. Thanks.
When it comes to comments about research, I have a real problem—it frosts my shorts—when I hear someone say something like, "I have heard it said," or "Some people say," or any other similar comment made without providing a source. I don't know who told you this information, but it's obvious that he/she knows nothing about research. These types of people should avoid making any comments about research. OK, on to your questions . . .
Now, you say that you have "heard it said" that Internet music research isn't valid (what about reliable?) because it uses a "self-selecting" sample (formally called a volunteer sample. The reason I discredited your source is that someone with a research background would never say such a thing. Never. Ever. Why? Because, as I have said dozens of times in this column, in my research book, in articles in trade publications, and in interviews is this: All behavioral research (that is, research with human beings), uses volunteer samples—including Arbitron. I don't know of any behavioral research in radio, or in any other field, that uses a true random sample. (Random sample: Every person or element in the population under study has an equal chance of being selected.)
A true random sample in behavioral research is virtually impossible to achieve. A researcher may select a random sample of people, but when one person refuses to participate, the "true randomness" of the sample disappears. In order to be a true random sample, every person who is selected (using one or more of a variety of random selection approaches) would have to participate. This never happens, and all legitimate researchers know it never happens, which is why a person who knows anything about research wouldn't criticize Internet research because it uses a "self-selecting" sample. A true random sample has never been used in radio research and will never be used in the future. The proof of this is the necessity to 20,000 or more phone calls to complete a sample of 400 respondents.
OK, so a research study involves 400 or 500 respondents (or any size). What kind of sample is this? What are these respondents? What kind of sample is this? As I mentioned, it's a volunteer sample, not a random sample. I will repeat . . . no radio research uses a true random sample—all radio research uses volunteer samples, the same type of sample used as in all media research and all consumer research. Another repeat: All behavioral research uses volunteer samples. Got it?
So how does this relate to what you heard? Well, the main problem with Internet research in not that the sample consists of volunteers since all radio research uses volunteers, but rather that much of the Internet research conducted does not verify who is answering the questions or rating music. This isn't true (for the most part) with telephone research. Oh sure, I know that there are cheats and frauds in every facet of life, and I'm sure that some people can lie about who they are on the telephone, but Internet research has no way to verify who is answering the questions.
Now, since all radio research uses volunteer samples and all research can be affected by people who lie and/or cheat, it's necessary to do two things: (1) Use a large sample; and (2) Validate or verify all respondents' answers. It would be a mistake to blindly assume that all respondents' answers, regardless of how the data are collected, are accurate.
Verifying data involves looking for outliers, or people who differ markedly from the other respondents in the sample. Outliers usually answer in ways known as response sets, where their answers form a pattern—scores or ratings are all high, low, or neutral, or answers that just don't fit with the other respondents. These people must be eliminated from the sample.
In summary, what you heard is incorrect. What is correct is that Internet research is a good research methodology (valid and reliable) if the correct questions are used and if the respondents are identified (some companies have already developed very good ways to do this). The fact that Internet research uses volunteer samples is a moot point since all behavioral research does the same thing.
Behavioral research is a tough business because it involves human respondents who constantly change, have a difficult time expressing their thoughts, and misunderstand many things. The results for any behavioral research should not be interpreted as the literal truth, but rather as indications of what may or may not exist. That's why there it's necessary to interpret results within the guidelines of sampling error.
Volunteer samples are part of behavioral research and there is no getting around this fact. However, history shows that if a research study is designed correctly, volunteer samples tend to provide valid and reliable information.
Any questions? Oh, and don't take personally my comments about what I said in reference to "I heard it said." I was venting. (Did I repeat enough that all radio research uses volunteer samples?)
Do those banner ads I see about "boosting your internet speed" or "optimizing" really work? What does it do, when you download it. It sounds too good to be true to me, but then again I never tried it. - Apprehensive
Apprehensive: I checked with some computer people and the comments were pretty much the same—"Some work and some don’t. It all depends on what they’re selling." As with everything you buy, follow the rule: Caveat emptor.
With that in mind, go to this site to start your search for information: CNet.
Love the site, Doc. With the explosion of the web, what are the best metrics for analyzing a website's audience? Is it impressions, hits, unique users, or some other new metric? It seems like everyone has their own system, but for a small company looking to incorporate a web component to its marketing, it seems overwhelming and confusing. Anonymous
Anon: I'm glad you enjoy the column. Thanks. On to your question . . .
If you check some of the articles on the Internet about audience measurement, you'll see that the approach to audience documentation isn't much different from the other media. In other words, there are several approaches to pick from in order to develop a good story about a website.
While some advertisers are interested in unique hits, others are interested in length of time a person is on the website, and others want impressions. However, if I had to guess, I think the most often requested information is unique hits, similar to cume in radio. That is, what is your website's reach? How many different people visited your website during a specific period of time?
Interview for PD Position
Hi Roger, I was wondering if you could share a few of the questions that are tossed at candidates interviewing for a PD job? Thanks in advance for answers! - Anonymous
Anon: Maybe some readers who have gone through the process will submit some questions you may be asked, but here is a start (not listed in order of importance):
What is your programming philosophy?
How do you work with on-air talent?
How do you plan to increase the radio station’s numbers?
What makes a good PD?
What makes a good radio station?
Why do some radio stations fail?
What are your greatest strengths?
What are your greatest weaknesses?
What do you like most about your current job?
Why did you leave your previous job?
Where would you like to be in five years?
What is your salary/bonus requirement?
Why should we hire you?
Those are a few examples. However, you may also be asked:
Is it OK if your office is in a closet?
Can you go without breakfast, lunch, and/or dinner?
Will you work every weekend?
Will you take 100% of the blame if the station doesn’t do well?
Can you make the GM look like he/she knows what’s going on?
Are you allergic to Maalox?
Can we assume that you’re against long-term relationships and/or marriage?
How much do you spend for clothes each week at Goodwill?
We do provide our PDs with station vehicles. Would you prefer a bicycle or skateboard?
Do you prefer to live in an apartment or motel?
OK, so you may not be asked the questions in list #2.
I do a morning interview show and would like to be better at interviewing people. Are there any materials other than the $300 books available on the Internet? - Anonymous
Anon: Don’t make me come out there! I just did a search on Google for "interviewer skills," "interviewer tricks," "interviewing skills," and "interviewing tricks" and found several hundred good sources.
I also went to Amazon.com and did a search for books on "interviewing." There are more than 1,200 listed—most of which are around $20.00.
If that’s not enough and you live near a college or university, visit the school’s bookstore. Or, and I know this is probably foreign to many people, visit your local public library. You will find a lot of material on interviewing.
Hey Doc, great deal you got goin’ here. My question is about job interviews. What is the best answer to the dreaded question, “What would you do if you caught your co-worker/boss/friend/etc. doing something wrong?” It is asked at every interview, and honestly, I don’t think anyone who hasn’t been in that situation really knows what they would do, but that seems like a cheap way out answering it. What do you think? - Anonymous
Anon: Thanks for the comment about the column. I’m glad you enjoy it.
OK, so let’s see what we have here. I know this question can be debated forever, but to simplify the discussion, I think the question forces people into one of three major answer categories:
Report the person to authorities regardless of the perceived severity of the wrongful act.
Report the person to authorities only if the perceived act is considered too severe to overlook.
Don’t report the person to authorities regardless of the perceived severity of the wrongful act.
(Notice that I used the word, “perceived” in each answer. I did this because it’s possible that the observed wrongful act isn’t wrong at all, only a misperception. In this case, I think it’s necessary to begin the answer to this question with something like, “Assuming that the act I observed is truly a wrongful act, then….” proceed with your answer.
The answer you give to the interviewer provides an indication about how you handle ethical situations. What are your ethics? Do you report the person, or do you overlook the situation? Why be ethical? The best answer I know of is, “Because it’s the right thing to do.” You need to determine what is right and wrong, and if there are shades of right and wrong. I can’t tell you that. Only you know the answer.
But…here is one alternative answer to the question. A person might say,
“The first thing I would do is verify that a wrongful act was committed. I may have perceived the act to be wrong, but after further investigation, I may find that I misperceived the situation. However, let’s assume that a wrongful act was committed.
What I would do first is determine the severity of the wrongful act. If I perceived the act as extremely severe, I would report the person to authorities. However, if I perceived the wrongful act as misdemeanor or insignificant act, I would first talk to the person to encourage him/her to correct the situation without interference from authorities. (By insignificant, I mean an act that does not cause harm in any way to anyone or any property.) If the person did not remedy the wrong, then I would consider the option of reporting him or her to authorities.
In the end, it’s difficult to answer the question because I don’t know the exact situation. If I saw a co-worker/boss/friend/etc. commit murder, I would surely report the person to authorities. If I saw the same person take a pencil from someone’s desk, I probably would try to solve the problem without intervention from authorities. Being ethical is the right thing to do, but I also think that ethics occasionally involves personal judgment. In situations where there is no harm toward another person or against another’s personal property, then I would probably not involve authorities unless provoked by the person who committed the wrongful act.”
That’s one possible approach. I’m not saying it’s the right one. Only you can determine how to answer the question. Whatever you come up with will be right for you.
Hi doc! I have searched the Internet for sites dealing with inventions, but most are the same—“you send us money, we'll tell you if it's a good invention.” I’m hesitant to send my money, not to mention my idea, to these companies. Would you point me in the right direction? - Anonymous
Anon: Don’t deal with these companies. You’ll either give up a substantial percentage of the ownership of your idea, or your idea will be stolen. If you have a good idea, the best thing is to do the work yourself.
This isn’t easy, so don’t expect to get it all done in a few months. A copyright is simple, but if you go for a patent, expect the process to take 7-10 years and expect to invest several thousand dollars (for an attorney and filing costs).
By the way, you can search the U. S. Patent Office files to see if someone else has already received a patent for an invention similar to yours. Click here: Patent Search. This will give you a general idea. If you want a definitive answer, you should hire a patent attorney to conduct a detailed search (cost is about $500).
If some Wicks are Slicks, and some Slicks are Snicks, then some Wicks are definitely Snicks. True or False? - Stephen
Stephan: This is a question that is included on some IQ tests. The answer is “false” because only some Wicks are Slicks and only some Slicks are Snicks. Because the word “some” is used, there is no reason to conclude that Wicks = Snicks. In other words, the formula is not A=B and B=C, therefore A=C.
I recently took an IQ test, and I want to know what my score means. Are there basic criteria related to score segments, such as 75-100 means x, 101-130 means xx, etc.? If you don’t know the criteria, do you know where I can go to find out? My Internet searches to date have failed to give me what I am looking for. - Anonymous
Anon: I’m not sure which IQ test you took, but all IQ test categories/descriptions are based on Z-Scores and the normal curve. If you read this column frequently, you have seen Z-Scores mention often. Anyway, here are some basic descriptions of IQ categories (Average/Mean = 100; Standard Deviation = 15):
Below average: Under 85
Average: 85 - 115
Above average/Gifted: 116 - 130
Highly gifted (borderline genius): 131 - 145
Genius: 146 - 160
High genius: 161 - 185
Super genius: 186 - 200
Universal genius: 201+
Here is an article to find out your IQ percentile rank—where your IQ ranks among the general population. Find your IQ and look in the second column for your rank. For example, if your IQ is 130, your percentile rank is 97.7%…that is, you’re higher than 97.6% of the general population (approximately). Click here: IQ Percentiles.
IRS Email Scam?
Doc: I received an email that looks like it came from the IRS. It refers to the "economic stimulus payment" and has a link to click on to get information. Here is what I received:
If you haven't yet filed a tax return to get your stimulus payment, you still have time to do so. But you must file by November 25th to get your payment this year. For eligible individuals, the Economic Stimulus Act of 2008 provided for stimulus payments of up to $620.50 ($1,241 for married couples) or the amount of the taxpayer's 2007 net income tax liability, whichever is less. There also is a $310 payment for each qualifying child.
To access the Economic Stimulus
Act of 2008 CLICK HERE.
Note* If you received this message in your SPAM/BULK folder, that is because of the large amount of e-mails we are sending out or because of the restrictions implemented by your ISP.
Do you know if this is real or is it a scam? Thanks in advance. - Anonymous
Anon: This is a scam and you can learn a lot by reading information provided by the IRS—click here.
I found the email on the Internet and you can tell it's a scam by putting your cursor over the link to get more information. It's not an IRS link. The link shows that it's a directory on the scammer's website (/irs.gov/online_form/www.irs.gov). Don't click on the link. Just delete the email.
Is Research Ever Wrong?
Is research ever wrong? – Cindy
Cindy: Oh, sure it is. Research can be wrong because of a poor design, bad questions, incorrect sampling, violations of one of more sources of internal validity, incorrect interpretation, and other reasons. Just because something is called "research" doesn’t mean it’s correct.
Isaac Hayes - Hyperbolicsyllabicsesquedalymistic
One of my friends had you as a teacher at the University of Georgia and he said that you used to put a question about Isaac Hayes on your final exam. The problem is that the can’t remember the question. What did you ask? - Anonymous
Anon: I’d like to know your friend’s name so I can look up the grade I gave him. He can’t remember the question? Hmm… I’ll explain what I used to ask, but you need a little background information.
When I taught undergraduate school, I never took attendance in class because I expected everyone to be there for every class. However, there was always a day when several students were gone due to nice weather, too much drinking the night before, or something else. On these sparsely attended days, I would write this on the chalkboard:
I would tell the students something like, “This is one of my favorite songs. It’s by Isaac Hayes and it’s on his 1969 album titled, Hot Buttered Soul. It will be useful for you to remember the correct spelling of this song.”
For extra credit on the final exam, I asked this question:
Which of the following is the correct spelling for the song by Isaac Hayes on his album, “Hot Buttered Soul?”
e. None of the above
Here’s a little sideline…During the early 1980s, I was in a restaurant in Atlanta with several other college professors. While we were talking, I saw Isaac Hayes walk in with several of his friends. I said to everyone at our table, “Hey, there’s Isaac Hayes.” They all looked toward him and none of them agreed that is was Isaac.
After a few minutes arguing—“That’s him.” “Is not.” “Is to.” “Is not.” “Is to.”—I wrote Hyperbolicsyllabicsesquedalymistic on a napkin and said to one of my friends, “Take this to his table. If he recognizes this word, it has to be Isaac Hayes.”
This was an expensive restaurant—very crowded and very quiet. The “target” table was about 10 feet away. My friend went over to the table, introduced himself, and showed the napkin to the man. After looking at the napkin, the man said in a very loud and very low voice, “Who wrote this?” My friend pointed to me, and the man said, “Come over here, please.” I went to the table and the man said, “How’s does a white boy know how to spell this?” I told him it was one of my favorite songs, used it on tests in class, and about our conversation related to his identity. He said, “I recognize it. I’m Isaac Hayes and I did the song on Hot Buttered Soul” Victory was mine! (“Is so.”)
Isaac Hayes enjoyed the encounter and he signed the napkin for me.
(“c” is the correct answer in the above question)
It Seems Like
I have two questions…
1. It seems like I have heard quite a few PDs, consultants, and researchers claim that Gold songs (for a CHR format) do not test well in callout or Internet research. Is that an "it seems like" notion, or is there truth to it?
2. Does the 80/20 rule apply to your section at Allaccess.com? That is, do 80% of your questions come from 20% of your readers? (I am wondering because I have asked my fair share of questions. - Anonymous
Anon: Here are two answers…
1. Oh I’m sure that somewhere Gold songs for a CHR radio station don’t test well, but this is one of those things you need to test with your own listeners. These "it seems like" things emerge because someone will see Gold songs test poorly and therefore it must be true for all markets. Don’t believe it.
2. That’s an interesting question, but it’s impossible for me to answer since I don’t know where the questions originate. Most of the questions are signed "anonymous," and even if they weren’t, there is no track of your email address. For example, you say that you have submitted your fair share questions, but I don’t know who you are, and I don’t have any way to find out (unless you tell me). Your questions come to me completely anonymous.
"I Want to Take You Higher"
Doctor: I know this isn’t a significant question in the grand scheme of things, but it is something I need to know. Please help me.
A few days ago, my boyfriend and I were scanning channels on TV and some movie (or something) had “I Want to Take You Higher” by Sly and the Family Stone playing in the background. OK, so that’s the scenario.
The “famous” part of the song came on and my boyfriend sang, “Boom shaka laka laka boom shaka laka laka.” I told him that’s wrong, that they actually sing, “Boom laka laka boom laka laka”—there is no “shaka” in the lyrics.
He said he bets me anything that he is right. He says they sing, “shaka, and I say they don’t. Who is right? – Melanie
Melanie: When I first read your question, I thought, “I have been listening to this song since 1969…the boyfriend is right and Melanie needs to pay up (whatever “anything” is.)” Then I started thinking that it may be possible that something “don’t be right” and I first decided to search the Internet for the lyrics. Every site I found has the word “shaka” in it. There may be one that doesn’t, but I can’t find it.
OK, so Melanie is wrong and she has to pay up. But I thought I should listen to the song just to make sure….even though I have heard it hundreds or thousands of times since 1969. Give Melanie a break.
I leaned over to my bookcase and pulled out Sly and the Family Stone Greatest Hits CD. What the heck, I’ll give her a break. I put the CD in my computer and hit Track 1…”I Want to Take you Higher.” This is what I heard…and the important stuff comes from the right channel:
Beat is gettin’ stronger
Music gettin’ longer too
Music is a flashin’ me
I want ta, I want ta, I want ta take you higher
I wanna take you higher
Baby baby baby light my fire
Ha…Wanna take you higher
Boom laka laka laka Boom laka laka ook a boom
WHAT?? Where is the shaka stuff?
There is NO…Nada…None…Zilch use of the word “shaka” in the original Sly and the Family Stone version of “I Want to Take You Higher.” I can’t believe it. I also can’t believe that not ONE song lyrics site on the Internet has the correct lyrics. They are ALL wrong. Shame on them and good for you.
You can also hear and see them sing, "Boom Laka Laka" in several videos on YouTube if you click here.
OK, Melanie…you have not only changed my understanding of a song I have heard for decades, you have also won the bet with your boyfriend. Tell him pay up and pay up NOW. I hope you make the payoff REALLY good.
By the way…you also demonstrated one of the tenets (rules) of scientific research. Scientific research is self-correcting…scientists are always willing to change long-held beliefs (or facts) when new information proves that the old information is wrong. Good work.
All Content © 2015 - Wimmer Research All Rights Reserved