Music Testing Procedures: A Lesson in Arguments and Logic
Roger Wimmer, Ph.D.
There has been a lot of published information about radio station music testing methods. Much of the information has come from research company advertising; the remainder has come from infomercial articles written by columnists in a variety of media publications. Music tests are important to music radio stations because music is the product. It is equally important for people interested in music testing to investigate the information that is being disseminated.
A basic understanding of arguments and logic is necessary in order to analyze music test information. Why is this necessary? Because the type of information being presented must be assessed in order to make a fair evaluation. Just as with any advertising, debate, or argument (all of which use various forms of logic), we need to ensure that the information about music tests is valid (not fallacy). It would be a mistake to make a decision to use a new research methodology based on unsupported statements. If the supporting information is valid, then there is no problem. If the information is invalid, then something is wrong.
First, what is logic? Basically, logic is the science of reasoning, proof, thinking, or inference. Logic allows us to analyze a piece of information and determine whether it is correct or not. That is, we can determine if the information is valid or invalid. Logic comes in all forms. One major area of logic is fallacies, which are arguments that sound or appear valid but are not. Letís look a few types of logical fallacies. (Note: A discussion of logic involves the use of Latin terms. There is no way around it.)
Some logic fallacies to consider when analyzing information about music tests are:
Argumentum ad Hominem (abusive form): This translates literally toó argument (abusive) directed at man. A variation of this is an abusive argument directed at a concept, phenomenon, or other entity. Instead of trying to disprove the truth of something, the arguer attacks the person or people making the assertion, or attacks a concept, phenomenon, or entity.
Argumentum ad antiquitam: This fallacy asserts that something is right or good simply because it is old, or because "thatís the way it has always been." (The Method of Tenacity in the four Methods of Knowing.)
Argumentum ad novitam: The opposite of the argumentum ad antiquitam. This fallacy asserts that something is more valid and/or reliable simply because it is completely new or newer than something that already exists.
Red Herring: Irrelevant information is introduced to the issue being discussed so that a personís attention is distracted from the main point(s) and thereby toward a different conclusion.
Non sequitur: An argument where the conclusion is drawn from premises (conditions) that are not logically connected with it.
Argumentum ad baculum: (Also known as Appeal to Force.) This fallacy uses force or the threat of force to try to push the acceptance of a conclusion, such as "might makes right."
Circulus in demonstrando (circular argument): This fallacy occurs when a premise is used as a conclusion. Often, the premise is rephrased so that the fallacy appears to be a valid argument. For example, many books on logic contain this example: "God exists because the Bible tells us so. And we know that the Bible is true because it is the word of God."
Bifurcation: (Also known as Black and White fallacy): When a problem or decision is presented as having only two alternatives, when in fact other alternatives exist or can exist.
Argumentum ad Logicam: This is the "fallacy fallacy" that argues a premise is false on the grounds that it has been presented as the conclusion of a fallacious argument.
Donít misunderstand the purpose of this article. It is not to criticize or condemn new research methodologies or refinements of current research methodologies. The true scientist/researcher has one major goalóto expand the base of knowledge and understanding about the topic(s) under investigation. There is nothing wrong with developing new radio research methodologies.
There is also nothing wrong with improving or refining radio research methodologies. What is wrong is promoting new methodologies without following the appropriate scientific steps. Information gained from the Scientific Method of learning is unique among the ways we learn. Among other tenets, scientific research is objective and self-correcting. Information presented from the use of the Scientific Method must stand up to public scrutiny. Ideas, concepts, hypotheses, theories, or laws cannot be stated merely by saying, "It is true because I/we say it is true." Two University of Utah chemists found out what can happen when a "fact" is presented without supporting information. In 1992, the professors claimed to have accomplished nuclear fusion at room temperature, only to be denounced later when other chemists could not verify their results in replicated (repeated) studies. The chemistsí reputations will likely be tarnished forever.
Music test information can be important to music radio stations. If the test is conducted properly using a valid and reliable measurement instrument (the scale used to test the songs), and the results are interpreted properly, the data can help a PD determine which songs should and should not be played on the air. For this reason, it is important that we completely examine any claims made for new research procedures.
From numerous studies on auditorium music tests (going back to 1982), we know that the auditorium music test is valid and reliable method for testing a respondentís opinions about music. Some of the many things that are know about auditorium testing include:
1. The control of the situation ensures that extraneous variables (outside influences) are minimized. The situation does not cause artificialityóa respondent who likes or hates a song in an auditorium will rate the song the same way in other situations.
2. Respondent fatigue is not present in tests of up to 600 titles.
3. Spouses or close friends invited to the same test do not negatively affect the results of the test.
4. Song order does not affect song scores.
So what? The "so what" is that changes in research methodologies are expected; they are also welcome because they may produce better information. However, before a new methodology or a refinement to an old methodology is accepted, it is necessary for anyone who uses the method to analyze the available information. In essence, are the arguments valid or are they merely one of the fallacies presented in the beginning of the discussion?
If you arrived here via a search engine, click here.