Re: A Simple "Yes" would of been sufficent... (eom)
> I am.
>
> But even in a your strong defense of Arbitron, you have to
> revert to statements like "wobble" & "wander".
Wobble is a statistical term. In fact, statistics is the only science where "error" is not a dirty word.
As long as radio can not do a census of every citizen every day, we will use some form of a probability sample (trying to make it a random sample at the same time). Whenever you do a sample of a highly variable universe, you will have a margin of error. Such margin of error is easily calculated for the degree of confidence one wishes in the sample.
Sample size is almost always determined by a trade off of cost and margin of error; what is the biggest one-standard-error-range that is tolerable within the range of meoney we are willing to spend.
Sample error and sample to sample wobble will affect any listening poll, irrespective of the methodology used.
Wobble is not a diary issue. Wobble is not a people meter issue. Wobble is not a telephone coincidental issue. It is a "law" of statistics.
>
> Doesn't it make you the least bit uneasy that our
> "customers" (ie advertisers) are so approving of the system?
> Maybe morning drive spots should be selling for 10x what
> they're going for. At this point, we simply don't know
We do know within a very tight range. Bigger stations seldom wobble by more than around 10% or so, and much of that is due to the competitve array int he market also changing, not the sample. Smaller stations (in terms of share and cume) wobble more as there are few "evidences" of listening, so one or two diaries can affect the results more.
Keep in mind that advertisers, as I said, use multi-book averages to flatten the waves, and they know that any sample has inherent error.
One proof of the validity of Arbitron is that many radio companies do thier onw in house tracking, and find that they see the same numbers (even thougth they only track themselves and direct competitors).
The ability to replicate results proves a sample to be valid. For example, the reason we use 100 person AMTs is because you can replicate exactly 9 out of 10 times with a sample of 80 persons done over and over, so 100 guarantees that the variables of show rate by demo will not ruin the sample. That is righ... you can test your music with 100 people or 1000 and the changes will be insignificant as you make the sample larger and larger.
>
> The similiar results produced by the PPM shouldn't come as
> any surprise to anybody. It's basically the same system,
> produced by the same company, with a diffenent method of
> gathering of data.
No, it is a totally different system.
The diary is a new sample every week, consisting of 8.5% of the total book sample. The PPM has a panel, and it is the same sample every minute, hour, day, week and month. The diary is active, and depends on recall. The PPM is passive, and automatically tracks listening... even phantom listening. The PPM has a near-perfect panel which is proportional on every one of about 20 stratification variables, while the diary has a sample that can vary weekly in make-up, and has to be adjusted for by placement balancing as the first diaries come in, and weighted in each cell to some extent to achieve proportionality.
You could not find tow more different research techniques, no matter who does them.
> It would be pretty embarrasing for them
> to come up with a set of radically different answers.
They knew this was statistically impossible going in. In fact, both methods are MRC approved... and no other method is. The MRC is not going to approve a method that wastes the money of the advertisers and agencies that support it.
> If
> you compare poor results with more poor results that doesn't
> confirm the previous poor results... It only means you got
> more poor results.
You are building a straw man. Nobody questions Arbitron's systems. They do question the delivery time of the diary book, the declines in response rates, the difficulties involving cell phone homes, 18-24 male sampling, etc. But any company using any method is going to have the same problems doing a poll.
Again, there is no "other" method that costs within the same range and that would get MRC approval. Without MRC approval, the users of the data would not emply it... meaning the advertisers.
>
> And just by saying that statistical results are accepted
> does not mean they are necessarily accurate... It only means
> they are accepted.
See above. Any poll will have the same margins of error based on universe and sample size within the universe. Unless you increase the sample by perhaps 20 to 30 times, you will still get the kind of wobble you see today IN ANY SYSTEM of measurement.
>
> I contend that radio will continue to sell itself short up
> until the day they can accurately pin point your listener
> and get inside their head. Unfortunately, I don't have that
> system developed yet and if I did, you can bet I wouldn't be
> posting it on a message board.
Other than having every US resident implanted with a computer chip, there is no other method besides a sample of the universe. And all samples can be seen to have error in proportion to size and the universe being measured.
>
> On the other hand, I guess the possiblity exists that radio
> is over-selling itself...
When I can put a dozen or so people and some software on a computer to work calling a market for a couple of days and get the same ranking of stations Arbitron does, I think we can say that the results are very close to reality... close enough for advertisers not to waste a second of time debating the system.
> I am.
>
> But even in a your strong defense of Arbitron, you have to
> revert to statements like "wobble" & "wander".
Wobble is a statistical term. In fact, statistics is the only science where "error" is not a dirty word.
As long as radio can not do a census of every citizen every day, we will use some form of a probability sample (trying to make it a random sample at the same time). Whenever you do a sample of a highly variable universe, you will have a margin of error. Such margin of error is easily calculated for the degree of confidence one wishes in the sample.
Sample size is almost always determined by a trade off of cost and margin of error; what is the biggest one-standard-error-range that is tolerable within the range of meoney we are willing to spend.
Sample error and sample to sample wobble will affect any listening poll, irrespective of the methodology used.
Wobble is not a diary issue. Wobble is not a people meter issue. Wobble is not a telephone coincidental issue. It is a "law" of statistics.
>
> Doesn't it make you the least bit uneasy that our
> "customers" (ie advertisers) are so approving of the system?
> Maybe morning drive spots should be selling for 10x what
> they're going for. At this point, we simply don't know
We do know within a very tight range. Bigger stations seldom wobble by more than around 10% or so, and much of that is due to the competitve array int he market also changing, not the sample. Smaller stations (in terms of share and cume) wobble more as there are few "evidences" of listening, so one or two diaries can affect the results more.
Keep in mind that advertisers, as I said, use multi-book averages to flatten the waves, and they know that any sample has inherent error.
One proof of the validity of Arbitron is that many radio companies do thier onw in house tracking, and find that they see the same numbers (even thougth they only track themselves and direct competitors).
The ability to replicate results proves a sample to be valid. For example, the reason we use 100 person AMTs is because you can replicate exactly 9 out of 10 times with a sample of 80 persons done over and over, so 100 guarantees that the variables of show rate by demo will not ruin the sample. That is righ... you can test your music with 100 people or 1000 and the changes will be insignificant as you make the sample larger and larger.
>
> The similiar results produced by the PPM shouldn't come as
> any surprise to anybody. It's basically the same system,
> produced by the same company, with a diffenent method of
> gathering of data.
No, it is a totally different system.
The diary is a new sample every week, consisting of 8.5% of the total book sample. The PPM has a panel, and it is the same sample every minute, hour, day, week and month. The diary is active, and depends on recall. The PPM is passive, and automatically tracks listening... even phantom listening. The PPM has a near-perfect panel which is proportional on every one of about 20 stratification variables, while the diary has a sample that can vary weekly in make-up, and has to be adjusted for by placement balancing as the first diaries come in, and weighted in each cell to some extent to achieve proportionality.
You could not find tow more different research techniques, no matter who does them.
> It would be pretty embarrasing for them
> to come up with a set of radically different answers.
They knew this was statistically impossible going in. In fact, both methods are MRC approved... and no other method is. The MRC is not going to approve a method that wastes the money of the advertisers and agencies that support it.
> If
> you compare poor results with more poor results that doesn't
> confirm the previous poor results... It only means you got
> more poor results.
You are building a straw man. Nobody questions Arbitron's systems. They do question the delivery time of the diary book, the declines in response rates, the difficulties involving cell phone homes, 18-24 male sampling, etc. But any company using any method is going to have the same problems doing a poll.
Again, there is no "other" method that costs within the same range and that would get MRC approval. Without MRC approval, the users of the data would not emply it... meaning the advertisers.
>
> And just by saying that statistical results are accepted
> does not mean they are necessarily accurate... It only means
> they are accepted.
See above. Any poll will have the same margins of error based on universe and sample size within the universe. Unless you increase the sample by perhaps 20 to 30 times, you will still get the kind of wobble you see today IN ANY SYSTEM of measurement.
>
> I contend that radio will continue to sell itself short up
> until the day they can accurately pin point your listener
> and get inside their head. Unfortunately, I don't have that
> system developed yet and if I did, you can bet I wouldn't be
> posting it on a message board.
Other than having every US resident implanted with a computer chip, there is no other method besides a sample of the universe. And all samples can be seen to have error in proportion to size and the universe being measured.
>
> On the other hand, I guess the possiblity exists that radio
> is over-selling itself...
When I can put a dozen or so people and some software on a computer to work calling a market for a couple of days and get the same ranking of stations Arbitron does, I think we can say that the results are very close to reality... close enough for advertisers not to waste a second of time debating the system.