Your true feelings on the diary or PPM's

[davideduardo]

From today's reports, it doesn't sound like 200 agrees with you:

The issue is totally different than diary when weekly cume participation changed every week. As you know, with the PPM, the participant's usage may be tracked for years.

The issues of the last month had to do with Arbitron applying a stricter boot-from-panel criteria at the same time that summer began. They simply could not replace the panel losses fast enough. They have added staff, and instituted new procedures, and should have the average daily in tab back at around 900 by late September.

The system is not going away. The system is being perfected to get better response rates, and more backup panelists are being obtained. It's still a lot better than the diary with a couple of hundred booklets a week... vs. nearly 1000 a day for PPM.

The average on-panel time is about 8 months. And the longest period Arbitron issues a report for is 28 days.

 

[InTIMadate]

The issues of the last month had to do with Arbitron applying a stricter boot-from-panel criteria at the same time that summer began. They simply could not replace the panel losses fast enough. They have added staff, and instituted new procedures, and should have the average daily in tab back at around 900 by late September.

Not true: Those records were recorded but not reported or caught by Arbitron quality control systems. This had nothing to do with boot-from-panel criteria, according to Arbitron. That's another serious issue entirely.

ARBITRON SVP/Press And Investor Relations THOM MOCARSKY released the following: "There was a data processing error that impacted the sample report and weekly cume estimates in HOUSTON PPM Weekly report for JULY 26th through AUGUST 1st, 2007. There was no precipitous decline in panelist cooperation in HOUSTON for that week.

The file used to process the weekly cume estimates (the unified weekly sample) was incompletely loaded from the data warehouse to the production system. It appears that exposure records for almost 200 panelists who were in-tab 6 of 7 days were not included in that file transfer. This is why the average (unified) weekly in-tab appears to be abnormally low. "

According to All Access: down about 33%.

 

[davideduardo]

Not true: Those records were recorded but not reported or caught by Arbitron quality control systems. This had nothing to do with boot-from-panel criteria, according to Arbitron. That's another serious issue entirely.

I meant the sample frame issues of not meeting in-tab quotas. A data processing error is fixable, and it was handled in 3 days. I downloaded my file yesterday at noon, and most of the information except weekly cume was the same.

The weeklies are not currency. They do not go to clients, and are for internal tracking. To expect everything to work in a new system is ingenuous; I have had two books reissued due to diary review... two months after the data collection was finished. Now we have Arbitron reissuing a correction in 3 days. I think that is pretty good.

The current panel size is fine, and it will increase to the full guarantee in less than 60 days. Replication studies show the sample is actually higer than needed, so there is good confidence in the data... but we don't want, as subscribers, less than what we are paying for or to risk excessive weighting on any cell.

 

[Johnny.Sunshine]

A data processing error is fixable, and it was handled in 3 days.

DavidEduardo, has Arbitron disclosed the ethnic in-tabs by age/sex, or the ppmv as promised? Just curious...

 

[davideduardo]

DavidEduardo, has Arbitron disclosed the ethnic in-tabs by age/sex, or the ppmv as promised? Just curious...

The only report so far is the sample report, which shows Blacks and Hispanics on 6+ and 12+, with no breaks.

We still see a huge issue with Hispanic females. Examples: KLTN #3 18+ men, #17 18+ women. KOVE #4 18+ men, #14 18+ women. We know there is less TSL in the PPM by women than by men, but the difference is not anywhere near what the Spanish langauge stations reflect in male-female imbalance (TSL, when looking at share in men and women, should not be a factor anyway as we ware comparing women to women and men to men).

 

[michaelshiloh]

Either way, though, the PPM is shaking things up in Houston and that's a good thing.

 

[davideduardo]

Either way, though, the PPM is shaking things up in Houston and that's a good thing.

Yeah, it's great that it shows radio listening off 40% and radio revenues are considerably off for the market.

 

[michaelshiloh]

I'm not sure what your point is, David. If the PPM method more accurately reflects the true marketplace, even if that truth is painful, it's a good thing. If you're saying radio revenues in Houston are off because the PPM is inaccurate, that's another thing.

 

[davideduardo]

I'm not sure what your point is, David. If the PPM method more accurately reflects the true marketplace, even if that truth is painful, it's a good thing. If you're saying radio revenues in Houston are off because the PPM is inaccurate, that's another thing.

The fact that radio revenues are off significantly is due to two things... agencies are very cautious due to the data glitch two weeks ago and the iability to get a good metric comparison with the diary method... so they are buying less radio overall. When this happens, stations add commercial uints to make up for the difference. End result: we will be back to 15 ant 18 minutes of spots an hour on music stations.

Many of us are not yet satisfied that Arbitron has its panel correctly built demograpnhically and we know that the panel is undersized and not totally proportional. The PPM may, indeed, be the death of radio as we know it.

 

[michaelshiloh]

Look, David, I know you to be a serious booster and defender of radio as an advertising medium. Your remark about the death of radio as we know it is a little harsh, don't you think? With an increased sampling size and the stabilizing of the system later on, the PPM will be an accurate reflection of what people listen to, without the "ballot box stuffing" and weighting of the diary system. I guess you disagree?

 

[davideduardo]

Look, David, I know you to be a serious booster and defender of radio as an advertising medium. Your remark about the death of radio as we know it is a little harsh, don't you think? With an increased sampling size and the stabilizing of the system later on, the PPM will be an accurate reflection of what people listen to, without the "ballot box stuffing" and weighting of the diary system. I guess you disagree?

The situation may play out favorably, but I have called on too many agencies to believe they are going to happily pay 40% more per point. So stations, faced with a decline in AQH persons, will be cutting deals. And that means adding units. At some point. the listeners pick alternative entertainment sources.

I think this a a critical turning point for radio; I am not encouraged by the potential outcome.

 

[wgliradio]

The fact that radio revenues are off significantly is due to two things... agencies are very cautious due to the data glitch two weeks ago and the iability to get a good metric comparison with the diary method... so they are buying less radio overall. When this happens, stations add commercial uints to make up for the difference. End result: we will be back to 15 ant 18 minutes of spots an hour on music stations.

Many of us are not yet satisfied that Arbitron has its panel correctly built demograpnhically and we know that the panel is undersized and not totally proportional. The PPM may, indeed, be the death of radio as we know it.

Thus, the business model has failed radio. Leave it to radio to rely on a third party for its survival... and now that third party is dropping the ball.

How difficult would it have been to a transitional period where we had some time with both systems in play?

 

[davideduardo]

Thus, the business model has failed radio. Leave it to radio to rely on a third party for its survival... and now that third party is dropping the ball.

There is no failure of the business model. There are difficulties in recruiting persons for research, whether it is for raido or diapers.

How difficult would it have been to a transitional period where we had some time with both systems in play?

Showing how little you know, there were 24 months of PPM tesing in parallel with the PPM.

 

[wgliradio]

There is no failure of the business model. There are difficulties in recruiting persons for research, whether it is for raido or diapers.

This research is key to the business model. If it fails....

Showing how little you know, there were 24 months of PPM tesing in parallel with the PPM.

Funny, we've only had PPM installed for 5 months in Market #1. Doesn't sound like 24 to me. And NY is going PPM for the next book.

 

[davideduardo]

There is no failure of the business model. There are difficulties in recruiting persons for research, whether it is for raido or diapers.

This research is key to the business model. If it fails....

Funny, we've only had PPM installed for 5 months in Market #1. Doesn't sound like 24 to me. And NY is going PPM for the next book.

Houston had 24 months of PPM prior to the service going currency. Arbitron has been running the NY panel since nearly the beginning of the year. That is plenty of time to build a panel and recruit as proportional sample and to work out any technical bugs in the encoding, etc.

They tested in parallel with the diary in Philly for two years, also. How much testing do you suggest they do?

Stations are not going to pay for parallel research, as the PPM is already 60% more expensive than the diary. One thing is to test the system in multiple markets, another is to do simultaneous studies in all markets. Nobody will absorb the cost, and Arbitron can not reasonably be expected to take years of losses to have parallel data.

The PPM concept has not failed; it got MRC certification. What we are seing now are issues of long-term compliance, sample balancing, daily in-tab and such. Arbitron is generally responsive, but some issues will always come up over time. The diary based book and sample are very different from that which Arbitron rolled out in 1965... things like HDHAs and HDBAs, more follow up calls, DST (incentives, etc), elimination of personal placement and retrieval, addition of more stratification variables, like language preference among Hispanics, redefinition of metros, etc., have happened regularly and in response to "new" issues each time.

 

[wgliradio]

Houston had 24 months of PPM prior to the service going currency. Arbitron has been running the NY panel since nearly the beginning of the year. That is plenty of time to build a panel and recruit as proportional sample and to work out any technical bugs in the encoding, etc.

Obviously it's not enough, or the system itself is flawed to the point where it just may not work. I quote you

Many of us are not yet satisfied that Arbitron has its panel correctly built demograpnhically and we know that the panel is undersized and not totally proportional. The PPM may, indeed, be the death of radio as we know it.

They tested in parallel with the diary in Philly for two years, also. How much testing do you suggest they do?

Obviously until they truly feel the numbers with PPM look right. During this 24 month PPM test, what did the data show compared to the old diary method? Did Arbitron make those numbers available to subscribers so they could see the type of data collected vs the old system?

Stations are not going to pay for parallel research, as the PPM is already 60% more expensive than the diary. One thing is to test the system in multiple markets, another is to do simultaneous studies in all markets. Nobody will absorb the cost, and Arbitron can not reasonably be expected to take years of losses to have parallel data.

It is in the best interest of broadcasters, which rely heavily on this service as part of its business model, to expect a drastic change in the way data is gathered. Broadcasters interested in ensuring accurate data gathering should absorb the cost to study the differences in the data.

The PPM concept has not failed; it got MRC certification. What we are seing now are issues of long-term compliance, sample balancing, daily in-tab and such. Arbitron is generally responsive, but some issues will always come up over time. The diary based book and sample are very different from that which Arbitron rolled out in 1965... things like HDHAs and HDBAs, more follow up calls, DST (incentives, etc), elimination of personal placement and retrieval, addition of more stratification variables, like language preference among Hispanics, redefinition of metros, etc., have happened regularly and in response to "new" issues each time.

But these were tweaks to the overall system over time that evolved it. This is a total move to a new system

 

[davideduardo]

Obviously it's not enough, or the system itself is flawed to the point where it just may not work.

It works, but is still being perfected. There is not any way to run the two systems in parallel any longer. We simply have to hope that the issues that have arrisen will be worked on and fixed.

I quote you
Many of us are not yet satisfied that Arbitron has its panel correctly built demograpnhically and we know that the panel is undersized and not totally proportional. The PPM may, indeed, be the death of radio as we know it.

Those are two issues. One is getting the panel the right size and fully proportional, and the other is the sales issue coming out of a system that shows much less total listening.

Obviously until they truly feel the numbers with PPM look right.

And that is being done as the system is rolled out; the numbers will "look" better as the panel is improved so we get the target daily and weekly in-tabs. Nobody knows what "right" is since this is a new system.

During this 24 month PPM test, what did the data show compared to the old diary method? Did Arbitron make those numbers available to subscribers so they could see the type of data collected vs the old system?

The numbers were given to subscribers in the test markets, and, towards the end of Houston testing, to those subscribed to PPM. They are the subject of confidentiality agreements. In addition, a few of us were invited to receive the Philly numbers around 2002 and 2003, as the first large US test was done. We had to sign confidentiality there, too.

It is in the best interest of broadcasters, which rely heavily on this service as part of its business model, to expect a drastic change in the way data is gathered. Broadcasters interested in ensuring accurate data gathering should absorb the cost to study the differences in the data.

Broadcasters don't have the resources to do this. It might have been nice, but nobody would have paid for it. That's the difference between reality and utopia.

But these were tweaks to the overall system over time that evolved it. This is a total move to a new system

The changes I mentioned, and many more, were significant ones that affected the listening measurements and rank of many stations. They were hardly tweeks... they were changes so significant that the MRC required over a year of testing in some cases to give certification. For example, the change to language preference took about four years from the agreement to make the change to full implementation.

 

[wgliradio]

But the changes to the old diary method is not nearly as drastic or different as this.

 

[davideduardo]

But the changes to the old diary method is not nearly as drastic or different as this.

Since you have obviously been working closely with Arbitron, on industry committees, and participating in advisory groups for four decades, I'll take your word for it.

I simply thought that they had changed paper and pencil for a digital recorder.

 

[jd]

Folks, you may find this interesting. Call it a "Hitchhiker's Guide to The PPM Galaxy." Agree or not, it addresses some important issues:

www.paragonmediastrategies.com/theblog/?page_id=73

Maybe you saw Paragon's "PPM Rulebook" earlier, which started with ten rules. Now they've added three more, including #11, which reads in part: “View all research with skepticism.”