SF PPM undexplained swings

[Geek-O-Rama]

From Inside Radio-
Despite conversion to PPM, ratings “bounce” is alive and well. Last May, long-running smooth jazz bastion KKSF, San Francisco flipped to classic rock, giving Inner City’s urban AC “The Quiet Storm” KBLX a ratings lift. Among persons 25-54, KBLX grew from a 2.7 in April to a 3.7 in May, then a 4.1 in June and a 5.2 in July. Then something changed. KBLX fell precipitously to a 3.5 in August, followed by a 3.1 in September. The ratings roller coaster continued in the fall and winter: 3.7 (October), 4.6 (November), 4.5 (December), 4.7 (Holiday), 4.1 (January). From last May to January, the station ranked as high as second and as low as eleventh. Welcome to the world of PPM ratings bounce. One promise of electronic measurement was that it would greatly reduce or eliminate bounce caused by sample fluctuations. But in San Francisco and other markets, there are sometimes wide ratings swings that can’t be explained by changes in programming. “Even if KBLX picked up every single person [KKSF] lost, you still wouldn’t get to KBLX’s July number,” Inner City Broadcasting-New York VP/GM Deon Levingston says. Between February and July 2009, the station’s 25-54 AQH persons more than doubled, from 7,500 to 15,100. Ratings bounce isn’t confined to urban stations. In a big rock and sports market like Chicago, you wouldn’t expect hot AC to be #1 in men 25-54. But that was the case for Bonneville’s “101.9 The Mix” WTMX last summer. The station was tracking in the low-to-mid three-share range in the demo from October 2008-February 2009. Then it jumped into the low fours in March and April and the low fives in May, June and July, making it first or second in men 25-54. It’s since fallen off and ranked 14th in the demo with a 3.0 in January. “There’s always going to be sampling error around whatever sample you select,” Arbitron VP of research methods and quality Beth Webb says. “The PPM has more stability month to month and is better than the diary at providing real trends. But we still have sampling variation and panel turnover and that affects the estimates

 

[SFStatic]

You have approx. a third as many meter panelists as there were diary keepers. When you lose a meter, it makes a HUGE difference. The swings are far greater than they ever were with the diary. This is especially true with the weekly data. ARB itself even advises in their training not to get too exercised about the weekly data. With some software, you can actually watch a graph and see the number of meters fall off while listening to programming. The most dramatic example of this I saw was when a Rosanne ad for Consolodated Resorts came on. The meters dropped to zero almost instantly!

 

[Jay F]

It looks like Urban AC listeners dropped off the panel and were replaced by classic rock fans. Both KBLX and KISQ were below their normal range and both KKSF and KSAN were up in Feb. Just a few changes in the panel must be able to make that kind of impact..especially considering the relatively low sample size.

 

[davideduardo]

You have approx. a third as many meter panelists as there were diary keepers. When you lose a meter, it makes a HUGE difference. The swings are far greater than they ever were with the diary. This is especially true with the weekly data. ARB itself even advises in their training not to get too exercised about the weekly data. With some software, you can actually watch a graph and see the number of meters fall off while listening to programming. The most dramatic example of this I saw was when a Rosanne ad for Consolodated Resorts came on. The meters dropped to zero almost instantly!

In fact, the daily and weekly sample size is about 5 times that of the diary. And the daily and weekly sample size is, for all practical purposes the same as the full book. That is because the entire sample is active for the entire book, not just 8% of it in each week as we had with the diary.

We can no look at individual days and dayparts with the same accuracy we looked at the whole book before.

The ability to track tune ins and tune outs is one of the major plusses of the pPM

 

[SFStatic]

So the 2400 some meters add up to the 6000 some diaries in SF. Yeah, that's practically the same number. Plus, panelists can stay on for a year and a half, so stations that lose loyalists to their stations take a huge hit when those panelists leave. I guess I just didn't understand what Arbitron told us about this or what the software showed.

 

[RadioStarOne]

The PPM is totally flawed if just because a station is off the air for a brief period and the ratings take such a large dive.

 

[DavidKaye]

The PPM is totally flawed if just because a station is off the air for a brief period and the ratings take such a large dive.

But is this not a more accurate reflection of listening than diaries? If a station goes off the air for a brief time, say 20 minutes, the listener may be away for 21 minutes or a full day. But diary wouldn't reflect that listener being away at all.

 

[SFStatic]

As ARB will tell you, it isn't a reflection of listening at all...just of what the panelist hears. If you walk through a store shopping for more than 5 minutes, and are carrying a meter, and KOIT is on the storecasting, they will get credit for you listening to them for that period. This is what happens, even though you had no clue it was them, and paid no attention to it. With the diary , people did write down more of their favorites, and over-emphasized the listening to them, so both the diary and the meter have their flaws. Overall, with all the problems, I believe the meter is the better measurement. We can only hope they fix some of the obvious problems, like low participation of some demos that really screw certain formats.

 

[davideduardo]

So the 2400 some meters add up to the 6000 some diaries in SF.

In the PPM, there are around 2000 metered persons participating each day/wek. In the diary method, it was around 550 (and for most of the market, the equvalent of about 400, since San José was oversampled to be able to make an MRC accredited separate report).

This means that one can look at daily and weekly data... even daypart data, with the same accuracy as the full book. That is what both stations and agencies wanted. With the diary study, an individual day or daypart had too small a sample, and, what is more, a sample with no effort at proportionality. The diary was balanced and weighted on the 12 week period. In a panel, the sample is balanced always.

Yeah, that's practically the same number.

In many senses, it's a bigger number for the expressed purpose, which is more granularity and faster data delivery.

Plus, panelists can stay on for a year and a half, so stations that lose loyalists to their stations take a huge hit when those panelists leave.

Panelists don't leave. Panel households leave. And they can stay up to 2 years, which is the objective of a panel. With a significant sample the adding and removing of households has minimal effect.

I guess I just didn't understand what Arbitron told us about this or what the software showed.

I concur.

 

[davideduardo]

As ARB will tell you, it isn't a reflection of listening at all...just of what the panelist hears. If you walk through a store shopping for more than 5 minutes, and are carrying a meter, and KOIT is on the storecasting, they will get credit for you listening to them for that period. This is what happens, even though you had no clue it was them, and paid no attention to it. With the diary , people did write down more of their favorites, and over-emphasized the listening to them, so both the diary and the meter have their flaws. Overall, with all the problems, I believe the meter is the better measurement. We can only hope they fix some of the obvious problems, like low participation of some demos that really screw certain formats.

One of the things agencies (for whom radio buys ratings) wanted was measurement of exposure.

On the other hand, Arbitron has a committee working to explore the creation of an index or figure showing commitment or engagement with each station, based on a very sophisticated metric... so we may have an even better indicaator of that then we did with the diary. The diary often did show long listening to a station someone else put on at work, so this is nothing new. Those little 5 minute exposures really don't do anything except bring down the total TSL for a station, so they are of no benefit.