RATINGS METHODOLOGY

[bossbill]

There's been discussion comparing PPM to Diary. Has anyone seen side-by-side comparisons of diaries vs. telephone methodology? Maybe looking at an Arbitron market that also conducted an Eastlan at the same time?

I'm curious because I see stations making significant changes based on PPM measurement, but wouldn't it be more thorough to compare the methodologies and make decisions based on more factors?

I recognize that PPM indicates actual listening, but says little about preferences, or "hearing" the messages, and the other ratings services have disadvantages as well. That's why I would be interested in the side-by-side comparisons.

 

[PTBoardOp94]

I don't know of any studies, but anecdotally Eastlan ratings are far inferior to even Arbitron's diary method. Unless you're the type who believes that the #1 station in a market will swing from country to hot AC and back in the span of 3 books or that an NPR station airing 18 hours a day of classical music will be in the top ten consistently.

 

[davideduardo]

I don't know of any studies, but anecdotally Eastlan ratings are far inferior to even Arbitron's diary method. Unless you're the type who believes that the #1 station in a market will swing from country to hot AC and back in the span of 3 books or that an NPR station airing 18 hours a day of classical music will be in the top ten consistently.

Telephone (or personal) interviews and the diary both depend first on memory, and then station listening.

The diary at least has the advantage of true weekly cumes, which the 24 hour recall method, aided or unaided, does not have.

 

[SaltyDog]

on listening.

The diary at least has the advantage of true weekly cumes, which the 24 hour recall method, aided or unaided, does not have.

Interesting. What makes the diary method "true" and the "24 hour recall method" untrue? I've generally viewed the diary method as 7 day recall since my diary reviews left me with the impression that diaries are filled out at the end of the week.

 

[davideduardo]

on listening.

The diary at least has the advantage of true weekly cumes, which the 24 hour recall method, aided or unaided, does not have.

Interesting. What makes the diary method "true" and the "24 hour recall method" untrue? I've generally viewed the diary method as 7 day recall since my diary reviews left me with the impression that diaries are filled out at the end of the week.

I've reviewed diaries starting with my first visit to Beltsville in 1970 and I never say much evidence of entire diaries being filled in on Wednesday when the diary week ends. I have not done a diary review for a couple of years since PPM has been currency in the majority of the markets I have been involved with, but I did not see much change over the near-40 years before that.

Of course, there was evidence that daily diary keeping improved as Arbitron instituted multiple call-backs to each diary household during the week. While no doubt people would fill in Saturday and Sunday at the end of the weekend, that only points out the advantages in the PPM.

My point about the diary is that it records 7 days of listening, and thus can pick up single day incidents to create a true weekly cume... a key element in doing reach and frequency.

The Pulse method, the 24 hour aided recall system, was based on one call or personal visit where people were asked what they listened to in each daypart (that's the aided part: "and this morning from 6 to 10 what did you listen to?"). Because a "live" person did the interviews, there was a tendency to answer with the stations and behaviour that was "acceptable" and not always reality. I even saw this in the field when accompanying survey crews for that methodology... people not naming a station you could hear from the front door because the station was perhaps considered to be in poor taste....

Any measurement is better than none at all. I've been in what is now a top 15 market that had no measurement for two years in the 70's... billing for the market fell by over 50% without a way to justify pricing. But some methods are just better than others for the purpose of establishing a metric for advertisers.

 

[SirRoxalot]

Because a "live" person did the interviews, there was a tendency to answer with the stations and behaviour that was "acceptable" and not always reality. I even saw this in the field when accompanying survey crews for that methodology... people not naming a station you could hear from the front door because the station was perhaps considered to be in poor taste....

You don't really think that this is a phenomena that only affects the 24-hour recall methodology, do you? The effectiveness of TV advertising during a book, and the variance between diary and PPM results indicates that the same phenomenon happens when people write down their listening over a longer period of time as well. People tend to be more conscious of the "propriety" of what they write than what they say, especially on the phone.

 

[davideduardo]

Because a "live" person did the interviews, there was a tendency to answer with the stations and behaviour that was "acceptable" and not always reality. I even saw this in the field when accompanying survey crews for that methodology... people not naming a station you could hear from the front door because the station was perhaps considered to be in poor taste....

You don't really think that this is a phenomena that only affects the 24-hour recall methodology, do you? The effectiveness of TV advertising during a book, and the variance between diary and PPM results indicates that the same phenomenon happens when people write down their listening over a longer period of time as well. People tend to be more conscious of the "propriety" of what they write than what they say, especially on the phone.

The diary is anonymous, so there is nobody to "impress" or "do the right thing" for. The personal interview is subject to all kinds of interviewer bias... in fact, the term was created to cover the influences of dress, age, gender, hair style, language usage, accent and inflection, etc., that all create interviewer bias.

The variance between the diary and the PPM all have to do with diary rounding (greater TSL than real), diary keeper failure to register breaks in listening in the diary (greater TSL than real) and the failure of diary keepers to notice or register listening to stations someone else selected or even to write in stations used regularly but only briefly.

In order of accuracy, the personal interview trails the diary, and the diary trails the PPM. The diary caused Pulse, Hooper and even MediaTrack/MediaTrend to go under... because time buyers believed in the diary as a superior methodology and because it gave true weekly cume based on listening, not an algorithm.