Measuring Audience

[John FL]

Ratings. It’s a subject that has been talked about and written about extensively. Despite that, I believe a good number of us, including me, are not too clear on the process. In Florida as in states across the country, there are two types of assessments – PPM and diary. I see tremendous advantages for PPM markets.

It was interesting seeing the results in the holiday book in PPM markets. A good number of AC stations that presented the Christmas/holiday format performed extremely well. But, even beyond that, stations in PPM markets that are subscribers can see the fruits of their labor in a timelier manner.

Here on The Treasure Coast, the diary method is used. We’re market # 96 - Ft. Pierce-Stuart-Vero Beach, FL. There’s a spring and fall book and that’s it! I was interested in seeing how AC WOSN performed as the station flipped to Christmas programing a couple of weeks prior to Thanksgiving. The share looked bleak. But then, I have no idea what the reporting period is. Does the “fall” ratings period correspond to the actual days in autumn like from Sept 21- Dec 21 etc.? I’m not sure if WOSN’s Christmas/holiday programing is a part of the fall ratings.

If I were managing a radio station in a non-PPM market, I’d find it challenging to say the least. Tweak the playlist and how can you be sure what the audience reaction is? There's social media comments but that's not really scientific. At least with PPM, if ratings change for the better or worse, adjustments can be made quickly if necessary.

We hear a good deal about rolling averages that are in PPM that can help establish billing rates. But what about diary markets? Rolling averages for 3 ratings periods would go for a year and a half. Does that make sense? Maybe rolling averages aren't used in diary markets. It's a weird way to figure billing I would think.

For the most part, it’s the Top 50 markets that are PPM. There are some exceptions. I’m not sure why that is. There’s a couple of markets just out of Top 50 that are PPM. Perhaps, they were in the Top 50 at one time and so they remain as such. Will PPM be expanded to other middle and small markets?

Accuracy. A lot depends on ratings outcomes, including one’s livelihood. What I especially like about PPM is that it reflects actual exposure to a radio station, regardless of a person’s preferences. If we use the WOSN example, if a rocker or a country music fan visits a restaurant where “97.1 The Ocean” is playing in the background, can we rely on a person for figuring out the station calls and recording it? Seems a stretch to me.

The average person is consumed by technology. Perhaps it explains why attention spans are the way they are. I came across this site for the Nielsen diary ratings. The frequency asked questions tab was rather revealing.

http://www.nielsenradiodiary.com/mainhome.htm

This is such a cumbersome and outdated process. It’s laughable seeing a picture of a guy sitting in his car holding a radio diary so he can document his listening. One would think there would be an app and/or website where listening information can be captured in a more user friendly format.

I realize the big markets such as The Top 20 have the potential to earn lots of revenue. But smaller markets should be able to at least be treated with some degree of fairness. Every radio station is a cost center. If ratings were more accurate, it can benefit those stations that do a stellar job. If we take the AC example again. A person may not like the format but if his place of work or place of leisure is featuring that station, it has earned ratings credit but not it's not assured in diary.

I threw a lot out here but it’s a good start. I’d especially like to hear from those who manage in a diary market and what their views on this subject are.

 

[b-turner]

From what I know of radio, it seems more dollars are generated from direct sales versus ad agency placed sales the smaller your market becomes. The smaller the market the less emphasis on ratings and the greater the emphasis on direct advertiser results seen through heavy advertising concentrations for a limited time sale or live remote broadcast as a couple of examples. In fact, business owner perception has something to do with it. And don't discount the relationship between client and sales rep. It is true businesses buy from people to 'get what they do' and are liked by the decision maker.

In lots of markets where there are only a couple of books a year, it seems there is less attention paid to ratings and more emphasis on the sales end. For example the thinking for going all Christmas music might be tied to the thinking it could produce a large boost in the sale of 'audio Christmas cards' or season greetings packages. The fact it would be more of a natural for in store play is an associated benefit.

On the sales end, I got on buy sheets through being the station with the right format for the target demographic and creating a custom proposal. In fact, I frequently tried for all the market's radio dollars in an attempt to create a trend in buying for that agency when they placed a buy in my demo range. You really have to make the agency buyer look like they really strong-armed you to get them to take that chance. In other words, I worked to not make ratings the only deciding factor. And we did have a good sized audience to get them the desired results.

This is not to say programming is not analyzed and evaluated, it is, but the individual station's resources might be limited causing an educated guess more than actual result of a thorough testing. So to know your position in the market matters more to the point of gauging the success of the educated guess and to analyze how you might maximize those numbers. The ratings simply become less about getting the sale.

From a programming standpoint, lots of stations, although they might do some good research, might have a group of stations around the country they watch and emulate to a certain degree because they have the same target audience. The thinking is if it works for them, it will work for us if we modify it for our market.

This does not in any way say the #51 market is like #201. It's simply that ratings mean much less to #201 than they do to #51. That #51 market is much more reliant on ratings because the biggest buys are ratings oriented.

I recall assembling information for an advertising buy for a country music artist upon his CD release and 'homecoming' concert in his home town. The buy would be on one of the two country stations at the time in Lake Charles, Louisiana. The market leader in the prior ratings period pretty much said they were the #1 station in market and gave me a rate. In fact that station had been at the top of the heap for many years. The competing county station asked lots of questions and presented a detailed plan to maximize results. The promoter looked at the options and said there was only one, the underdog country station. It was a successful event. In this instance, ratings mattered less than a plan that made the most sense. It boiled down to the station that worked harder to earn the business.

In one small market (40,000 population), the rock edged top 40 always was #1 in the county annual ratings and easily triple the nearest competitor. Our station countered these ratings were 'old data' that was months old. We countered we reached the buyers and that the #1 station was #1 because of all the 12 to 24 year olds. The #1 station billed about half the dollars we did and we still got about 90% of any advertising agency directed business partly because we developed relationship with those store managers and asked them to recommend using our station. Our format was certainly adult based 25-49.

 

[PTBoardOp94]

Fall 2018 book covered September 13 to December 5, 2018.

 

[davideduardo]

We hear a good deal about rolling averages that are in PPM that can help establish billing rates. But what about diary markets? Rolling averages for 3 ratings periods would go for a year and a half. Does that make sense? Maybe rolling averages aren't used in diary markets. It's a weird way to figure billing I would think.

Keep in mind that Nielsen does not publish "rolling averages" of multiple books. Advertisers, primarily agencies, who use ratings use rolling averages of ratings to see stability and less book to book "wobble" in the ratings. Agencies have software that produces special analysis output of many kinds, including multi-book averages.

In diary markets, we have single book markets, two book markets and four book markets. Agencies will usually look at both books in a two book market, and at least two if not three in a continuous measurement markets.

But all that is moot now. Starting this year, Nielsen is going to do do sampling all year long, and will issue monthly rolling averages of multiple months. This way, advertisers will have the freshest data rolled into older data at a much more frequent interval.

For the most part, it’s the Top 50 markets that are PPM. There are some exceptions. I’m not sure why that is. There’s a couple of markets just out of Top 50 that are PPM. Perhaps, they were in the Top 50 at one time and so they remain as such. Will PPM be expanded to other middle and small markets?

There are a couple of Top 50 markets that did not want to pay for the PPM. One is at the cut-off, and the other, Puerto Rico neither wanted to pay nor had the population databases and communication infrastructure to support the PPM. I also believe there is a case of a market that moved into the top 50 and another than moved out due to population growth differences in the decade since the PPM was introduced.

The PPM will likely not move into too many smaller markets as the cost is very high compared to the diary, and those smaller markets don't have as much agency sales to justify the expense.

Only when Nielsen combines the PPM device to measure TV, radio and other media could it roll into smaller markets. Keep in mind when the PPM was being developed and tested in the very early 2000's it was a joint Nielsen and Arbitron effort.

Accuracy. A lot depends on ratings outcomes, including one’s livelihood. What I especially like about PPM is that it reflects actual exposure to a radio station, regardless of a person’s preferences. If we use the WOSN example, if a rocker or a country music fan visits a restaurant where “97.1 The Ocean” is playing in the background, can we rely on a person for figuring out the station calls and recording it? Seems a stretch to me.

Those tiny incidents of listening due to exposure are not of much value to radio stations. About half of a station's listeners account for 90% or more of its time spent listening. That other half increases cume, but actually reduces the AQH numbers because the weekly listening is so brief.

This is such a cumbersome and outdated process. It’s laughable seeing a picture of a guy sitting in his car holding a radio diary so he can document his listening. One would think there would be an app and/or website where listening information can be captured in a more user friendly format.

We have always (back to the 60's when Arbitron introduced the diary) that listeners don't carry the diary with them. They fill them in at one time of the day, often evenings, and employ memory to register listening during the day. That is why stations became so reliant on branding their name and dial position at every opportunity.

This is also why diary listening spans are often hour after hour of continuous listening. But in the PPM, we see the effects of coffee breaks, potty breaks, lunch breaks, absence from the place the radio is located, and so on. The Diary time spent listening is much longer than in the PPM because the PPM only measures cume and TSL, while the diary measures cume, TSL and memory.

I realize the big markets such as The Top 20 have the potential to earn lots of revenue. But smaller markets should be able to at least be treated with some degree of fairness. Every radio station is a cost center. If ratings were more accurate, it can benefit those stations that do a stellar job. If we take the AC example again. A person may not like the format but if his place of work or place of leisure is featuring that station, it has earned ratings credit but not it's not assured in diary.

Radio stations pay almost all the cost of ratings. If stations could benefit from what we used to call "phantom cume" they would pay for the PPM. They don't and that is because there is no return on the investment.

Another reason is that, while stations would pick up that "hidden" listening, on average the Persons Using Radio in PPM markets dropped 40% from the diary to the PPM when the transition occurred in the 2008 to 2010 period. While "share" does not reflect this, station rating numbers declined severely with the PPM.

 

[John FL]

From what I know of radio, it seems more dollars are generated from direct sales versus ad agency placed sales the smaller your market becomes. The smaller the market the less emphasis on ratings and the greater the emphasis on direct advertiser results seen through heavy advertising concentrations for a limited time sale or live remote broadcast as a couple of examples. In fact, business owner perception has something to do with it. And don't discount the relationship between client and sales rep. It is true businesses buy from people to 'get what they do' and are liked by the decision maker.

In lots of markets where there are only a couple of books a year, it seems there is less attention paid to ratings and more emphasis on the sales end.
From a programming standpoint, lots of stations, although they might do some good research, might have a group of stations around the country they watch and emulate to a certain degree because they have the same target audience. The thinking is if it works for them, it will work for us if we modify it for our market.

Appreciate your comments. After reading through your analysis, I can't say I disagree, at least about the amount of attention for ratings. There's a company I'm familiar with in N.E. Florida that does not describe to Nielsen. Yet, they manage to sell the cluster of stations quite well. They are in an outlying county and super-serve the local community. Two major markets near them don't come close.

In all my "involvement" with radio, ratings were everything but then I grew up in the biggest market in the country and spent my young and middle-age life in Miami/Ft. Lauderdale. In many ways, ratings have been drummed into me when one speaks of the business of radio. I've been reminded that there are other ways besides ratings to be a success in the market.

The only thing I will add to your comments is this. No matter what anyone does for a living, their performance is measured. Usually when a person gets a raise, it's accompanied with a performance review. For those who work hard, it's rewarding to read favorable comments about your performance. For on-air talent, there has to be a rush to know you're at the top of your game. My understanding is the diary method does allow for the mentioning of a talent's name as a way of giving ratings credit to the station. While I have no evidence to back up what I feel, I strongly suspect local, small-market talent just doesn't always get the kind of recognition they deserve. At the heart of it is a ratings system that is used more as a point of reference than anything else. In smaller markets, I've seen talent actively participate in community events. Between that and perhaps via social media, talent just may feel rewarded.

As someone who has managed teams of people for a long time, feeling valued is critical and why I brought it up here.

 

[John FL]

Keep in mind that Nielsen does not publish "rolling averages" of multiple books. Advertisers, primarily agencies, who use ratings use rolling averages of ratings to see stability and less book to book "wobble" in the ratings. Agencies have software that produces special analysis output of many kinds, including multi-book averages.

In diary markets, we have single book markets, two book markets and four book markets. Agencies will usually look at both books in a two book market, and at least two if not three in a continuous measurement markets.

But all that is moot now. Starting this year, Nielsen is going to do do sampling all year long, and will issue monthly rolling averages of multiple months. This way, advertisers will have the freshest data rolled into older data at a much more frequent interval.



There are a couple of Top 50 markets that did not want to pay for the PPM. One is at the cut-off, and the other, Puerto Rico neither wanted to pay nor had the population databases and communication infrastructure to support the PPM. I also believe there is a case of a market that moved into the top 50 and another than moved out due to population growth differences in the decade since the PPM was introduced.

The PPM will likely not move into too many smaller markets as the cost is very high compared to the diary, and those smaller markets don't have as much agency sales to justify the expense.

Only when Nielsen combines the PPM device to measure TV, radio and other media could it roll into smaller markets. Keep in mind when the PPM was being developed and tested in the very early 2000's it was a joint Nielsen and Arbitron effort.



Those tiny incidents of listening due to exposure are not of much value to radio stations. About half of a station's listeners account for 90% or more of its time spent listening. That other half increases cume, but actually reduces the AQH numbers because the weekly listening is so brief.



We have always (back to the 60's when Arbitron introduced the diary) that listeners don't carry the diary with them. They fill them in at one time of the day, often evenings, and employ memory to register listening during the day. That is why stations became so reliant on branding their name and dial position at every opportunity.

This is also why diary listening spans are often hour after hour of continuous listening. But in the PPM, we see the effects of coffee breaks, potty breaks, lunch breaks, absence from the place the radio is located, and so on. The Diary time spent listening is much longer than in the PPM because the PPM only measures cume and TSL, while the diary measures cume, TSL and memory.



Radio stations pay almost all the cost of ratings. If stations could benefit from what we used to call "phantom cume" they would pay for the PPM. They don't and that is because there is no return on the investment.

Another reason is that, while stations would pick up that "hidden" listening, on average the Persons Using Radio in PPM markets dropped 40% from the diary to the PPM when the transition occurred in the 2008 to 2010 period. While "share" does not reflect this, station rating numbers declined severely with the PPM.

I found your comments in answer to many of my questions regarding diary vs. PPM extremely interesting. I'm pleased to see Nielsen will do sampling all year. That is a step in the right direction. I knew subscribing to Nielsen was expensive. However, I failed to consider the expenses involved in setting up a market. That explains a good deal. There's a great deal of inadequacies with the diary method. You touched on a number of issues, some of which I too was aware.

I also think of radio's competition on other platforms and the methods used to measure performance there as well. I'm not sure what is used but something must exist. It may be another good discussion point. More to come.

 

[davideduardo]

The only thing I will add to your comments is this. No matter what anyone does for a living, their performance is measured. Usually when a person gets a raise, it's accompanied with a performance review. For those who work hard, it's rewarding to read favorable comments about your performance. For on-air talent, there has to be a rush to know you're at the top of your game. My understanding is the diary method does allow for the mentioning of a talent's name as a way of giving ratings credit to the station. While I have no evidence to back up what I feel, I strongly suspect local, small-market talent just doesn't always get the kind of recognition they deserve. At the heart of it is a ratings system that is used more as a point of reference than anything else. In smaller markets, I've seen talent actively participate in community events. Between that and perhaps via social media, talent just may feel rewarded.

In the diary, anything that can be claimed exclusively to one station alone can be used to credit listening.

In order of frequency of mention are:

Dial position
Station name or slogan
Call letters
Talent name

Diary data is manually entered. Nielsen has a crediting tool that allows the editors to look for station names, talent names, etc. and when there is a match, the credit is put under the appropriate call letters. When there is confusion, such as "106 FM" when there are two stations serving the market with "106" in the dial position, credit is given in proportion to each station's historical performance.

Stations file a Station Information Packet with Nielsen with the slogans and other info. In the last decade or so, Nielsen does not require as much information as they prefer to seek data on their own instead of accepting station input which may be biased or wrong.

Obviously, stations that use calls get more call letter mentions and stations that use names get more name mentions. WIOD vs. Y-100.

 

[John FL]

In the diary, anything that can be claimed exclusively to one station alone can be used to credit listening.

In order of frequency of mention are:

Dial position
Station name or slogan
Call letters
Talent name

Back in 2000 or 2001, someone on our floor at work received a ratings diary. This was in Jacksonville which was not yet on PPM. So, I got a chance to see what the actual diary looked like, including the letter instructions he got. What you described is what I recall so it would appear little or nothing has changed over the years. Maybe that's a bad thing, maybe not.

I would have probably behaved similarly but this guy was obsessed with the oldies station that existed at the time. He had not yet filled it out but as I recall, he needed to document listening from Thursday to Wednesday. I figured that's why most stations that had cash contests back then doubled up the winning etc. on Thursdays. I cautioned him that indicating you listen to just one station morning, noon and night could raise a red flag. From what I remember, he indicated listening here and there throughout the day. I think he filled a diary for his wife too who couldn't be bothered.

It's probably more reasons why I have long questioned the integrity of the diary system. I know there are safeguards in PPM. Leaving a meter next to a radio tuned to a specific station as you go about your business is just one anomaly that is detected and results discarded.I would think and hope that the individuals at Nielsen who look through the diaries manually actually do their job. In my own business, I've seen employees throw in monkey wrenches once they think of quitting etc. Anyway, it is what it is.

On a related subject involving ratings whether PPM or diary is the effects on talent. Today, there is very little talent interaction after morning drive. I'm not questioning the merits of that or not but rather questioning the role ratings plays when talent is dismissed or even promoted/given additional responsibilities.

I've seen a number of people on extremely popular radio stations, mainly here in Florida, leave the station involuntarily. When I look at their replacement, it's almost always someone younger and a little better looking. That's just my opinion but I'm starting to think "image" weighs very heavily into talent decisions. Perhaps, it's been like this for a while now. I'd like to hear what others think.

 

[John FL]

Regarding talent and ratings, I neglected to add a few things from the last post. Radio is not different than most business. New senior management comes on board and often times, they begin to bring in those they worked with in the past. The thought process is about surrounding yourself with those who brought you success in the first place. In my own career, I saw that happen way too often. In radio, ratings may say leave well enough alone but yet changes are made. Sometimes changes are good ones and sometimes they are not. Too many changes can make listeners uneasy. You'd think lessons would be learned. Similar to other business, those who do the damage move on or are reassigned transferring the problem elsewhere. I sometimes wonder where ratings fit in.

But, on the subject of ratings, virtually everything is rated. Public opinion polls are a constant. The wording of questions can determine outcomes.There's radio and TV ratings, of course, and the outcomes can prove interesting over time. If you ever watch nostalgia TV, you may think all the shows were big hits. But they were not. Some lasted season after season and yet they never cracked the top 30. I've talked about this a good deal in past posts. There's many reasons this occurred but lots was centered around the decision makers. Some people in key roles were visionaries. Radio could use more of them too, in my opinion.

Then there are budgets. A daypart goes jockless or voicetracked if management sees the economic merits in doing so. Contracts come up and there's an opportunity save some $$. The incumbent may have gotten great numbers but when your time is up, it's up. The point is ratings are interpreted and decisions are made. Obviously, there's lots of variables that affect what ultimately happens. Sometimes, we're shocked. These decisions may even seem illogical. They can generate lots of buzz on the radio boards but then we never really are in the know. Speculation can be interesting as a subject.

I wanted to go into a lot of different directions on the subject of ratings including rating the music. There's formal music testing where P1 listeners "rate" the songs in a controlled environment. Then there's the more informal rating of songs on a station's website. I'm not sure how that fits into music decisions or even rotations. But it's still under a ratings umbrella. Picking the right songs is a critical part in achieving ratings success. But there's also a lot more than ratings alone.

Even here on Radio Discussions, there are subjects, certain cities, and some posters who enjoy great popularity. Then there's subjects that go nowhere and posters who have simply lost their mojo. This does relate to ratings if you think about. How frustrating it must be to see your numbers drop and you just can't do anything to get back on track. There's a people side to ratings, especially when people lose their livelihood. There's not much discussion about that anymore unless it's a really huge personality.

Anyway, some random thoughts about ratings for your consideration.

 

[John FL]

As I look at ratings, I often think about the sample size and if it's enough to truly represent whatever the community is being surveyed. We hear a great deal about 25-54 and 18-34 when ratings are discussed in detail. Obviously, there are numerous demos that will be target for some radio stations in the market and not others. There's woman 25-54, men 25-54, or woman 35-49 just to name a few. The pie is certainly divided into tiny pieces if we look at 25-54 alone.

Then that brings me to 55+. My first thought has long been why does Nielsen or formerly Arbitron. Eastlan, or whoever, even counts these people. Whether we are talking about PPM or diary, how do people get selected? I've long assumed that anyone from 6 or is it 12 to geriatric age is somehow scientifically gathered to represent a market. Eliminating those 55+ may sound like a great idea but that wouldn't be an accurate measure. Perhaps I'm answering my own question but I'm curious about the drivers behind sample sizes and the demos percentages that make up that sample.

We can look at markets with large minority populations and do those who are chosen to be part of the sample adequately represent that market as well? As this is the Florida board, let's look at Miami/Ft. Lauderdale. According to the latest ratings, that market is described as 53% Hispanic and 21% Black. Does Nielsen determine ethnicity of potential participants when putting a ratings panel together? Or, does the fact that the market is naturally high in diversity would increase the odds that Hispanics and Blacks would be represented well? One of my driving forces behind this string was to discuss the accuracy of the data.

As I mentioned earlier, a great deal rides on the accuracy of ratings. We can look at polling data and that's very similar to ratings I would think. There seems to be many flaws in that system as well. Hardly any polling group got the last Presidential election correct. In the last Florida election, the polling didn't exactly call every race correctly. I wonder why that was and I still wonder about the accuracy of radio ratings and you can throw TV in as well.

I look forward to being enlightened. When I started this string, I said I had a lot of questions. Don't misunderstand. I'm not trying to discredit the system. I'd like to just get better understanding of it, especially with regard to sample sizes and the methods used to pick the participants. The courts randomly pick people for jury duty. I'm thinking that's not the case when being chosen for a ratings panel.

 

[PTBoardOp94]

Then that brings me to 55+. My first thought has long been why does Nielsen or formerly Arbitron. Eastlan, or whoever, even counts these people. Whether we are talking about PPM or diary, how do people get selected? I've long assumed that anyone from 6 or is it 12 to geriatric age is somehow scientifically gathered to represent a market. Eliminating those 55+ may sound like a great idea but that wouldn't be an accurate measure. Perhaps I'm answering my own question but I'm curious about the drivers behind sample sizes and the demos percentages that make up that sample.

The driver for sample size is cost. More members of the ratings panel, the more it costs to recruit them and process the data. However, certainly some stations would be upset if their 12+ numbers shrank dramatically because the older demographics were no longer included.

We can look at markets with large minority populations and do those who are chosen to be part of the sample adequately represent that market as well? As this is the Florida board, let's look at Miami/Ft. Lauderdale. According to the latest ratings, that market is described as 53% Hispanic and 21% Black. Does Nielsen determine ethnicity of potential participants when putting a ratings panel together? Or, does the fact that the market is naturally high in diversity would increase the odds that Hispanics and Blacks would be represented well? One of my driving forces behind this string was to discuss the accuracy of the data.

To some extent, Nielsen tries to select a representative panel, for example by mailing to ZIP codes with certain ethnic makeup. If they don't get it "right", they correct it by weighing. So if they had a 47% hispanic Miami panel, those results would get weighed a little heavier to reflect the true makeup of the market.

As I mentioned earlier, a great deal rides on the accuracy of ratings. We can look at polling data and that's very similar to ratings I would think. There seems to be many flaws in that system as well. Hardly any polling group got the last Presidential election correct. In the last Florida election, the polling didn't exactly call every race correctly. I wonder why that was and I still wonder about the accuracy of radio ratings and you can throw TV in as well.

This is not exactly an accurate statement. Polls got the election pretty darn close when you look at them in more detail than what the media typically reports. For example, the ABC News/Washington Post poll found Americans preferred Hillary Clinton by 3 points -- she won the popular vote by 2.1 points. The average national poll conducted within one week of the election found Clinton would win by 3.6 points, just to point out I was not cherry picking that ABC/WaPo poll.

In the states, many states were close enough that the "don't know/no preference" option, which almost never gets mentioned in the media, was a significant factor.
For example, in the Michigan polling average, only 94.5% of Michiganders expressed a preference with the other 5% saying they did not know who they would vote for. Almost all of them voted Trump.
Similarly in Wisconsin. In polling, Clinton led by 6.5 points, but 6.1% were undecided, and almost all of them voted for Trump, along with several who told pollsters they would vote 3rd party coming home to the Ds/Rs.
Other states, such as your state, were so close that it really doesn't count as a polling miss. Florida polls in 2016 showed an essential tie, Trump won by 1%.

Now, to bring this all back to Nielsen. There definitely is a possibility that Nielsen data contains errors. Nielsen does make an effort to correct them when they occur, whether that is one panelist who showed unusual dedication to a particular station, or a panel which is not properly constructed. In some ways, producing ratings is different than political polling, because there is no "truth" (election) to compare it to. The ratings are the ratings. In most cases/markets there is only Nielsen.

 

[John FL]

PTBoardOp94, I very much appreciate the information you provided in answer to my many questions. In particular, I’m pleased to see Nielsen strives to reflect the community served in its data gathering.

When I thought about posting this topic, you might say I was cynical about the system. But then, I’m suspicious of a good number of business practices. My career was mainly in the financial sector and I witnessed enough shenanigans to sour my disposition about Corporate America. Friends in other businesses told me their horror stories as well.

I’d like to also thank you for your detailed analysis regarding polling. When I post on this board, you can be sure I do my homework, especially when I’m trying to make the case about something. Where I’m usually specific, here I was rather general and I didn't back-up my opinions.

I had recalled commentary from various pundits, networks and certain polls. Frankly, there’s too many polls and many are just not worth talking about. Social media will look at poll numbers in one poll that favors a particular candidate or party and next thing you know, that information is trending and going viral. It happened countless times where we can be manipulated in thinking certain outcomes will happen.

No matter what anyone’s political beliefs may be, there’s a strategy that involves demoralizing and diminishing enthusiasm for a political opponent and/or party. Voters won’t bother if they feel their vote won’t matter no matter how many times they are told it does.

I don’t want to get off on a political discussion but I do believe there was a tie-in to ratings. Most people want a system that is fair whether we are talking media ratings, elections, polling, or even performance appraisals in our job.

Radio Discussions is at its best when people are engaged. Sometimes, we (posters) need to be grounded too. I've changed my views many times over the years based on comments from the pros. I try to keep an open mind. Thanks also for not being condescending in pointing out differences of opinion. We could use more of that.

 

[davideduardo]

The driver for sample size is cost. More members of the ratings panel, the more it costs to recruit them and process the data. However, certainly some stations would be upset if their 12+ numbers shrank dramatically because the older demographics were no longer included.

With the PPM, Arbitron added pre-teens (6-12) because in its genesis the PPM device was intended to measure TV, radio and other media. As Nielsen develops the PPM system, it will again be used to measure "everything". So a panel with all ages is needed.

To some extent, Nielsen tries to select a representative panel, for example by mailing to ZIP codes with certain ethnic makeup. If they don't get it "right", they correct it by weighing. So if they had a 47% hispanic Miami panel, those results would get weighed a little heavier to reflect the true makeup of the market.

99% of Nielsen's recruiting goals in the PPM are to achieve proportionality on all the stratification variables, not "to some extent".

That means proportionality on geography, ethnicity, age group, gender, income level, education and the like. The recruit quotas are specific and totally proportional on every one.

Mail is used as to assist to recruiting in the form of "we'll be calling you and this is why" type door openers. Recruiting is done on landlines, cell phones and in some cases in person. This is where it is made clear that all members of the household must participate, and when media affiliation bans are advised.

This is not exactly an accurate statement. Polls got the election pretty darn close when you look at them in more detail than what the media typically reports. For example, the ABC News/Washington Post poll found Americans preferred Hillary Clinton by 3 points -- she won the popular vote by 2.1 points. The average national poll conducted within one week of the election found Clinton would win by 3.6 points, just to point out I was not cherry picking that ABC/WaPo poll.

Ratings are paid for by radio to be used as a sales tool. A margin of error of 5% is not significant, as in ratings there are many "winners". In an election there is just one, so a few percent error can be fatal.

Now, to bring this all back to Nielsen. There definitely is a possibility that Nielsen data contains errors. Nielsen does make an effort to correct them when they occur, whether that is one panelist who showed unusual dedication to a particular station, or a panel which is not properly constructed. In some ways, producing ratings is different than political polling, because there is no "truth" (election) to compare it to. The ratings are the ratings. In most cases/markets there is only Nielsen.

All sample-based data has a calculable margin of error. Users know this, and they know that to reduce the error significantly requires about quadruple the sample. No station or group will pay for that.

There is a standard process based on an algorithm that identifies unusually high usage of one station, particularly if multiple members of the dwelling unit have identical behavior. This usually triggers a call to the household to ask about the behavior. Nielsen calls frequently, with the most frequent reason being members of the household not wearing / carrying the meter (it has a motion sensor).

Other deficiencies can be lack of 100% proportionality, but that is handled in two ways: first, by mathematically over or under-weighting the disproportional cell. Second is making sure recruiters bring the panel back into full proportionality.

The advantage of a panel is that it can be crafted to be "nearly" proportional all the time. A random weekly sample such as the diary or the Eastlan phone methodology can vary wildly week to week. In the case of the nearly 200 diary markets, Nielsen adjusts quotas every week as each survey period progresses to make up for deviations from proportionality. With the change to continuous rolling measurements slated for July, the sample corrections will be ongoing to approach proportionality.

 

[davideduardo]

We can look at markets with large minority populations and do those who are chosen to be part of the sample adequately represent that market as well? As this is the Florida board, let's look at Miami/Ft. Lauderdale. According to the latest ratings, that market is described as 53% Hispanic and 21% Black. Does Nielsen determine ethnicity of potential participants when putting a ratings panel together? Or, does the fact that the market is naturally high in diversity would increase the odds that Hispanics and Blacks would be represented well? One of my driving forces behind this string was to discuss the accuracy of the data..

When a household or dwelling unit is contacted to participate in the PPM, details on the family/group are collected and they include ethnicity, geography (at the county level and then down to the GeoZones within counties and the HDHA and HDBA areas for Blacks and Hispanics), age, gender, income, family size, education, etc.

If Nielsen does not need a contacted household, as they don't need such a unit at the moment in the sample, they may ask if they can call back at a later date. Otherwise, if the panel "needs" them, they proceed to implementation (delivery of meters, training, etc.).

PPM recruitment uses cellular phones, landlines and even in-person recruitment across a metro in proportion to population distribution. Recruitment is a huge expense for any research project. As an example, for a radio station music test, the recruiter is frequently paid more than the participants for getting people to participate in the test.

 

[b-turner]

As a former Nielsen household I can tell you it is very detailed on the user end. For questions it is akin to the Census with plenty of seemingly unrelated data such as the car you drive. In fact, if anything changes you must tell them (get a roommate, buy a new car or buy a new computer or your job gives you one that stays with you, it must be reported). They hook up equipment. They register every TV, VCR, computer and such and tag it requiring you to report any new purchase and if one of the items is removed. They wanted to know type of computer operating system, how you use it, etc. They checked with you every couple of months by mail, by phone or in person. In fact, I was on the phone on a very lengthy phone call and had the TV muted about 2 hours but still on. I got a call asking why with details. They gave you a remote you had to punch about every 40 minutes when watching TV so they'd know the TV was just not being left on but not viewed. If somebody came over you had to punch it in on that remote. I think I was doing it for 2 years. I was told it was for TV viewing. The equipment reports what you watch and maybe some other information that defines your viewing habits. They downloaded a program on my home computer that required me to click okay every time I got on the computer. I understand they used my internet to report data. I was told not to tell anyone I was a 'reporting household' during the time my data was gathered.

With all the contact I had along the way including equipment, it had to cost them hundreds of dollars. I haven't even considered the person that went door to door to qualify and sign up people. The rep said they had a goal of a small number of people signed up monthly per representative because not everyone qualified and there were so many follow-ups attempted for homes where nobody answered the door or responded to literature left. I got the impression if they came to your house they kept after you until you said no. Given a salary and gas for their employee I'm sure it is a couple of hundred buck spent here as well. I'd hate to see the cost on a large market.

 

[PTBoardOp94]

As a former Nielsen household I can tell you it is very detailed on the user end. For questions it is akin to the Census with plenty of seemingly unrelated data such as the car you drive.

I live in a diary market. I did the Nielsen television survey on a paper diary (at a time when I was not working in radio, so it was germane to participate). The diary treatment was nothing like you or David described. I got a letter with a questionnaire with some basic questions (ages, name, ethnicity of all persons in the household, contact info, etc) which I filled out and returned. They sent a diary a week or so later. They called a couple of times to provide instructions and encourage participation. At the end of the week, they called to remind me to mail the diary back. Easy, and a fair bit cheaper than sending around recruiting agents!

 

[davideduardo]

I live in a diary market. I did the Nielsen television survey on a paper diary (at a time when I was not working in radio, so it was germane to participate). The diary treatment was nothing like you or David described. I got a letter with a questionnaire with some basic questions (ages, name, ethnicity of all persons in the household, contact info, etc) which I filled out and returned. They sent a diary a week or so later. They called a couple of times to provide instructions and encourage participation. At the end of the week, they called to remind me to mail the diary back. Easy, and a fair bit cheaper than sending around recruiting agents!

First, I was describing how a panel is recruited and constructed. The Diary method is a poll using a random probability sample. Two very different methodologies.

A panel is "permanent" with households lasting up to 24 months, while the diary is a single week.

TV diary sampling is much simpler than radio. They seem to use database recruitment using pre-screened address lists (allthough I have never spent much time looking at the difference between the radio diary and the TV one).

I was "recruited" for the TV survey in the Palm Springs market the same way as you. I got the diaries, but immediately called Nielsen's contact number to indicate a media household and I called our company Nielsen rep to report receiving diaries.

So, obviously, there was only limited pre-screening as they managed to send diaries to the home of one of the VPs of a larger radio and TV company!

In person recruiting for a one week survey is not practical. But for a panel, in markets with huge CPO household counts and unlisted numbers, it is needed since a panelist is a potential 2-year investment.

 

[John FL]

Many readers have probably come to realize just how little they understood about this thing called ratings. There's been great commentary and clarification so I thank the previous posters for their detailed analysis.

As far as the PPM goes, I was not aware that a panelist makes a two year commitment. I also should have realized demographic data was gathered on a potential panelist PRIOR to them being considered. That makes a lot of sense. While I believe no system is perfect, we can see that Nielsen does endeavor to get things right including uncovering "cheaters." We've all seen stories about those instances and the repercussions.

As I read David's accounts about PPM, I found myself thinking about competition and moving that needle. I've experienced a good number of radio stations and formats that showed a great deal of promise. But, despite music tweaks, clever promotions, good imaging etc. eventually wind up flipping to something else. I talked about all the right ingredients that can help deliver success in another post. I'm a firm believer in that. But in everything that has been discussed thus far, I've the most uncomfortable with the two year length of time for the PPM panelist.

It has to be a tough job to go against the market leader with a new competing format. Does a 2 year PPM commitment make that tough job even harder? In two years a good deal can happen. A panelist moves out of the market, there may be illness, death, or voluntarily quitting. These are probably exceptions so for the most part, there's a panel in place for two years. A number of months can go by in a new station competition. The PPM panelists are like everybody else. We all have favorite stations and personalities. Getting a relatively small group to start noticing you and tuning you in cannot be easy.

I've long believed that quality broadcasts and all the right ingredients that spell success are requirements. Even with all that, getting a panelist to add your station to their car's presets or to change their listening behavior are not easy tasks. I've often criticized those who flip formats too quickly not giving the current format an opportunity to grow in time. Is it possible that some management just opts for a different course because those panelists are just not budging being set in their listening ways?

While I understand the costs involved in setting up PPM panelists, I have to wonder if ratings outcomes would differ if panelists served a shorter duration. In looking at ratings of various markets, every once in a blue moon, I notice changes from the norm and then those numbers become the new norm. I now wonder if new panelists do bring about ratings changes. With ratings, there's lots to consider.

 

[davideduardo]

It has to be a tough job to go against the market leader with a new competing format. Does a 2 year PPM commitment make that tough job even harder? In two years a good deal can happen. A panelist moves out of the market, there may be illness, death, or voluntarily quitting. These are probably exceptions so for the most part, there's a panel in place for two years. A number of months can go by in a new station competition. The PPM panelists are like everybody else. We all have favorite stations and personalities. Getting a relatively small group to start noticing you and tuning you in cannot be easy.

The average panel household (everyone has to participate) is on the panel for about a year. The turnover is about 8% monthly; some people tire of it and drop off and Nielsen eliminates some due to non-compliance issues.

While I understand the costs involved in setting up PPM panelists, I have to wonder if ratings outcomes would differ if panelists served a shorter duration. In looking at ratings of various markets, every once in a blue moon, I notice changes from the norm and then those numbers become the new norm. I now wonder if new panelists do bring about ratings changes. With ratings, there's lots to consider.

The system was designed to satisfy ad agencies, and they wanted to have longer term behavior. So they got it.

 

[John FL]

The average panel household (everyone has to participate) is on the panel for about a year. The turnover is about 8% monthly; some people tire of it and drop off and Nielsen eliminates some due to non-compliance issues.

The system was designed to satisfy ad agencies, and they wanted to have longer term behavior. So they got it.

I had a feeling that would be the case.

I've also wondered about measuring audience for non-commercial radio competition such as SiriusXM. I've been a subscriber for a good number of years. There's formats that have come and gone. There's also specialty type, limited duration formats that come about because of listener demand. One that comes to mind is the Billy Joel channel. So, my question is, does SiriusXM engage in some kind of measurement that determines the popularity of their many channels?

Some channels run with jocks. Others are automated. Sometimes there are "guest DJs" who will appear. On Valentine's Day Barry Manilow DJ'd SiriusXMLove (channel). Between the incredible love songs that were presented and Mr. Manilow's perspective on various songs and artists, I have to wonder why "the Love Channel" is jockless most times. It would seem a friendly voice would lend itself well with the music. But then, there must be a determination made that figures what formats warrant talent and which ones don't. Not sure how that happens.

Like their conventional radio's counterpart, I've seen the highest rated radio stations, for the most part, have a talent roster. The lower power stations/low rated stations may be lucky to have one staffer on-air. As always, there's always exceptions. But still, with a subscription service, there must be some kind of pecking order where the most popular channels are identified. These would be the channels that are behind most of the subscriptions. But then, I'm not sure about any of this.

Lastly, it brings me back to conventional ratings such as diary and the PPM. For the PPM, is a subscription service such as SiriusXM encoded so that they get ratings credit? Likewise, if a diary listener mentions listening to another platform, does that other service get credit? It would seem a SiriusXM would have to ensure the vast majority of their channels are attracting new subscriptions while keeping attrition as low as possible. But how do they do it?