PPM System is Terrible

[4CX1000A]

If you want to get rid of the disclaimer, do a daily census of radio listening. Measure every single person, every moment. Get back to me on the cost.

Of course, on an Internet stream it's easy and costs almost nothing.

It's a pity Internet stream listening statistics are not expressed in measures directly comparable with radio listening (e.g. AQH, cume, TSL). I would love to know how listening patterns for the average radio station compare to listening patterns for the same station's Internet feed. If stream listening can be known with perfect accuracy, does it have any predictive value for radio listening?

 

[WNTIRadio]

Why can't various methodologies be used to calculate listenership? Use PPM. Use diaries. Use call out research. Combine all three into a rolling average. It couldn't be that costly to throw 1k diaries in a city the size of NY. And to call another 500-1000 people with a scripted set of questions.

Funny that this discussion is never focused on Neilsen meters. I could turn on the TV and leave the room and it registers as ratings. At least with PPM you have to be in earshot of the station.

 

[davideduardo]

Why can't various methodologies be used to calculate listenership? Use PPM. Use diaries. Use call out research. Combine all three into a rolling average. It couldn't be that costly to throw 1k diaries in a city the size of NY. And to call another 500-1000 people with a scripted set of questions.

Let's look at a couple of the many issues.

First, combining a panel with a random probability daily (phone) or weekly (diary) sample is a complex issue. I really doubt that the MRC would sanction a mixed methodology.

Then there is the time issue. Advertisers pushed for the PPM because they wanted a measure of exposure and faster results. A diary sample takes a looooong time to get returns and to tabulate.

The New York diary survey had less than 1000 weekly diaries. And it was expensive. It would add about 60% more to the cost of the survey, which stations have no interest in paying... particularly since it would slow down delivery.

Phone surveys for ratings were abandoned nearly three decades ago by users in large markets. They don't give a true weekly cume, and they are horribly memory dependent and subject to interviewer bias more today than ever. But more than anything, the response rate is terrible today and cellulars and answering machines.

But back to the start... advertisers pushed for the PPM, and would not buy into going backwards in methodology.

Funny that this discussion is never focused on Neilsen meters. I could turn on the TV and leave the room and it registers as ratings. At least with PPM you have to be in earshot of the station.

But agencies accept Nielsen for cable and network TV, and the more recent enhancements of log-in of individual users gives a closer view of the demos.

The key issue in all of this is that ratings don't have to be 100% accurate. They just have to be accurate enough to be useful for establishing a metric for the sale of advertising. So there is a give and take between sample and cost, with the margin of error being acceptable as part of the trade-offs.

 

[w9wi]

But agencies accept Nielsen for cable and network TV, and the more recent enhancements of log-in of individual users gives a closer view of the demos.

Nielsen now uses something awfully similar to a PPM. (I wouldn't be entirely sure it isn't the same technology...)

_________________________________________________

I note they (Nielsen, for TV) go to some lengths to explain to the local station just where the encoder needs to go in the chain, and how important it is that it go there. That leads me to believe a non-negligible number of stations get it wrong -- which means that while maybe the PPM encoder isn't *supposed* to go before a DA feeding some outside destination other than the transmitter, I wouldn't be betting any money that it *doesn't*

 

[davideduardo]

Nielsen now uses something awfully similar to a PPM. (I wouldn't be entirely sure it isn't the same technology...)

TV meters are People Meters. Arbitron meters are Portable People Meters. TV meters are installed to each TV, the radio meters are like pagers and travel with the person, not with the radio.

The TV meter detects the tuning of the TV, while the PPM detects a code on the broadcast signal.

 

[N.E. Streetz]

PPM Crap is not going to pick up me listening to my Walkman ( Ipod or whatever what is on my head phone ).

 

[davideduardo]

PPM Crap is not going to pick up me listening to my Walkman ( Ipod or whatever what is on my head phone ).

Actually, it is. PPM panelists are given a pass-through device for precisely that situation.

We can discuss and argue about whether people will actually use the adapter, but there is definitely a procedure for measuring headphone listening.

 

[WNTIRadio]

Here's an idea... All smartphones have a microphone in them and can download apps. Panelists are selected the same way as usual, except they download an app given to them in a link from Arbitron. They carry their phone with them all the time anyway, so this wouldn't be an additional device. The app could also "phone home" at any point, avoiding the docking procedure. How hard could it be to have an app run software to decode the PPM signal? Most of them have motion sensors in them, so the problem of someone cheating the system by leaving it in front of a radio all day would be solved.

It's a lot easier and MUCH cheaper to write an app than it is to place hardware in the hands of everyone. Plus panelists could be rotated faster, and more of them could contribute this way.

The app can be programmed to only report data when they are selected to be on the panel, a "private" app. It could also log any streaming done via the phone too.

Can someone give me a downside to this? I can't think of one. Cheaper for Arbitron, larger sample due to decreased cost of hardware, ability to update itself and send data back multiple times a day for nearly real time results and only one device that 95% of the population has glued to themselves.

 

[davideduardo]

Here's an idea... All smartphones have a microphone in them and can download apps. Panelists are selected the same way as usual, except they download an app given to them in a link from Arbitron. They carry their phone with them all the time anyway, so this wouldn't be an additional device. The app could also "phone home" at any point, avoiding the docking procedure. How hard could it be to have an app run software to decode the PPM signal? Most of them have motion sensors in them, so the problem of someone cheating the system by leaving it in front of a radio all day would be solved.

It's a lot easier and MUCH cheaper to write an app than it is to place hardware in the hands of everyone. Plus panelists could be rotated faster, and more of them could contribute this way.


Can someone give me a downside to this?

There is a boatload of downside issues.

But first, there is no desire to reduce the time on the panel. In fact, every effort is made to keep panelists for the full 24 months.

One of the objectives of the PPM is to be able to track the same listeners over time. Stations, for example, have learned that there are few permanent P1's because listeners sort of rotate among their several preferred stations, giving more listening to one now, another in a month or so and so on. That means that P2's and even P3's may be tomorrow's P1's naturally.

Arbitron spends a lot of time and money getting panel households through the first month... once they are accustomed to the system, they are most reliable and most accurate. We would not want new panelists... we went through a new sample every week in the diary and did not like it.

As to the phone solution. The issue is that the microphones in cellphones are directional and have all kinds of noise cancellation to make phone calls clear and understandable. Those "features" make for a limited usefulness of a cellular phone; MediaAudits tried to use a cellular system and even tested it in Houston a few years back and it did not work for several reasons.

Arbitron is working, nonetheless, on a cellular phone technology. As with the initial development of the PPM (which goes back to the early 90's on the third floor in Columbia), most of the development data is proprietary. We'd assume they would have to provide modified phones for detection to work.

The first design obstacle, then, is overcoming the microphone directionality of the average cellular phone and also overriding the noise cancellation and audio filtering they have. Obviously, a phone could turn the call enhancement features off when the phone was not engaged in a phone call... and mike sensitivity could be amplified when not on calls. Whether this could be done in software in an app, or whether Arbitron could provide adapted phones is part of the issue. One of the ideas often mentioned is that PPM households would get free cellular service while on the panel.

Since the majority of cellular subscribers don't, today, have smartphones, and the fact that different smartphones use different operating systems complicates the "run an app" possibility. Also, it might be necessary to use only one carrier, requiring people to drop, with fees to be born by Arbitron, their present carrier.

We know this is all being worked on. It is, as you suggest, the most logical future of measurement for all media.

 

[WNTIRadio]

As to the phone solution. The issue is that the microphones in cellphones are directional and have all kinds of noise cancellation to make phone calls clear and understandable. Those "features" make for a limited usefulness of a cellular phone; MediaAudits tried to use a cellular system and even tested it in Houston a few years back and it did not work for several reasons.

Arbitron is working, nonetheless, on a cellular phone technology. As with the initial development of the PPM (which goes back to the early 90's on the third floor in Columbia), most of the development data is proprietary. We'd assume they would have to provide modified phones for detection to work.

The first design obstacle, then, is overcoming the microphone directionality of the average cellular phone and also overriding the noise cancellation and audio filtering they have. Obviously, a phone could turn the call enhancement features off when the phone was not engaged in a phone call... and mike sensitivity could be amplified when not on calls. Whether this could be done in software in an app, or whether Arbitron could provide adapted phones is part of the issue. One of the ideas often mentioned is that PPM households would get free cellular service while on the panel.

Since the majority of cellular subscribers don't, today, have smartphones, and the fact that different smartphones use different operating systems complicates the "run an app" possibility. Also, it might be necessary to use only one carrier, requiring people to drop, with fees to be born by Arbitron, their present carrier.

The noise cancellation is only active when you're on a call. Record a voice note on an iPhone, Blackberry or Android phone... the quality will surprise you.

I would beg to differ that the majority of cellular subscribers don't have smartphones! Have you looked around you?? Not many flip-phones. Most are iPhone, Android or Blackberry.

I have done remotes with an iPhone "calling" back to a Tieline BridgeIT codec. The internal mics are pretty damn good. Phones have come a long way since those tests with MediaAudits. And I don't see much of an obstacle as far as making an app that runs on Android and iOS (Blackberry is digging their own grave as I type). Apps are written for the different platforms every day. They don't even need to decode the "tones" in the PPM audio, just send the tags with time/date stamps back to home base. Actually, they could send time/date and PLACE tags as most phones have GPS now. That would be interesting... just based on the speed of movement you could surmise someone is in a car, walking or stationary.

The current PPM model with the "beepers" that people wear is about as hip as a beeper. My iPhone can make calls, surf the web, be a remote link, be a GPS, stream audio, take 1080p video with great sound, take 8MP pictures in low light... getting it to hear some embedded tones shouldn't be rocket science.

The whole phone is software driven, so turning up the gain for that app alone, and leaving it at unity for video recording and voice notes is easy. Just have the app amplify the incoming signal taken from a patch point (virtual of course) and only amplified by that app. It's all 1's and 0's after it goes in to the phone... manipulation of the audio is just some programming.

The main point is that no matter how they do it, a larger panel would be beneficial.

 

[davideduardo]

The noise cancellation is only active when you're on a call. Record a voice note on an iPhone, Blackberry or Android phone... the quality will surprise you.

The issue, as I mentioned, is a combination of the noise cancellation, directionality and sensitivity. Some of these issues, like directionality, are in part mechanical (the case) as is sensitivity.

For a cell phone system to be accredited by the MRC, every phone the system is on has to present an equal and level playing field... same sensitivity, same frequency response, same equalization, and the same directivity... otherwise, certain phones would be more likely to detect than others in identical conditions.

I would beg to differ that the majority of cellular subscribers don't have smartphones! Have you looked around you?? Not many flip-phones. Most are iPhone, Android or Blackberry.

As of the end of last year, in the US there were 91 million smartphones out of 234 million cellular phones in use; that's 40%.

Phones have come a long way since those tests with MediaAudits.

That was only about 4 or 5 years ago. Microphone and case technology have not changed much since then.

They don't even need to decode the "tones" in the PPM audio, just send the tags with time/date stamps back to home base.

The tag is a 4-second data burst, using the most maskable frequency at the time of broadcast. The burst contains a time stamp and the unique code for each station or stream. If it is captured, it is essentially decoded.

You can't have 13 tags a minute each streamed to Arbitron. Many phones don't multitask, meaning that most of the time the tag has to wait till some other task is finished...

Actually, they could send time/date and PLACE tags as most phones have GPS now. That would be interesting... just based on the speed of movement you could surmise someone is in a car, walking or stationary.

Already one of the significant recruitment problems is the fact that many people are very hesitant to be "tracked" and think the PPM can track them. If it were known that the phone device will send someone your every movement, recruiting would be much more difficult.

... getting it to hear some embedded tones shouldn't be rocket science.

The tag is masked, not "embedded." While digital filters are eminently doable, any peculiarities of the frequency response and EQ curve of the device will alter the filtering out of the tags.

The main point is that no matter how they do it, a larger panel would be beneficial.

Since stations are already paying, after adjustments over the years of PPM, about 70% more than they last paid for the diary survey, there is about zero chance that stations would pay any more for added sample. Every one of your suggestions is more expensive or makes recruitment harder.

At some point, cell phones will be a viable collection device, but the reason that we do not have them now is that they are not yet a viable method of data collection.

 

[WNTIRadio]

The tag is masked, not "embedded." While digital filters are eminently doable, any peculiarities of the frequency response and EQ curve of the device will alter the filtering out of the tags.

Okay, bad choice of words on my part. Psychoacoustically masked is really what I meant. But for the layman, they're embedded. I took the engineering "class" on PPM, I do know how it works.

And bull! I can hold my phone up to the speaker and Arbitron can verify the PPM tones. I've done it numerous times on many different installations. So no, frequency response has nothing to do with it. The PPM signal is also in the 1-3kHz range, so any cheap mic can pick it up. If it can pass through MP3 encoding on webstreams (does that with some clients) as well as being able to get picked up by my phone and verified at Arbitron home base, then the phone mic picking it up is no problem at all.

http://www.25-seven.com/257/publications/25-Seven_Tech_Paper_Arbitron_PPM.pdf

4 or 5 years is an ETERNITY in electronic development. 5 years ago, the first iPhone was just being released. Droid was still a thing only found in Star Wars. And yes, the mics have gotten a lot better. Shoot some video and listen to the audio with a phone from 5 years ago and compare it to now. Plus, if they were testing 5 years ago, most people's phones were at least a year old or more, so the technology was even farther behind. They also did not possess the processing power anywhere near today's units.

I wasn't suggesting real time streaming of the tags back to Arbitron, but a few times a day when the phone is inactive can work fine.

http://www.internetretailer.com/2012/04/17/smartphone-owners-are-not-birds-feather

This is plenty for a good cross section and enough, now 49% and growing, that use smartphones. Out of that 49%, there are enough slices in the demographic pie.

Sometimes I think that if you were present at the invention of the wheel, you would be telling them to make it square just to be contrary

 

[davideduardo]

Sometimes I think that if you were present at the invention of the wheel, you would be telling them to make it square just to be contrary

Short version of post made earlier that got lost in all the board technical problems:

Smartphones represent less than half of all cellulars. 15% of adults don't even have a cellular. 52% of those over 75 do not and 32% of those 65 to 74 don't.

So less than 40% of adults have cellular smart phones; less than half of cellphone owners have smart phones.

To try to get MRC accreditation, Arbitron has added address based recruiting, in-person recruiting, cell phone only households, language balancing, GeoZones and other new procedures to get as close to a perfect sample of the universe.

Sampling only among 40% of the universe would not yield a representative sample as the 60% without smart phones would be excluded. That means generally that lower income people, older people and some minorities would be undersampled / unavailable for the sample. And the remaining smart phone sample would not represent the universe. How would that fly?

Further, unless all smart phones have identical sensitivity, directionality and frequency response, some will pick up and detect what others will not, making for an un-level playing field. What if, by some coincidence of features, marketing, styling or whatever a particular smart phone appeals and is thus sold much more among a specific demo or ethnicity... and that phone is more directional and less sensitive than, let's say, the iPhone. Huge bias is introduced.

There are still huge issues to overcome.

 

[WNTIRadio]

Did you read my attachment? Right now 49 percent of all subs have smartphones. That's up from 41% late last year.

Sure, it's not 90% of the population. But more and more are being activated every day. Why not plan for the future?

When Edison invented the light bulb there wasn't a distribution infrastructure in place to get the electricity to them. But that didn't stop him from doing it and also starting to put the infrastructure in. Of course, Tesla trumped him with AC, but that's a whole other story.

Why not start working on the methodology now so it is ready when 75% of the population has smartphones? Why try and reinvent the wheel with a generation 2 of PPM decoders. In fact, it would be cheaper for Arbitron to send people smartphones than proprietary hardware manufacturing. Have you ever looked at the cost of having a product developed? Especially one with a limited customer base?

You're the type of employer or employee that I run from. One that tells me every reason why I can't do something instead of telling me to find a way to do it, or find someone who can make it happen. If I had the money and investors, I would start up my own company to challenge Arbitron. It's about time someone knocks them off their monopoly with a more cost effective, yet still effective, process to get audience metrics and demographics.

 

[reelyreal]

You're the type of employer or employee that I run from. One that tells me every reason why I can't do something instead of telling me to find a way to do it, or find someone who can make it happen. If I had the money and investors, I would start up my own company to challenge Arbitron. It's about time someone knocks them off their monopoly with a more cost effective, yet still effective, process to get audience metrics and demographics.

You mean like Eastlan is trying to do, Nielson tried to do a couple of years ago and failed, and like Birch and Pulse used to do?

For the record, I don't know David personally. I only know his participation in this board as well as his websites and his history and story in the industry. He's not wet behind the ears, he's not a minor player. David GETS IT.

It all comes back to money. It doesn't matter how great the idea is, or how viable the idea is: anything new will cost more, and broadcasters won't pay more. Many of them already won't pay what it costs.


I think the real conversation needs to be why, for many stations, the book doesn't matter. Arbitron doesn't matter. Saga as a company has done just fine, and they don't subscribe. Selling on rankers, at least for me, was a pain. I felt that I built stronger local direct relationships based on the reputation and effectiveness of my station. If the agencies and buyers want to subscribe, God bless 'em. I don't care.

 

[davideduardo]

I think the real conversation needs to be why, for many stations, the book doesn't matter. Arbitron doesn't matter. Saga as a company has done just fine, and they don't subscribe. Selling on rankers, at least for me, was a pain. I felt that I built stronger local direct relationships based on the reputation and effectiveness of my station. If the agencies and buyers want to subscribe, God bless 'em. I don't care.

Thanks for the kind remarks, and without creating a sappy mutual admiration society ( ) I have to agree with you.

I have managed situations where I could not afford to do local direct selling as the rates were too low, the costs too high and the results not rewarding... but the station was WIVK style #1 with more than double the key demo numbers of the next station... we could only get our rates from agencies. But it was like living on the edge of a cliff every day.

And I have managed stations in markets with no agency business where local direct is all there is.

I think the ideal is where there is a balance of sales made based on service and rated by the cash register and sales made by ratings and metrics.

You know if something is wrong if you stop getting results... direct accounts are among the best feedback systems that exist.

 

[davideduardo]

Why not start working on the methodology now so it is ready when 75% of the population has smartphones?

75% of the population is not an acceptable pool for a sample. There would be no hope of accreditation or even credibilithy; when the national CPO number was around 10% Arbitron was oblicgated to include a proportional sample of CPO households in the sample. 10%... or a loss of the ability to hope for accreditation. You speculate that we may have 75% smartphone penetration, which leaves a full quarter of the adult universe unrepresented. Impossible.

Arbitron has been working on cellphone technology for well over a decade, but can not yet implement it.

I saw the PPM being developed in 1995; the device was the size of a brick, just like early cellular phones. They kept at it and did the first real tests in Wilmington and Philly at the beginning of the 2000's. There were two years of Philly tests and then two more in Houston before the first market went currency.

So, it takes a lot of time to develop the technology, even today. And right now, there are too many obstacles for an immediate future prospect of cellphone PPMs.

Why try and reinvent the wheel with a generation 2 of PPM decoders. In fact, it would be cheaper for Arbitron to send people smartphones than proprietary hardware manufacturing. Have you ever looked at the cost of having a product developed? Especially one with a limited customer base?

Arbitron would gladly move to cellphones if they could, at the same cost. But if they sent everyone smartphones, lot's of people would not participate as I guarantee you they are not going to get manufacturers to modify designs just for a few thousand units? Remember, technically, all devices have to be able to perform identically or the playing field tilts.

You're the type of employer or employee that I run from. One that tells me every reason why I can't do something instead of telling me to find a way to do it, or find someone who can make it happen.

And I am telling you they are working on it, but they can't do it yet, due principally to the fact that 100% of the universe to be measured does not have a smartphone.

If I had the money and investors, I would start up my own company to challenge Arbitron. It's about time someone knocks them off their monopoly with a more cost effective, yet still effective, process to get audience metrics and demographics.

Nielsen tried. And gave up. So did Burke, Audits & Surveys, Birch, SRC, and others.

Pulse and Hooper tried to survive. They didn't.

Eastlan measures markets that are too small or where there are philosophical differences with Arbitron.

With radio revenues off 30% in non-inflation-adjusted dollars since 2007, AM declining and new media growing, nobody is going to start a radio ratings company.

 

[WNTIRadio]

How about a mic, provided by Arbitron, that plugs into your phone? Now all mics are equal in terms of sensitivity and directionality. I can see where, especially varying sensitivity would skew results. But I don't think frequency response is a problem here, as all of the encoding happens at regular POTS line frequencies. The cheapest carbon mic should be able to pass the masked tones.

Sure 100% of the universe doesn't have smartphones, but 100% of the universe doesn't have PPM meters either. Suppose they ask "do you own a smartphone, if so which model?" and it is one of the "approved" models. That now frees up a PPM meter to go to someone else who fits the criteria that doesn't own a smartphone. Now, the biggest problem is getting attention, which is the sample size. And the cost doesn't go way up, because once a software app is developed, there is little additional cost to distribute it. Getting 1,000 more PPM meters does have hardware and shipping costs.

I think we can all agree that the sample size is too small. Sometimes 5 or 6 people can determine whether you have a 3 share or a 2 share.

 

[davideduardo]

How about a mic, provided by Arbitron, that plugs into your phone?

Do you think anyone would really do that?

Now all mics are equal in terms of sensitivity and directionality.

Actually, mics come in all forms of directionality ranging from the very directional to near omnidirectional in a particular plane.

I can see where, especially varying sensitivity would skew results.

That's a horrible skew factor. One phone might detect a "hearable" station, and another might not. If certain phone types or models are more used by one demo or ethnicity or gender or socioeconomic level than another, then the phone itself introduces bias.

But I don't think frequency response is a problem here, as all of the encoding happens at regular POTS line frequencies. The cheapest carbon mic should be able to pass the masked tones.

All depends on whether the cell phone and the app can do the notch filtering the current PPM does.

Keep in mind that any new device has to be backwardly compatible with the roughly 125,000 installed PPMs in use today, otherwise the ratings will be severely compromised when a transition period is made if that occurs.

Sure 100% of the universe doesn't have smartphones, but 100% of the universe doesn't have PPM meters either.

But the PPM panel represents 100% of the measured universe (persons 6+) in almost exact proportion to that universe on every stratification variable (age, gender, ethnicity, geography, income, and bunch of other characteristics like income, language preference among Hispanics, etc.).

A cellphone sample that uses smartphones excludes nearly 60% of the universe, and is thus not a proportional sample.

Now, the biggest problem is getting attention, which is the sample size. And the cost doesn't go way up, because once a software app is developed, there is little additional cost to distribute it. Getting 1,000 more PPM meters does have hardware and shipping costs.

The cost of the sample is not much dependent on the cost of the meter... that is a tiny part. It is made up of recruitment costs, training costs and maintenance costs.

Also, remember that the PPM is household / dwelling based. Every member of the household has to participate, or the household is taken off the panel. So it is not a matter of "calling and asking" but of calling landlines, calling cellulars, making personal visits, sending letters, etc., to recruit households that conform to the specific needs of any vacancy in the panel.

I think we can all agree that the sample size is too small. Sometimes 5 or 6 people can determine whether you have a 3 share or a 2 share.

Not in the broad demos. I looked at a station that is around 10th 12+ in New York City; of around 4,300 average daily in-tab meters, that station averaged about 200 daily in-tab meters. 5 or 6 people would not make a significant change in the broad numbers.

There is not going to be any significant increase in panel size because of the immense cost of maintaining a panel. Remember, in the broadest terms, you have to nearly quadruple the sample to improve the results by a factor of one standard error. And, again, using a cellphone app vs. a meter is going to make a very, very, very insignificant change in costs.

 

[WNTIRadio]

5 or 6 people DO make a difference if your station doesn't cover the entire metro, that does happen. There are a lot of those in the NYC market. Had it happen. Panel changed and the ratings drastically changed, turned out that's how many people dropped off and messed up the numbers.

Actually, mics come in all forms of directionality ranging from the very directional to near omnidirectional in a particular plane.

Really? Didn't know that being an radio engineer and recording engineer. I'm talking cellphone mics. They are roughly equal, engineered to be most sensitive to the human voice (and that's where the PPM signals hide, 1-3kHz) and post processing of the signal can fish out the PPM tones back at home base.

I feed them down a POTS line to test with Arbitron all the time. Sometimes even my cellphone, so they survive the codec and transit through the network. This is holding the phone near the speaker. In fact, I've been on the phone with them to do a verification with a competitor on in the background and they can pick that up and say, "oh, you're listening to WXYZ?"

The thrust of my posts have been not that this is a 100% lock and perfect way to do it, but that someone needs to think outside the box and come up with better ways to skin this cat. The methodology CAN be improved. These are just ideas, and maybe one of them will grow into something. Who knew dots and dashes from a spark gap would grow into what we have now?