Ironically, Bruce St James did a show pretty much about this exact same topic - regarding why radio was the way it was, and I don't know if he was speaking the truth or making things up (as people would believe because he's a conservative leaning talk show host), but (in my opinion) it made precise sense.
However, since I don't remember specifically what he said, verbatim, I'll just answer / say everything in my own words. Radio plays less music because people want familiarity, and it's easier to become familiarized with a smaller playlist of hits rather than a large playlist of music or playlist with mixshow time slots within them. Smaller playlists with more consistency also insures that it will be more likely that a familiar tune will be playing at whatever time the listener tunes in. It's easier to get a bunch of people to agree on 100 songs than it is to get people to agree on 1,000 songs. Most people listen to radio for short periods of time, and when they tune in, they want to hear something they love, or at least like and know, playing immediately (or else they'll seek another station that is doing so). It's similar to when you get into a car or some place and play a cd or mp3 player. That cd or mp3 player has a limited amount of tracks on it, but if you had the choice, you'd rather play that cd or mp3 player of specific familiar songs over and over anytime than play a cd of new songs you didn't know - especially if you're listening in route to somewhere. Why? Because you want what you like and know on demand and don't want to waste time "learning or discovering" tunes you might like. Radio is for the masses. Majority rules, and how people act/react & respond to certain things is also more important than what they say they want - when it comes to radio. It's all about what works in regard to sales & business first, and entertainment along with everything else is secondary.
As far as commercials; Some may wonder why anyone would play commercials when we know people hate that and may change the station, but wont use any time for some extra variety music. Well, commercials are played because basically media has no choice. It's "mandatory" for radio business survival. I'm sure radio stations would and would love to get rid of commercials if they could.
MIXSHOWS - IN MY OPINION
When it comes to mix shows and other certain sounds, I share the same enthusiasm. However, if many people in the business are just in it to get by or make a living rather than out of having a passion for the job, what do they care about a mix show when it's easier to just stick to basics 24/7? A mixshow takes more effort and having to set up & monitor more things...etc. If you're a programmer, you're most likely not going to go out of your way to do something optional that you don't necessarily care for, and it's only worse if your team is full of like minded people who couldn't care less about specialty programming. My thinking (and this was recently) was "wouldn't it be easy to find a dj that's willing to follow the rules for little to no pay at all?" However, now that I think of it, I could see the potential problem with doing such a thing.
When it comes to higher variety and more specialty programming and deeper playlists, "no one will do it until everyone else does it, and does it successfully" -KDM7000 If it was more about entertainment and being edgy and had nothing to do with business and finance, then I could imagine things would be drastically different.