There will always be some in your audience who would be OK with you starting 24/7 Christmas music in August; however, I've always believed (and listeners have supported the idea that) Christmas music starts either noon Thanksgiving day (after Santa passes in the Macy's thanksgiving Day Parade, basically) or on "Black Friday," the day after Christmas.
On the other hand, the stations that go all-Christmas benefit from being the first to switch in the market. In the Tulsa market, there was one year where one of the local stations went all-Christmas ON Halloween to make sure they got the jump on everybody else. If you don't have another all-Christmas station in your market, you can wait until the "last moment" (whatever your listeners think that is) to go all-Christmas.
It's also worth mentioning that those stations that go all-Christmas tend to have poor ratings at the beginning of their programming, then dominate the market by the time we get to Christmas Day.
Since you said you WEREN'T going all Christmas, I'd stick to the "day after Thanksgiving" rule.
Have you thought about asking your listeners? It couldn't hurt...