You're not insane. They do have a "gee whiz" factor that's pretty substantial. (If you didn't see the L.A. Times article this week, it's pretty interesting -
http://www.latimes.com/features/health/la-he-0308-gear-20100308,0,852938.story . $5,446 for a Co-Motion? Geeze.)
That said, you'd probably be better off doing a build up of a standard chain-driven bike with an internally geared hub / fixed gear / singlespeed, as they offer almost all of the advantages of a belt drive (quiet, low maintenance, cool aesthetics) with none of the drawbacks (expense, lack of options, lack of availability, etc.)
There are a few issues with belt drives, in addition to the one you pointed out about it requiring an opening in your rear frame, which, if not done correctly, has the potential to weaken the frame.
1. Finding parts. If you have a chain driven singlespeed or internal rear hub, and you need a new chain, you can go into any bike store, buy a chain, and get on your way. If you have a belt drive bike and it needs replacement for whatever reason, good luck finding one. (Seriously, try google.)
2. Expense. A standard singlespeed bike chain costs $8-15. A drive belt for a belt drive bike costs $55. And that's just the belt, the cog and chainwheel are also made by very few manufacturers, and not cheap, and then there's the modifications you need to make to your frame.
3. Adaptability. One of the reasons chain drives are so popular on bicycles is that you can customize the length to meet your bike. If you buy a regular chain driven singlespeed/fixed gear, and then decide that you want to raise or lower the gearing, you buy a new cog / freewheel, then add/remove a few links from your chain, and off you go. Most of the current belt drives that I've seen have a standard sized belt designed for the manufacturer's choice of gearing. This is fine if you've got an internally geared hub, but you're kind of SOL if you want to change the gearing on your belt driven singlespeed.
Either way, the Bikerowave will be happy to help you build up your next bike.
I've had my eye on these new-fangled belt drives for a while. Anyone had any good/bad experiences with them?
I was contemplating putting one on an old frame, but of course this requires modifying the frame somehow to install the belt, because you can't disconnect a belt like you can a chain. The frames I've seen that are made for belt drive have an opening in the rear triangle where the dropouts would be, and then a separate piece with the dropouts that gets bolted to the frame. Seems like something any machine shop could handle... right?
Am I insane to even think about doing this? Will I hopelessly weaken my frame?