You're gonna need the hard drive for Live arcade games or demos, if nothing else. You may be able to get along without for awhile, but at some point the hard drive will be necessary.
Also, I read on the Oblivion fan Q&A that the game runs faster if you have the hard drive. Arcade, to me, is worth the hard drive alone. Also, considering you get a wireless controller and headset along with the hard drive for the price of the hard drive alone, there's no way I would have gotten a Core system. I actually had a chance to get a Core system many weeks before I got my Premium and I passed it up because I knew I would regret it if I just got the Core.