Just adding that the RasPi power supply is a standard micro-USB so if you happen to have one already (old phone or tablet charger?) and a spare SD card (old camera?) then you don't even need to pick those up.
Heck even the case isn't strictly necessary. I've got a similar setup at home and for quite a while mine was hanging off the back of my TV on a short HDMI cable.