I'd recommend that you not start with the rpi if you have zero experience.

If you're in the USA (edit: at a local RadioShack), you can buy an Arduino, a breadboard, wire, some resistors, switches, and LEDs for less than $60 -- the cost of a typical newly-released video game.

Take it home, read the online getting started guides, and start learning how to get basic circuits working.

Start reading http://hackaday.com/ to get an idea of what's possible. Bookmark projects that look interesting and proceed from there.