If you're in the USA (edit: at a local RadioShack), you can buy an Arduino, a breadboard, wire, some resistors, switches, and LEDs for less than $60 -- the cost of a typical newly-released video game.
Take it home, read the online getting started guides, and start learning how to get basic circuits working.
Start reading http://hackaday.com/ to get an idea of what's possible. Bookmark projects that look interesting and proceed from there.
In no particular order, I suggest:
Check out makezine.com.
Find a local makerspace[1]
Work through tutorials and references for beginners (great hands on electronics book [2], more in-depth info to augment an electronics tutorial [3], arduino [4])
[1] https://duckduckgo.com/?q=makerspace
[2] http://www.amazon.com/Make-Electronics-Learning-Through-Disc...
[3] http://www.amazon.com/Practical-Electronics-Inventors-Third-...
[4] http://www.amazon.com/Arduino-Workshop-Hands--Introduction-P...
I'd recommend buying an Arduino and some basic parts and a breadboard to toy around with. The Arduino.cc Playground (Wiki) has some great resources for interesting projects. You could also buy a MAKE book and an accompanying kit, although the kits tend to be a bit overpriced.
If you're interested in a bit more thorough understanding of electronics, I would recommend checking out the Students Guide to the Art of Electronics. You can find a pdf floating around online and buy the book to support the author (who is an amazing teacher)! It runs through all the basics of analog and digital electronics while keeping it interesting and the math to a bare minimum.