As others have said, Hack-a-Day is great for inspiration, and MAKE is great for beginners (although falls short once you gain more experience). MAKE has a bunch of great intro books to Arduino, etc. in both paper and eBook form I'd recommend checking out.

I'd recommend buying an Arduino and some basic parts and a breadboard to toy around with. The Arduino.cc Playground (Wiki) has some great resources for interesting projects. You could also buy a MAKE book and an accompanying kit, although the kits tend to be a bit overpriced.

If you're interested in a bit more thorough understanding of electronics, I would recommend checking out the Students Guide to the Art of Electronics. You can find a pdf floating around online and buy the book to support the author (who is an amazing teacher)! It runs through all the basics of analog and digital electronics while keeping it interesting and the math to a bare minimum.