Head over to http://makezine.com/ for lots of projects to get started. Many are based on the Arduino, which you can pick up for under $30 USD:

http://www.makershed.com/SearchResults.asp?Cat=43

It's probably the easiest and lowest cost way to get your hands dirty.