Yaesu is a pretty solid "cheap" brand. BaoFeng/BTech is a less solid, even cheaper brand (my advice would be to avoid them, despite the fact that that's where I started...)
The downside though, is that they can be very frustrating in some environments (esp with a high noise floor, or with a strong transmitter nearby - the frontend is _very_ easy to overload), and they're generally pretty bad about spurious emissions (though the newer tri-band model seems pretty good in that regard).
For 500, maybe you can get a used FT-857D. You'll need a power supply, too. And an antenna.
If you have a local club, talk to the folks there and tell them you're interested in getting started. Someone might have stuff to sell or give to you.
Second this. I'm VP of an active club and for someone with a real interest we can do a lot. Here is a way to find a club, in the US: http://www.arrl.org/find-a-club
That kinda depends on what you consider "interesting". IMO, there's plenty of "interesting" stuff to do on 2m / 70cm, plus anything you can do in receive only mode on other bands. But to me, "interesting" includes things like listening in on air-traffic control using an RTL-SDR, participating in the local ARES group, working the local repeaters in voice mode, and experimenting with decoding various digital modes.
This is not to say that working HF doesn't open up additional doors, because clearly it does. My point is just that there's a lot of fun to be had with just what you can do with a Technician license, a $40 Baefung radio, and a $25 RTL-SDR dongle.
That's basically it. Shit, I was a tech for twenty years, but never did much because I wasn't interested in club stuff or chatting on the local repeater. At some point I got an RTL-SDR, but dealing with noise and frustrating software issues killed my interest in it... Then around the time my license was up for renewal again, I studied for a week and got my extra ticket, and bought a rig that could do HF-70cm and suddenly I was able to try anything that interested me. If you've got a $500 budget, you can do the same... no reason to limit yourself.
If you want to get into satellites, you'll want to a specialized antenna for that, and then it helps to have a computer setup or a fine-tuning dial that can help deal with the Doppler shift and tracking, etc. Then you're easily getting up there in price, probably well past $500.
The other thing I've seen people do is set up extreme long-range setups. They'll get the General license that allows them to use frequencies that bounce around the atmosphere better and build their own antennae, get higher-power transmitters, etc. and their rigs are easily into the thousands of dollars.
$500 will get you a good HF rig, but as another comment mentioned, you'll need to factor in the cost of a DC power supply, some decent coaxial cable and something to use for an antenna. Antennas can be dirt-cheap - just wire cut to length and lofted however you can.
/r/hamradio is a solid online resource. There's a semi-official IRC channel too.
If you prefer to buy not build your antennas, you can still do that for less than $500 all in, you just might have to look at used antennas.