(Thankfully, I am not a lawyer, but I've lived outside the USA for all but 30 months from age 18 to 31. This is not legal advice.)
For US citizens, you need to be aware of a lot of additional regulations -- US citizens are taxed globally, and if you make more than the $6k or so minimum, you must file a return every year. You have a certain exemption (approximately $95k now) on your income, so if you make less than $95k per year, you don't owe any tax, but must still file.
If you are in a location which has its own taxation, you may need to pay, and then use a tax treaty with the US to deduct foreign taxes paid, avoiding double taxation.
Basically, if you have substantial income or assets, you should use an international tax attorney to handle these matters for you. Only an attorney has attorney-client privilege, and your average US tax preparer (or even worse, H+R Block or something) has no clue whatsoever. If you make <$95k/yr and have pretty much just regular taxes, and are not subject to foreign tax (due to traveling on tourist visas and not really establishing yourself), you can probably file yourself -- you don't have a whole lot of deductions, etc., so it's actually fairly straightforward. If you have your own business, it becomes really complex, and I'd really go the tax attorney route.
There are lots of weird and non-intuitive special case rules built into the tax code, such as a restriction on "controlled foreign corporations", where >50% or so of the equity is owned by US persons, and are then subject to lots of extra US reporting requirements. This prevents a lot of the naive "keep all income in the corporation, pay yourself a trivial salary, expense all your personal expenses through the corporation" schemes, but also complicates legitimate foreign businesses owned by Americans. This is the kind of stuff you want the tax attorney for.
Right now I am personally stuck outside the US for the rest of 2010 due to taxes -- I worked in Iraq until August, making more than $95k, and thus it is a net savings for me of about $20k in taxes by remaining outside the US for the rest of the year, thus keeping my 330 days out of 365 out of the US, than it would be to return to the US. I have 6 days left in the US this year, which I might use for YC interview if I apply.
I plan to do my next startup in the US (maybe in tax-advantaged TX or WA vs. CA, but possibly just in the default of Palo Alto); taxes are annoying, but living outside the US is even more annoying.