Yes, you can't avoid federal taxes. You just don't have an additional ~10% on top of that from the state.

Besides federal income tax (as noted by another commenter), there's local sales tax, state property tax, and who knows what-all else. Income tax is sometimes one piece of a state's tax system, but it's never the only piece.