Nevada for example also has no corporate income tax. Montana, South Carolina, and New Mexico don't specifically regulate money transmitters.
Delaware is great for share holder rights, but if you don't have a lot of growth or external investors yet, incorporating in Delaware may be overkill as you can always do so later.
Mostly, startups should be focused on product, not things like incorporating until you need to. And by then, spend the lousy 2k on an attorney to advise you and do it properly.