And database/fixture set up is something that you will have to do, assuming that you test your app, and that your app is more sophisticated than "I have some strings which I want to upper case". Ditto for working with legacy databases, migrating data, bla bla blah.
The default option from what I've seen tends to be to set your DB up once and hope for the best, or else repopulate it after every test. Both of these options work great(ish) when you first start your project, but then grind you down six months later when your test suite takes an hour to run. A decent ORM, along with in-memory SQLite and fixture setup is generally what's settled on for most integration suites that I've seen, and Django does that out of the box.
Also - a minor nit, but WTForms and Jinja2 are based on Django's form and template libraries. Switching to them doesn't get you that much - they certainly don't replace any of Django's core infrastructure. And I've found SQLAlchemy to be basically unusable unless you use the new 'declarative base' stuff, which looks suspiciously similar to... Django's ORM :)