You don't necessarily even need a framework. You could use some libraries. You only need: routing, HTTP request processing, and HTTP response preparation.
In general, I strongly support the idea of tying together tiny libraries, rather than using a framework. It takes months to learn every line of code in an end-to-end framework, but if you're pulling in just the 2-3 libraries you need, you can understand every line of your code. That's really important for efficiency/bugfixing/maintenance.
You might want to check out Bobo: http://bobo.digicool.com/en/latest/index.html
Note: Flask is basically just a combination of werkzeug and jinja for templating.
Flask just gives you the routing part maybe you can even strip flask out of it and just use wsgi.
You can try http://python-eve.org/ too