For a little perspective, the remaining 20ms still leaves enough time for oh, 55 random SSD IOs and time spare to zlib a megabyte of text. Or about 4.8 million instructions of a modern 2.4ghz CPU. Just what does this app do exactly?
Aah.. Django.
Seriously, 20ms isn't good even for Python. That's not something to be proud of, that's reason to rip up whatever overelaborate mess you've constructed to figure out just how you're blowing so much time.
As a reminder, a 200 byte HTTP request can be parsed in around 600 instructions or less (about 250 nanoseconds at 2.4ghz).