Postgres is plenty fast. If you think it can only do 3 users or 800ms, you're doing something wrong.
If you are using redis and you don't care about perf you are probably doing it wrong. If you are using postgres and you care about perf you are probably doing it wrong. "Probably" because context is key.
There are a number of tips about that in the discussion above, such as turning off logging of and partitioning a job table.