Tough to say without knowing more about the stack. But in terms of first byte latency, some things you can look at:

* TLS False Start

* HTTP/2 + ALPN

* Optimizing TLS record size

Some 2013-era advice for nginx here: https://www.igvita.com/2013/12/16/optimizing-nginx-tls-time-...

And some other advice and links here: https://istlsfastyet.com/

Pretty much all of them matter for first byte latency. OCSP & the key exchange both happen before any data can be exchanged.