To claim "the ergonomics and the performance of not being integrated into Varnish are pretty bad" you would need to show some numbers. In my view, https://vinyl-cache.org/tutorials/tls_haproxy.html debunks the "ergonomics are bad" argument, because using TLS backends is literally no different than using non-TLS. On performance, the fundamentals have already been laid out in https://vinyl-cache.org/docs/trunk/phk/ssl.html - crypto being so expensive, that the additional I/O to copy in and out another process makes no difference.

But, again, if you have numbers, show them.