It's been over a year since I had to set this up, but I used HAProxy in a similar setup and still needed stunnel.

In the whole of my DevOps career, getting SSL working with web sockets was one of the most frustrating.