Looks awesome, but haven't been able to get a result back yet. I think you may be getting hugged to death :)
Using Chrome stable
Thanks for the help and info!
I'm using version 84.0.4147.89 (Official Build) (64-bit) and getting back responses.
I got the following response headers:
access-control-allow-origin: https://vo.codes
content-length: 151689
content-type: application/json
date: Mon, 27 Jul 2020 15:55:37 GMT
vary: Origin
x-backend-hostname: tts-group-1-965d444f5-7kvkm
edit: I must have an old browser. It works everywhere I'm testing it. CORS is hard. :(
I switched to Safari and Disabled CORS, but a 500 error is coming back now. So maybe the 500 response is the root cause, and the error handler is not returning CORS headers, masking the issue on Chrome.
Edit: by putting in a shorter input (sentence rather than paragraph) I was able to get a response.
What might've happened is that the instance your request was farmed out to might have been OOM killed. I've provided lots of memory, but these models are pretty massive and each inference run has to spin up a lot of matrices in memory.
This is all CPU inference, not GPU.
When the pods get OOM killed, they spin up again. The clusters for each speaker are about 5-10 pods apiece (with some double tenancy).