How does this compare to fauxpilot - https://github.com/fauxpilot/fauxpilot? Fauxpilot also uses Triton with fastertransformers and GPT-J style models (codegen).
[1] https://news.ycombinator.com/item?id=35471882
[2] https://news.ycombinator.com/item?id=35471390
I hope these can be of help and answer your questions as well!