Interesting project, I've heard a lot of good things about GraphPipe for Model Serving with support for multiple languages (Go, Python, Java) https://github.com/oracle/graphpipe
Graphpipe solves a very unique problem when building ML model serving system, although BentoML is trying to solve a very different problem. We think it would be interesting to support GraphPipe's flatbuffer format in BentoML's REST API model server down the line if people are interested in that.