I like how you made this ultra-simple. It makes it very easy to understand and use. There are times when a python+json server isn't performant enough. To help with this, my team recently built a protocol for efficient model serving called GraphPipe[1] that allows you to do serving like this. There is an example for serving from python[1]. The easiest way to get started is to use the go server in a docker container[2].

[0]: https://oracle.github.io/graphpipe/ [1]: https://github.com/oracle/graphpipe-tf-py/blob/master/exampl... [2]: https://oracle.github.io/graphpipe/#/guide/user-guide/quicks...