DOs:
If you are doing any kind of soft-realtime (i.e. not batch processing) inference, by exposing a model on a request-response lifecycle, use Tensorflow Serving for concurrency reasons.
Version your models and track their training. Use something like MLFlow for that. Divise a versioning system that makes sense for your organization.
If you are using Kubernetes in Production, mount NFS in your containers to serve models. Do not download anything (from S3, for instance) on container start up time unless your models are small (<1Gb).
If you have to write some sort of heavy preprocessing or postprocessing steps, eventually port them to a more efficient language than Python. Say Go, Rust, etc.
DO NOTs:
Do NOT make your ML engineers/researchers write anything above the model stack. Don't make them write queue management logic, webservers, etc. That's not their skillset, they will write poorer and less performant code. Bring in a Backend Engineer EARLY.
Do NOT mix and match if you are working on an asynchronous model, i.e. don't have a callback-based API and then have a mix of queues and synchronous HTTP calls. Use queues EVERYWHERE.
DO NOT start new projects in Python 2.7. From past experiences, some ML engineers/researchers are quite attached to the older versions of Python. These are ending support in 2020 and it makes no sense to start a project using them now.