> The needs of people writing and deploying HTTP/REST servers is just so different from the people writing PyTorch models and numerical simulations that no tool will ever satisfy the very different needs of both camps.
Is this true though? At the end of it they both need python packages and some system dependencies installed (let's ignore models and data for now).
Why is there umpteen tools for doing this in python when there isn't in other languages? I have to deploy both web apps and ML models, the first thing I do is convert any project to use pyproject and poetry.
Whilst Poetry comes with it's own issues I've not had a project yet that this doesn't work for and wish the wider community would just settle on one method. Instead we have stuff coming in with various conda incantations, pip, pipx, poetry, setuptools, setup.py!
Deployment of ML models needs a decent solution too, half the ML code I get goes and fetches stuff from NLTK or Huggingface at runtime. Some of it (like the LLAMA models) needs various API keys set and EULAs agreed to before it'll run, then pings back to Huggingface each time you run it to check the EULA again! This makes life difficult when trying to deploy and adds this massive dependency on 3rd party services.