Ideally ML tasks would get executed by background tasks e.g. Celery and use polling or a non-blocking event loop. It's generally not preferred to run such intensive process work inside the context of a web request.
Honestly I see a bright future for apache arrow flight but I think It's a bit immature right now.
But my main points is that 80% of the blog posts about FastAPI are for machine learning and people will just copy the code, run one or 2 requests see that it's fine and move along...
Ray uses apache arrow plasma store to avoid the copy and serialization costs that usually come with multiprocessing in python.
Most of the models I use can be converted to a more optimized formats for inference like https://treelite.readthedocs.io/en/latest/ so the code and interfaces are pretty similar and it makes the architecture less complex to update and deploy.
Depending on string names and handles is also very hacky and completely cripples PyCharm autocomplete / type checking.
The core of the project is great though. We're working on a video processing system that has a ton of downstream models and using Ray makes it possible to get away with running inference in pytorch without going down the TensorRT/inference engine rabbit hole (would be torture for some of our detection and sequential models).
For anyone looking into it I recommend starting with their Architecture Doc(https://docs.google.com/document/d/1lAy0Owi-vPz2jEqBSaHNQcy2...) and Design Patterns (https://docs.google.com/document/d/167rnnDFIVRhHhK4mznEIemOt...) Docs.