Apologies for the noob question but how would FastAPI/Flask know that the job has been successfully completed? Would the worker have to persist ml inference results somewhere and the FastAPI server poll it periodically?
One approach is to translate the synchronous call into an async call plus polling on the webapp side. You push onto a queue, with the callback queue in the message body. But that gives you problems when you want deploy a new version of your webapp - existing connection will be disrupted and the state lost.
Since you need to deal with retries, anyway, you can move the logic into the client itself. It will get the request id on the initial response and then ask the service for results.
You see, this solution can vary wildly depending on your scalability, durability and resiliency requirements. And on your budget. Its not wild to expect the response to be big, so you might want to upload it to s3. You might use websockets, too. Technology gives you a lot of options here, of different levels of complexity and scalability ;)
In that case, you trade off safety whilst the setup is still simple. But if querying the model takes less that 100ms I would argue whether thats cpu/memory intensive at all. Remember that you still need to allocate and populate the memory, and Python doesn't free the memory back to OS.