Background worker systems like Oban or sidekiq are where you want to run your async work imo. You can still do a pub/sub type notifier model to bubble up the job’s completion, or poll. Polling is perfectly acceptable for most apps and traffic levels and is simpler to mentally grok compared to websockets and persistent connections. Websockets require a central broker to pass messages through when you scale past one process and with elixir the options are the redis pubsub adapter, or for container based deployments or those wanting to use the built in distributed erlang functions, libcluster. More complexity. More points of failure. More potential centralized bottlenecks. More things to mentally model when debugging. More dev and prod drift. Just poll, until you can’t scale your web servers any more. It’s just an http request. It’s already implemented if you’re serving web traffic.
It’s not clear based on what you wrote but are the pdf’s contents being passed in the complete message? That sounds like an OOM termination that’s hard asf to track down just waiting to happen. PDF should go into blob storage somewhere and the task should bubble up a complete message, which then is used to signal the retrieval and proxying of the file to the user. If you have to background the pdf generation, it’s slow. If it’s slow to generate a document, it’s large. You don’t want that to be pushed through memory because your system stability is then coupled to not only the contents of a document that could exceed the memory of the system, but also is at the mercy of how many users/requests are all generating pdfs at the same time, and of course you’ll have that one guy spamming the “generate” button and queuing up 300 jobs because we didn’t need indicators on the UI and who adds uniqueness constraints to internal tools ;) Disks and file systems are for passing very large messages on a system