Hacker News Reader

Comment by solraph | Hacker News Reader

Not the parent, but it would be nice to have separate thread for things like logging and firing events into a queue.

Just curious what problem are you trying to solve, I never had the need for different threads to communicate in PHP, most of the time in my work a user makes a request, and a PHp scripts does something like update the database or does some searching/working and returns a result.

What if that request was a document upload and your system is calling out some external api or processing for long enough such that you’d rather return now and update later when the job finishes. Maybe through Ajax or a redirect.

The shenanigans that WP plugin devs go through to get around the lack of a queue is ridiculous (chunk the file and keep reloading the page until we’ve processed all the chunks or have an Ajax endpoint that processes a chunk at a time...) and the when I asked a marketing friend of mine who works with Wordpress he said that redis is an “advanced requirement” and I should keep things simple.

In Django land I’ve always set up queues even just to send emails because it keeps things snappy.

For that kind of problem we create a database table. then you schedule there your work tasks and background scripts will fetch jobs from the database and run them. This also works great if the third party is busy or fails, the background jobs are set to attempt a few times before giving up.

What solution would you prefer for this kind of jobs? launching a new thread directly? or have a worker project always running and send to it the job directly?

You can easily setup and use queues in PHP projects too.

.... As you identified at the end, the correct solution for this problem is a job queue. There is nothing about PHP that prevents the use of a job queue - in fact the shared-nothing, process-per-request model almost encourages the developer to offload as much work as possible to a job queue..

Using Wordpress (or it's plugins) as any kind of reference for what PHP is capable of, is like using McDonalds as the basis for a cook book.

If putting a job/event into a queue, or writing a log entry is taking so long that you need a separate thread just to enqueue/log that entry, something is dreadfully wrong with your application.

Ironically, the root cause that made me think of that example was AWS.

The example was where each SQS call (there could be as many as five) was taking between 50ms-200ms, and logging took another 75ms. The rest of the request processing time was approx 50ms.

There's a bunch of other architecture options we could have looked at, but due to the share-nothing nature of PHP, they weren't an option.

... you do know that you can send a response to the client, close the connection and do something else before the process "finishes" right?

I do. However, the process would have stuck around until it finished. It wouldn't have changed our overall RPS at all.

Arguably the same problem would exist in other languages, but at least there we would have had the option of using threads, and we could have executed multiple calls to SQS in parallel. Probably not something I'd want to do all the time, but it would be nice to have had that option in this case.

If it’s a http api, there’s always curl_multi to do multiple concurrent requests.

Reply on news.ycombinator.com