Exploring Coroutines in PHP | Hacker News Reader

Exploring Coroutines in PHP

doeken.org

116 pointsdoekenorg65 comments

Just to clarify my intent with posts like this:

I'm not suggesting that PHP is the ideal tool for every use case. The goal is to share a concept that might be unfamiliar to some developers, using PHP as the context.

Sometimes learning about a concept in a familiar language helps you recognise where it might be useful elsewhere or apply it in a language that supports it better.

Terms like coroutines, concurrency, promises, etc, can be confusing; I just like to demystify them with easy-to-grasp examples. That does mean that examples can be contrived or very simple, but they are designed to get the point across.

Thanks for all the comments so far!

Unfortunately, as HN has had more users, the quality of discussions has gone down a bit and is flooded with routine complaining of this kind.

You just discovered what happens when you talk about PHP, and there are similar "Godwin's laws" for other topics, such as IPv6 (we needed IPv4 with extra octets), Google/Apple/Microsoft (any company trying to achieve a commercial objective is always equivalent to "enshittification") etc. Don't mind these too much - it's a good article!

Contrived examples are okay if they help trigger a click in the reader's mind, "wow, I could use this to solve problem X in an actual project!"

The difficulty with these examples is that they are very different from the actual tasks that everyday web developers would like to parallelize, such as long-running database queries and API requests. These things often take orders of magnitude longer than any pure-PHP loop that a typical webapp might contain.

An example that fires off an async query and yields the result when it's ready will probably produce the right click in the minds of many more people. (mysqli can do this, but the interface is convoluted and badly in need of a Promise-like wrapper. I'm not sure if PDO/PostgreSQL even supports async queries.)

I agree, but I had to draw the line somewhere on this article as it was already getting pretty long. And since I'm tackeling concurrency in the next post, it made more sense to me to start talking more about async there, with examples. Bear with me. Once that post is out, I'm updating this one which will reference the other for async examples. Thank you for your feedback!

I've only read part of the article. So far, it's great.

I haven't used it in ages,but I like PHP...

I have read much about Fibers since they were introduced and have never come up with a real use case.

All the examples I find are like the trivial ones here where it just feels like instead of jamming a bunch of code into a single messy function that yields, you'd be better off particularly from a static analysis standpoint just having ordered method calls or chained callables where each step returns the next step as a callable.

I've yet to see a use case where I can't come up with a safer way to do it without Fibers, but I would love if someone could enlighten me because I feel like I am absolutely missing something.

I once had a use case where I wrapped a callback based library (Amazon's S3 library I think) call to instead return an iterator. Couldn't do that with only a generator since you can't yield in the callback itself. And it wasnt possible to write a custom iterator class since the library didn't give you back control until it was done.

That was the first and only time they were kinda useful to me.

Fibers are incredibly powerful, as they can be used to implement seamless go-like concurrency with async, colorless functions.

They were added to PHP by the maintainers of amphp (https://amphp.org), which is the best library for async PHP out there, providing a clean, object-oriented and async API for files, databases and everything that can make use of async I/O.

A fiber is a colored function. It's a different color than an async function, but it's not blindly swapable for a regular function.

Async functions based on PHP fibers are explicitly uncolored, they do not require any special keywords to be invoked, and are explicitly swappable with any regular functions.

I have a service using them a lot. It ultimately relies on a couple dozen downstream dba and endpoints, with some calls requiring strict dependencies, and others running in parallel. So I just draw the dependency chart among the functions, and let the service manage all the parallelism and asynchronous bits with minimal babysitting. If it was all chained, then sure, it's not a significant gain

I think you're not missing anything (or at least, I'm missing it too - not much advantage over yield). Generators rock though.

>> This would require a yield to syntax, which doesn't exist.

There is however the 'yield from' statement.

This means, that most likely the keyword will not be "yield". PHP is not known for keeping a low number of keywords. Rather they introduce new ones, to tack on things to the language, like one can see in the implementation of anonymous functions, where one needs to use "use" explicitly, instead of the language having the semantics that most other languages have for lambdas. Which immediately turns in to a source for nullability errors.

That's not totally correct. PHP has a "short function" syntax for that specific use case, it automatically captures data without the 'use' statement.

    $a = 5;
    $b = fn ($c) => $a * $b;
    print($b(2)); // 10
    print($b(4)); // 20

This "arrow function" syntax was introduced much later, though.

I'd use `yield from` more if I didn't have to re-index arrays before yielding from them:

    function numbers() {
      yield from [1,2,3];
      yield from [4,5,6];
    };
    
    foreach (numbers() as $k => $v) { echo "$k => $v\n"; }
    0 => 1
    1 => 2
    2 => 3
    0 => 4
    1 => 5
    2 => 6

PHP generators/iterators is a hot mess. I don't know any other language which forces iterators to output a key for `foreach(... as $key => ...)`.

PHP doesn’t force keys... You can omit the key and simply write `foreach($items as $value)`

What's idiomatic way to get index (0, n-1) with the value? Parent example shows you could not use $key as a generic solution.

You can use the SPL LimitIterator, feed it the generator, and give it the offset and limit.

``` foreach (new LimitIterator($generator, 0, 3) as $value) { echo $value; } ```

https://www.php.net/manual/en/class.limititerator.php

BTW: https://www.php.net/manual/en/iterator.key.php

It's literally in the interface.

I’m not sure I follow, what exactly is your complaint? The Iterator interface is described as:

> Interface for external iterators or objects that can be iterated themselves internally

Note “external iterators or objects”. The Iterator interface is not exactly everyday PHP, it’s a specialist utility for making classes iterable so they can be accessed like arrays. Most developers will rarely use it directly, and it’s not being used in the parent comment’s example either.

Iterating over something requires knowing where you are in the sequence, so of course you would need to implement a method to get the current position of the iteration.

> Iterating over something requires knowing where you are in the sequence, so of course you would need to implement a method to get the current position of the iteration.

No you don't. Other languages don't require it. There is no issue to get a position outside of iterator and it's more generic approach.

I don’t think you understand the purpose of PHP’s Iterator interface.

Let me guess. To somehow patch iteration on associative arrays? And instead of bringing pairs or tuples as first class citizens it extends iterators with `key` and `value`. And now any Iterator implementation should track own sequentially increasing key. Very nice design indeed.

Associative arrays are iterable internally without needing an interface or key tracking. You can just foreach them.

> Let me guess.

This just proves my point, you call this a “hot mess” whilst completely misunderstanding what it’s even used for.

The Iterator interface isn’t even used in the comment you first replied to.

"It's a hot mess" - someone who knows absolutely nothing about what they're talking about.

Typical HN comment about php.

Defenders of PHP's iterators clearly tasted only carrots.

Yes, there is! However, even if the `iterable` it was yielding from a different Generator, the "outer" generator would still be "running", which still makes it asymmetrical. `yield from` is mentioned in the other post on Generators, though.

Personally I've never come across any problem in PHP that I feel like Fibers would help me solve.

Having Fibers in PHP is a nice addition but it definitely feels more like plumbing for other PHP extensions/frameworks to use, rather than something the average dev would use themselves directly day to day.

Of course you haven't come across any such problem in PHP, because up until this point, if you had, you would have had to switch to another language to solve the problem that required the feature PHP didn't have.

There's an evaporative cooling effect on languages that don't have certain features where all the people who need those features leave after some number of years of not having them, leaving behind only the people who don't need them. There's a survivorship bias as a result.

I worked with dynamic scripting languages primarily for the first 15 years of my career but the definitive split for me was precisely when I had a problem they couldn't solve because they couldn't handle tens of thousands of simultaneous connections in any reasonable way.

This reply applies to all the commenters posting "but I've never needed this".

That doesn't mean PHP was useless without this feature, as it observably has solved a lot of problems. It just means that you shouldn't draw out too many conclusions from "I've never needed it", because you're still using it precisely because you're working in a space that doesn't need it, and your community is not constantly complaining about it because the people who would be complaining are no longer in your community. It means if you do develop a need for it in the future you won't have to leave PHP.

As a result this is a bad metric to measure the utility of a feature with.

On the flip side, it is valid to say "we've come this far without it and maybe we should be focusing on what we can do and continuing to invest in making that better rather than chasing the things we can't", especially since features like adding true threading add a lot of constraints to an implementation and make everything else you ever do in that implementation more expensive. Personally I am of the opinion that all of the dynamic scripting languages really need to just settle down, accept that they are what they are, and stop adding feature after feature after feature to 25-30 year-old languages.

Genuine question, what kind of problems would using fibers help with?

To be clear, my point wasn’t that I think fibers are useless or that people shouldn’t use them. I think it was a great addition.

Just that they wouldn’t be directly useful to the average PHP dev, until they’re being used by frameworks/libraries/extensions, say for example: a HTTP library that can fire off multiple requests asynchronously.

One of my standard answers here is a chat server. You've got thousands upon thousands of connections to it, which are all live, and they're cross-talking with each other in arbitrary ways. The easiest way to write that is to assign a fiber per connection. You shouldn't do that in PHP today.

"Just that they wouldn’t be directly useful to the average PHP dev,"

But you're restating my point, whether you realize it or not. They're not useful to the "average PHP dev" because if the "average PHP dev" needed them, they would cease to be a PHP dev, just as I ceased to be a Perl dev when I needed something it couldn't do for similar reasons. All the use cases that PHP could have with fibers have evaporatively-cooled out of the community because PHP couldn't do them, leaving behind precisely that set of people who don't have problems that could be solved with fibers.

>>The easiest way to write that is to assign a fiber per connection. You shouldn't do that in PHP today.

Can I ask why? Is the implementation not performant enough?

Wow, it's cool to finally get closure (:-P) on a question I asked in 2012:

https://stackoverflow.com/questions/12939319/coroutines-in-p...

Looks like generators were released in php 5.5 in 2013:

https://versionlog.com/php/5.5/

I was interested in coroutines because most backend server logic eventually devolves into a sea of state machines where each request/response advances the state by updating database rows. This becomes unmanageable by humans, which is why server code can only reach a certain level of complexity, perhaps 1 million lines, before it becomes "enterprise" and triaging overtakes architecting as the main mode of operation for developers.

This is akin to how in the 1990s, object-oriented programming (OOP) limited the size of most desktop programs to around 1 million lines, due to similar state management limits under imperative programming.

I had hoped to replace the state machine soup of backend API endpoints with coroutines that guided users through stuff like their onboarding steps along one-shot functions made up of mainstream conditional logic and higher-order methods.

This history explains why most websites and apps today have so little actual business logic. Most would be considered entry-level or semester projects for desktop developers in the olden days, who were forced to wrangle the complexities of C++ and Java.

Today, most time is lost to the idiosyncrasies of managing build pipelines, version updates, boilerplate, etc. Meaning that we're too mired in babysitting our tools to see how old school approaches like using spreadsheets and batch files in office environments would make a mockery of our work. Now it's mostly all a waste of time, and we can feel it, but I digress.

Anyway, I had attempted to make this state machine <-> coroutine bridge by saving php functions and their state using the jeremeamia/super_closure package:

https://packagist.org/packages/jeremeamia/superclosure

https://github.com/jeremeamia/super_closure

Which gave way to opis/closure:

https://packagist.org/packages/opis/closure

https://github.com/opis/closure

Which gave way to laravel/serializable-closure:

https://packagist.org/packages/laravel/serializable-closure

https://github.com/laravel/serializable-closure

That way the coroutine would get resurrected during each user request and proceed through its logic. Thereby removing the mental load complexity limit imposed by state machines and allowing 1 or 2 developers to compete with larger enterprise teams at big companies.

Since then, I've abandoned these types of approaches and moved towards pure functional programming (less state management and fewer side effects), declarative programming (repeatable processes that eventually meet constraints imposed by integration tests), and data-driven development (higher-order methods on trees and graphs). So lots of work with spreadsheets, Terraform, Firebase, etc. I find that programming languages mostly get in the way now.

After a career mostly spent hacking on legacy code and tearing my hair out, I yearn to be free to get real work done. This would look like abandoning most approaches people are pursuing today. For example, most of the async/await stuff in Javascript is an evolutionary dead end, because we already went down the cooperative threading road in the 1990s and discovered that there was no there, there. Async is today's goto. Same with absurdities like the "final" keyword, which is a self-flagellation habit born from difficulties around name-mangling when exporting C++ methods in object files. The compiler should reorder structures and classes to match the constraints of the runtime, not humans. Otherwise we break Postel's Law:

https://martinfowler.com/bliki/Seal.html

https://martinfowler.com/bliki/SoftwareDevelopmentAttitude.h...

https://martinfowler.com/bliki/DesignedInheritance.html

https://martinfowler.com/bliki/OpenInheritance.html

https://martinfowler.com/bliki/TolerantReader.html

With this context, we can see how large powerful companies have doubled down on the Directing Attitude and Designed Inheritance to the point that we're handcuffed to our tools. They've done little or nothing to advance the state of the art of our languages and frameworks from first principles to be more freeing by providing more leverage. For every revelation like Erlang and Go, there are countless AWSs and Reacts. Forcing us to focus on specifics like regions and edges, side effects and performance, etc. Because nobody did the real work of designing distributed systems that "just work" via techniques like true multiprocessing, memoization.. I could rant forever. It's all bare hands work now, by us serfs under neofeudalism.

Sorry this got long, it's a passion project for me, a dream I may never have time to live if the rest of my life gets lost to making rent.

I think neither does PHP have the ecosystem for it, nor does it have the language primitives or standard library for it. Much cleaner languages than PHP already have enough issues getting concurrency things like fibers right. I don't see this ending in any implementation, that doesn't have surprising dysfunctional parts in it. Especially not, when again and again people behind PHP chose the easy way out for adding language features, like they did with lambdas, that don't capture their environment, unless you explicitly state what they capture.

Let alone the standard library being an unfixable mess, if they want to pursue backwards compatibility. What they would need to do is a clean break. Something like "PHP 10 is a breaking change, we are modernizing the base language and standard library and getting rid of decades of cruft." -- Which then many PHP users would hate.

No, PHP is a not a language whose design has what it takes. A library that claims to have such advanced stuff implemented in PHP is not to be easily trusted to be free of hassle and tripwire, because the language itself has those built-in. For example with Fibers, one has to manually call suspend and resume. Other languages get fibers done without this hassle.

EDIT: For all the downvoters, who don't care to actually give reasons or discuss, the cracks are already showing in the Fibers manual itself. Look at the first comment for example: https://www.php.net/manual/en/language.fibers.php#127282 which links to a bug/issue in the Fibers repo. It is a bug, even recognized via issue label, it is verified, as recognized via issue label, and it is ... closed. Apparently it will not be fixed, and has been simply closed.

> like they did with lambdas, that don't capture their environment

https://www.php.net/manual/en/functions.arrow.php

Yes this exists, but again introduces a new thing into PHP, rather than using an existing thing. Instead of using existing function syntax and leaving away the name, now one has 2 more things: The "fn" (why can this not be "function" which already exists?) and in combination with "=>". So it is similar to "function ... use ..." which also has the need to introduce a new thing, the "use", instead of simply capturing its environment.

The point is, that the designers of PHP decide again and again to add another wart, rather than designing the language in an extensible way. Here a little extra, there a little extra, here a new keyword, there a new thingy. Compare that with other languages. Even JavaScript has managed to have simply the same syntax for functions and anonymous functions. Although it did also introduce an arrow syntax. However it's arrow syntax is not really needed, even though shorter, in contrast to PHP, where you need it, unless you want to list all the parts of the environment you need in the anonymous function. At least in JS they didn't introduce a new extra thing like "fn".

I don't think the design of PHP is done with good oversight. It is done with the approach of tacking things onto what exists, in the easiest way they are able to do, without any goal of keeping the language concepts and keywords minimal. Designing something like "fn" or "use" is a hack to make implementation easier, because they are previously unused keywords, that will then make it easier to adapt the parser for the language, but these decisions are offloading mental load to the user, which ultimately is a bad design.

You might say "PHP is a fractal of bad design"...

The standard lib thing is overblown. What exactly is bothering you?

And PHP has deprecated a lot of things along the way.

PHP watched the absolute disaster of python 2 to 3 version upgrade and rightfully decided that it will be as much backwards compatible as possible.

PHP 10 with a clean slate is a cute idea that would wreak havoc. Perl, or whatever it is called now, tried this and it was a total disaster.

I personally never perceived Python 2 to 3 as a disaster, and the language has come out better on this end. It would have been a disaster to keep all the cruft of Python 2 forever and keep writing software in it. When Python 3 became stable, I simply never looked back at Python 2 again. Sure, many libraries had to change a bit, to accommodate Python 3. But any sane project would start with Python 3 anyway, and over time Python 2 support would simply fade away, as fewer and fewer people use it. Maybe during some transitional time people had backports of security fixes for Python 2, but in this day and age I wouldn't actually bother with that any longer. Whoever still runs on Python 2 has made a decision to do so, accepted responsibility for not transitioning and has no one but themselves to blame.

Overall they actually did a pretty decent job to transition from Python 2 to 3. Many people didn't have any issues at all, and simply used the new thing instead of the old thing.

Python's language design in itself is debatable of course, but it certainly is better now than with Python 2.

I was maintaining some debian servers back then, and for more than a decade there was great confusion around python. What you describe as "Sure, many libraries had to change a bit" was a major pain in the butt for many people for many years. I'd describe this as a disaster. The "just start a new project"-mentality is not what happens on servers and companies of the world. They support stacks, that are literary decades old.

I don't write PHP code anymore. I had a great time doing so for years but now I mostly write in Go for a company that writes a lot in PHP.

What I see from PHP is a missed opportunity for not having any native lightweight multi thread capabilities not a robust HTTP server.

I wish the situation changed.

My hope is that the parallel extension will get more widespread adoption when its integrated into FrankenPHP: https://github.com/krakjoe/parallel

I still have flashbacks from working with the pthreads extension, which caused extremely hard to debug, non-reproducible segfaults sometimes; I realise Joe has probably started from scratch and improved a lot on that (and I know he's a generally awesome guy), but without a properly financed maintainer team to support him, I'm not sure I want to take that risk again before parallel has gained some maturity.

I work daily with PHP and honestly nearly all my code I write is synchronous.

The shared-nothing architecture of PHP makes that really a non-issue for me. Requests never share any state with each other. Something like RabbitMQ can handle communication between systems.

That's in no way lightweight though, and most languages can easily do the same. Just launch multiple instances/VMs/processes. That's having multiple separate OS processes, each having everything that is needed to run PHP, and having no way to communicate with each other, other than what you implement. No channels, no task distribution, no queue on which to sync and take tasks from, no signaling of all processes being done and then accumulating the results. That is why you then need something like RabbitMQ or other things, and it does not mitigate the heaviness of the approach.

It is kinda funny, that you mention RabbitMQ, which is written in Erlang, which is famous for its lightweight processes. But also compare the approach with thread pools built into the standard libraries in other languages. And even many of those are heavy weight compared to Erlang's lightweight processes.

PHP is not about lightweight, it's about rapid development. A lot of implementations in PHP are dead simple and can be debugged even by beginners.

You generally do not implement efficient systems in php, they are easy to debug, fast to code and quick to fix though.

Most of the time you don’t really need to think about memory management either as the memory for the process is simply reset on every request.

PHP's approach is simple though, and in my experience that simplicity pays off when you do start scaling the systems.

In other systems once you get beyond a single machine you need that external communication mechanism anyway, and now you have multiple classes of comms which introduces bugs and complexity and performance cliffs.

In PHP you just throw another server at it, it'll act the same as if you just added another process. Nice linear scaling and simple to understand architectures.

Man I miss working in PHP

The shared-nothing architecture is great for some scenarios.

Long running processes and async I/O are a great tool to have though. They are present in PHP for almost two decades now, and despite having many incarnations (select(), libevent, etc) and even frameworks (amp, reactphp, etc) the knowledge is highly transferrable between them if you understand the fundamentals.

I agree, but I would always pick a different language (like Go) for long running processes. PHP is great for shared-nothing apps though.

Don't worry, we're not comparing languages here. You are free to chose the language and community that fits best to your approach to software development.

Why, thank you for your generosity, you are too kind!

It's better to build your app in e.g. PHP, prove its worth, then identify the bottlenecks, THEN determine if you need multi-threading. And if so, determine if PHP would be the best language for it or if you'd be better off going for a different language - one with parallelism / multithreading built into its core architecture.

The first logical step after PHP is NodeJS, which has the fast iteration cycles of PHP without the low-level memory management or the enterprisey application server headaches of other languages, AND it has the advantages of a persistent service without needing to worry too much about parallelism because it's still single process.

But if you still need more performance you have a few options. But if you're at that point, you're already lucky. Most people wish they needed the performance or throughput of a language/environment like Go.

By this logic, it seems like it would make more sense to just start with Node rather than PHP for the prototype and save the potential rewrite. Node does seem more popular than PHP nowadays to me as an outsider to both, so maybe that's exactly what did happen.

> The first logical step after PHP is NodeJS, which has the fast iteration cycles of PHP

... not really, you still have to deal with bundlers in real-world applications.

I use bun and skip builds.

> Most people wish they needed the performance or throughput of a language/environment like Go.

Most people do need the performance and throughput offered by modern languages like Go, though. Time to market is the most important consideration for most. Maybe at Facebook scale you can spend eons crafting perfection in a slow-to-develop language/ecosystem like PHP or NodeJS, but most people have to get something out the door ASAP.

The execution model is definitely the biggest issue with PHP these days.

Although now that the PHP Foundation is officially supporting FrankenPHP maybe things will be evolving into a new paradigm.

https://thephp.foundation/blog/2025/05/15/frankenphp/

There was a long discussion recently on FrankenPHP.

https://www.reddit.com/r/PHP/comments/1lqpkfq/frankenphp_any...

But why pick PHP then? Why not use nodejs or similar where the language, application stack and the community is already in agreement on the execution model.

I don’t use php but it’s extremely popular around the world. There’s also Laravel which seems a lot more productive than anything in js for full stack dev.

Reply on news.ycombinator.com