Concurrency in Swift: One possible approach
gist.github.com
gist.github.com
Having worked with async/await in Node.js a lot, it is of course a significantly better solution than plain promises, but it is also quite invasive; in my experience, most async code is invoked with "await". It's rare to actually need to handle it as a promise; the two main use cases where you want to handle the promise as a promise is either when doing something like a parallel map, or when you need to deal with old callback-style code where an explicit promise needs to be created because the resolve/reject functions must be invoked as a result of an event or callback.
Would it not be better to invert this -- which is the route Erlang and Go went -- and make it explicit when you're spawning something async where you don't want to deal with the result right away? In Go, you just use the "go" keyword to make something async. So the caller decides what's async, not the callee. If callers arbitrarily decide whether to be async or not, a single async call ends up infecting the whole call chain (which need to be marked "async" unless you're explicitly handling the promise/continuation without "await").
The problem with the above approach, however, is that you can't create hundreds of thousands of threads while also transparently supporting the traditional C ABI. C ABIs aren't designed to dynamically grow the stack, and so any thread that needs to invoke C (or Objective-C) code must always create threads with very large stacks (on the order of hundreds of several hundred KB or even megabytes) if they want to support legacy code.[1]
Languages that don't want to put the effort into growable stacks have no choice but to implement a solution that requires annotating the function definition, directly or indirectly.[2] The annotation tells the compiler to generate code that stores invocation state (temporaries, etc) on a dynamically allocated call frame (usually allocated by the caller) rather than pushing them onto the shared thread stack. This is true whether or not the function is a coroutine that can yield multiple values before finishing. Basically, without using a thread-based model, you can never put the caller in full control.
[1] Work on GCC Go necessitated adding a feature to GCC called split stacks. So GCC can actually compile C and (I think) C++ code that can dynamically extend their stacks. However, for it to work properly you have to compile everything with split stacks, including libc and all dependent libraries.
[2] Technically a compiler could emit two versions of a function, one that uses the thread stack for temporaries, and one that uses a dynamically allocated frame. That would put the caller in control. Some languages with very complex meta-programming capabilities (like various Lisps) can do this by making the await keyword a function which literally re-writes the callee into an async function that stores temporaries on a caller-provided buffer. So you can implement it without any compiler support. But it's still limited because the functions invoked by the async function would have to be rewritten recursively. The thread-based design is really the best approach, but it's a non-starter for many languages because of concerns about interoperability and legacy support. JavaScript rejected a thread-based model because existing implementations were too heavily dependent on the semantics of the traditional C stack, and they didn't want to throw away their existing investments.
Where did this claim originate from? It gets tossed around all the time in greenthread discussions but it's completely false. When you create a real thread even though it has an 8MB stack or whatever it doesn't actually allocate 8MB. 8MB is not the allocation size, it's the growth limit. The stack grows dynamically allocating memory when necessary until it hits that limit. The C/C++/<insert any language here> ABI doesn't need to be compiled for this because it's just page faulting. Standard OS behavior for decades.
It'd be manual but you could madvise it to shrink. Unlikely to be worth the effort unless you have one code flow that has a particularly deep stack vs. the common case, though.
I think this might be caused by a lack of complementary programming concepts, such as actors.
Having worked with C# and the Orleans actor framework I found that I'm now using more complex async constructs, such as await Task.WhenAll() and async Linq statements that actually make code run more efficient.
Adding async and actors together to the Swift language seems like a very good idea to me.
It's easy to start a some task with a completion handler. Even error handling isn't that hard, since communication of a failure goes in the same direction as communicating success.
But cancellation goes into the other direction! So whenever you start an async operation, you need to somehow store a handle or something, so you can cancel it later on.
The actor model makes this even worse -- how do you cancel a command that you previously sent? How do you tell an actor that they don't need to perform an operation, if the operation is still on the queue, or that they should abort the operation if they are already working on it?
If the actor model doesn't have an answer to this problem, developers won't be able to use it as is, and they will have to build additional abstractions on top of it before they can use it.
Edit: to implement it in a well-performing way.
It's inherently a half-measure: it's used to avoid wasting resources, but it only comes into play after you've already wasted resources. If you want to minimize waste, you're going to do better if you can minimize initiating operations that end up needing to be canceled.
Not that it isn't a useful refinement. It should be planned for. But I don't think it can be considered a critical feature out of the gate.
If you want to write an app that feels responsive, you must be able to cancel operations.
In case you're talking about rolling back an operation (you probably weren't, but just in case), canceling doesn't help there either -- not generally -- since you don't know how far the operation got before canceling, how long the cancel will take to propagate (or if it can at all), etc.
Yeah, cancellation in most systems are a "patch" (as you said, you don't know how long it will take to propagate (again, assuming distributed system, but this is less relevant when we talk about multicore system, with multicore system, that really is just about when a boolean propagated to all cores from false to true) but a necessary one and need to be integrated deep in the runtime. C# as an example, passed cancellation context (I believe in C# they called cancellation token) through all their async API as optional parameter. Maybe this is something Swift should take some inspirations from and will be beneficial for client-side developers.
If the operation is expensive, that doesn't work. Say, if the user clicks a link to view a 2GB file, then changes their mind to view a different file instead, you really don't want the browser to continue downloading that 2GB file only to discard the result afterwards.
But the problem is that pretty much any operation is potentially expensive. Sending a REST request might become expensive when your phone has poor signal. All of the sudden all those "inexpensive" REST calls you just ignored are queuing up only to have their responses discarded when they arrive several seconds later, wasting bandwidth when you need it most.
If the language doesn't make cancelling easy, developers won't support cancelling -- and then you end up with unresponsive apps.
What if you are too late and, say, asynchronous write to a disk was already commited and the data is out of your program and on the drive?
And how do you cancel a sent email?
At some point there is no sense in sending a "cancel" message, or there is no way of cancelling an action. How would you proceed in such a situation?
Eg. imagine that a document is auto-saved in the background, but the network is busy. Then the user changes the document. Now the app wants to auto-save the new version, but the previous version hasn't been saved yet. Saving the previous version no longer makes any sense and would just waste time, so you want to cancel the previous save operation, and save the new version instead. (it doesn't matter if cancelling the previous save is successful or not -- all that matters is that you don't want to wait for the previous action)
Or imagine that an app opens and tries to show the last document that the user displayed. The document is big, so the app shows a loading indicator. But the user doesn't want to actually view the document, he wants to view a different document, so he closes the document and opens a different one. Now the app needs to cancel loading the first document, because otherwise it would take much longer to load the second document that the user actually wants to see.
Most of the posts that I see related to Swift are RFCs evaluating solutions to problems in other languages. I rarely get to see the actual solutions being integrated into the language. Is this just a result of HN readers caring more about language design than learning to leverage the Swift language?
I also feel like with Swift people are so focused on pumping out their mobile app that best practices go out the window, or even that the knowledge is seen as such a valuable and proprietary skillset that the senior and experienced developers simply keep it to themselves and don't publish (compared to other OSS ecosystems)...
1. https://benchmarksgame.alioth.debian.org/u64q/compare.php?la... 2. https://medium.com/@rymcol/linux-ubuntu-benchmarks-for-serve...
1. Specifically for the first set of benchmarks linked to, they are really irrelevant because they don't represent typical workloads unless you're writing a MAAS (Mandlebrot As A Service). Node was designed for I/O efficiency, not fastest CPU-bound computations. That's exercising the JS engine (e.g. V8) more than anything node-specific. With node becoming more VM-neutral, it's entirely possible for other engines (e.g Chakracore or SpiderMonkey) to be better at other types of computation.
2. Especially in the second set of linked benchmarks, it seems the author of the article was hardly a node.js developer because they not only used an older node branch at the time they wrote the article, but they left out a lot of common optimizations (some of which were pointed out in the comments section). Even Express (which the author used) is known to not be very well optimized.
With that in mind, benchmarks are not the only thing you should be looking at IMHO. For example (for me personally), using a single language for frontend and backend is a big deal because there is less cognitive overhead when switching between the two (previously I often found myself writing JS syntax in PHP scripts and vice versa and trying to remember the APIs for different languages/platforms is difficult). There are many other benefits as well, just watch some of Mikeal Rogers' talks to get a better idea.
Since everything that is not async will need to invoke CPU-bound computations in Node (e.g. for loops, string manipulation, JSON parsing, and generally everything that's not just delegating work elsewhere with a callback), this I/O efficiency doesn't buy much except for very lightweight uses.
Most services in the real world soon get closer to Mandlebrot As A Service than "pure I/O".
Node still does OK-ish there because V8 is fast serially too (and of course everybody runs it on cluster mode or similar), but it's not like fast I/O by itself is enough.
>With that in mind, benchmarks are not the only thing you should be looking at IMHO. For example (for me personally), using a single language for frontend and backend is a big deal because there is less cognitive overhead when switching between the two
What about the reduced cognitive overhead of not having to deal with JS on the server though?
Plus, aren't usually the backend and frontend teams different ?
These are different levels of CPU-bound-ness. Comparing for-loops (which barely cost anything) to encoding video or computing PI for example is not comparing apples to apples. The body of a for-loop will typically greatly dwarf any "overhead" incurred by the for-loop construct itself. Obviously any non-I/O tasks like these are going to be CPU-bound. What I meant was Node is good at waiting for databases, web servers, file systems, etc. to respond to requests without blocking other work.
>Most services in the real world soon get closer to Mandlebrot As A Service than "pure I/O".
I disagree with this. I think most web apps spend most of their time waiting on the network, file systems, etc. to respond.
>What about the reduced cognitive overhead of not having to deal with JS on the server though?
I don't understand this question. JS is not a hard language to learn/pick up and there is a large number of developers out there who already know the language from working in the browser. However, my main point there was about using a single language. Yes, technically you can use Ruby, PHP, etc. in a browser (via JS), but nobody does that because it'd be very slow compared to just using JS from the get-go.
>Plus, aren't usually the backend and frontend teams different ?
It depends. I would say for most small businesses (and including the many "one man team" developers who do remote contract work for example) this is not the case. I cannot speak for large corporations, but there is definitely a larger possibility of having separate teams there. However just because you have separate backend and frontend teams doesn't mean having separate languages on each is any more beneficial (e.g. code sharing in some cases can be a win when using the same language). There's also the benefit of being able to "reuse" a JS developer on either end, depending on the work that's needed.
1. Swift's ARC uses atomic reference counting underneath, which is normally very expensive, and relies on compiler optimization to remove as many reference count operations as possible. This is normally pretty effective, but there are situations where it's not possible.
2. Reference counting allows for arbitrary long pauses as the result of cascading deletions (i.e. where object deletions trigger other object deletions). You can work around that (by deferring deletions), but then you don't have any guarantees about the timeliness of deletions anymore. As far as I know, this is still an open issue for Swift.
3. Without a compaction scheme, you risk memory fragmentation. While this is a rare occurrence in practice, there are workloads where it can happen.
4. Reference counting cannot reclaim cycles without a mechanism for detecting cycles; such a cycle detector (e.g. trial deletion) poses pretty much the same challenges as tracing GC.
Obviously, tracing garbage collectors pose their own challenges; my point is merely that whether performance and memory usage are more consistent has to be judged on a case by case basis.
2. Hmm, cascading deletions. Is that really a big problem in practice? I'm skeptical because it seems like that would affect C and C++ programs too, but you rarely hear anyone mention it. Maybe Swift tends to use more objects whereas C++ programmers tend to be better at packing stuff together?
3. Fragmentation -- that's true, but again, it affects C and C++ too. I guess for long-running C/C++ programs you're likely to manage memory pools directly. I don't know if that's possible in Swift.
4. Cycles -- weak references work fine for this. I have never had trouble with cyclic garbage in Objective-C. (I mean, I've had leaks, but they're always easy to spot with a leak detector and easy to fix with weak references.)
Overall, it seems to me that reference-counting adds a small but consistent performance penalty, and otherwise should have comparable runtime behavior to malloc/free in C, which is known to work pretty well when used correctly.
Note that Apple got smooth and reliable 60fps performance on the original iPhone, which was extremely resource-constrained by modern standards, using Objective-C, which isn't usually considered a fast language!
On the GC side, it seems like you typically get bursty, unpredictable performance, in both time and memory. Modern GCs work very hard to keep collection pauses as short as possible, but almost inevitably that means keeping garbage around for longer, which means using a lot of memory.
2. It only takes freeing a tree with a few thousand nodes for it to become an issue. It happens in C++, too (heck, there've been cases where chained destructor calls overflowed the stack [1]). The reason why you don't hear more about it is because pause times just aren't that big a deal for most applications. In forum debates, people always discuss triple A video games and OS kernels and such, but in practice, only a minority of programmers actually have to deal with something even approaching hard real time requirements. Generally, most applications optimize more for throughput rather than pause times.
3. Yes, and it can be a problem for C/C++, too. It's rare, but not non-existent. Note that pools can actually make fragmentation worse for long-running processes.
4. Weak references work if you get them right. But for long-running processes, even a single error can accumulate over time.
> On the GC side, it seems like you typically get bursty, unpredictable performance, in both time and memory. Modern GCs work very hard to keep collection pauses as short as possible, but almost inevitably that means keeping garbage around for longer, which means using a lot of memory.
This ... is not at all how garbage collectors work, especially where real time is concerned. Not even remotely. I recommend "The Garbage Collection Handbook" (the 2011 edition) for a better overview. And ultra-low pause times are generally more of an opt-in feature, because they're rarely needed.
[1] E.g. Herb Sutter's talk at C++Con 2016: https://www.youtube.com/watch?v=JfmTagWcqoE&t=16m23s
1. A call of a library function that typically cannot be inline.
2. Analysis of the object size in order to pick the right pool or a more general allocator to allocate from.
3. A traditional malloc() implementation needs to also use a global lock; thread-local allocators are comparatively rare.
4. For large objects, a complex first-fit/best-fit algorithm with potentially high complexity has to be used.
Modern GCs typically use a bump allocator, which is an arena allocator in all but name. In OCaml or on the JVM, an allocation is a pointer increment and comparison.
Even without bump allocators, it's easy for a GC implementation to automatically turn most allocations into pool allocations that can be inlined.
Also: much as people love to talk about video games, video games with such strict performance requirements are not only just a part of the video game industry, they are a tiny part of the software industry.
That's true, but if (hopefully rarely) the object turns out to be needed later, it has to be copied to another heap, and that takes time and memory. Pointers need to be redirected and that takes a little work too.
Bump allocators are definitely a huge win, as good as anything you can do in C/C++ and much more convenient for the programmer, but they're not a completely free lunch.
> This ... is not at all how garbage collectors work, especially where real time is concerned.
Hmm, I'm certainly no GC expert, but is it really not the case that GC tends to be memory-hungry? Not exotic academic systems, but the languages people use day-to-day.
Most of my experience with GCs is in languages like Java and C#. Java in particular can be very fast but always seems to be memory-hungry, using like 4x the memory you'd need in C++. I haven't spent a huge amount of time fine-tuning the GC settings (it seems like Oracle is working to simplify that -- good!) but the defaults seem to assume at least 2x memory usage as elbow room for the GC.
That's on the server. On mobile, I've worked with iOS and Android and iOS undeniably gets the same work done with much less memory. Flagship Android phone have 4GB of memory and need it, whereas Apple hasn't felt the need to bump up memory so quickly even after going 64-bit across the board.
The last I heard about real-time GC, with guaranteed space and time bounds, it sounded like it was theoretically solved, but not used much in practice because it was too slow. That was a number of years ago though. Has that situation changed? Are there prominent languages or systems with real-time GC?
The PowerPC (or ppc) and IA-32 (or Intel x86, 32-bit) instruction set architectures are supported by Jikes RVM.
Ah, no ARM and no x64, that'd be it.
What's keeping this kind of GC technology back from the mainstream?
The Jikes RVM is designed for research, not production. It's pretty impressive, but (inter alia) does not implement all of Java and does not support as many platforms.
> What's keeping this kind of GC technology back from the mainstream?
The fact that successful commercialization is possible; the GC tech that you see in Metronome and C4 is seriously non-trivial and not easy to reproduce unless you spend money on it; and it's also technology that businesses are willing to pay for.
At the same time, only a minority of open source use cases really require this kind of hard real-time GC, so there's little pressure to create an open source equivalent. Shenandoah is the one open source GC that does try to compete in this space, and it is trading away some performance for getting ultra-low pause times.
I'll add that this is difficult only because of concurrency and arbitrary sharing of data between threads. If you have one heap per thread, then it becomes much, much easier (and is a solved problem if soft real-time is all you need).
Which is not that different than with a GC.
Worse, sometimes, you don't even know if you're creating a leak. For example, I recently had to call, given two gesture recognisers a and b:
a.requireToFail(b)
A is a long-lived object. B goes away when the current view controller is popped. But not if A keeps a strong reference to it. Does it? Probably no one without access to the source code of UIGestureRecognizer knows!
It's a modern, fast, statically compiled, statically typed, language (that also has shims for interacting with legacy Obj-C code, but that's not a very important aspect) that unlike Go keeps up with modern PL features and expressibility, and unlike Rust has automatic RC-style memory management that you don't have to think about as much. There's no over-arching principle at play -- it's a pragmatic language.
>Why would I use Swift on Linux?
Because you like the actual language and its ecosystem (or not). It's not like you should be using any language just because of some "design philosophy" it's supposed to have.
>Most of the posts that I see related to Swift are RFCs evaluating solutions to problems in other languages. I rarely get to see the actual solutions being integrated into the language.
Well, there are several free books from Apple and tons of material to see what the language itself offers.
Considering it uses the same compiler infrastructure, it's mostly because of using higher level more expensive constructs. It can also generate code as fast as C++ in some cases (see the language benchmarks game).
>which incidentally also has everything you mention (mostly in the form of libraries to prevent bloating the language).
Well, hard to bloat C++ further anyway. It does have a better ecosystem and more mature compilers, but on a purely language level, I'd take Swift over C++ (including "modern C++" that still comes with all the historical baggage and a trillion gotchas) any day.
Unfortunately the problem here happens to be of a theoretical nature and not something a few years of engineering can reasonably fix.
http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan...
As evidenced by C#, you can't avoid leaking the type signature of async operations if you actually support generic programming- so while that's a nice ergonomics improvement, it only adds complexity to the actual concurrency model. Go enthusiasts out there will appreciate that go solves this by refusing to support PROGRAMMABLE generic abstractions at all (looking at you, channels and map).
Referencing the actor model and making it first class is interesting, but probably a mistake. Actors are hard to reason about because they're so flexible. Pony is a good recent attempt at combining static types with actors, bit they didn't put performance into the "non-goals" section of their language spec.
If you want task level concurrency and you want it to play nice with your type system, you have to start with Scala and work backward to the alternative implementation choices you're going to make because it checks all the boxes of all the "goals" and ALSO has a very mature actor model implementation that doesn't require promoting actors to keyword status in the language.
sudo is something different
I'm not saying it's bad, but I am saying that the flexibility limits the ability to reason about actor interactions (especially concurrent interactions), which makes it akin to the dynamic vs static type system debates. One is strictly more flexible than the other, and that necessarily makes it harder to reason about.
Did you have any specific improvements? The big requirements for distributed programming are mostly protobuf serialization and tcp/http; swift has both already.
Wrapping specialized interrupt handlers into some higher lebel API, such as AIO, is the right way.
Async/await is a mess. Concurrency cannot be generalized to cover all the possible cases. It must have specialization.
Engineers of old times who created the early classic OSes were bright people, contrary to current hipsters.
Once popularized, flawed abstractions and apis such as pthreads would stick (only idiots would accept shared stack and signals).
Look at how an OS implements concurrency and wrap it into higher level API. That's it.
How does the OS implement concurrency, which is a userspace problem?
Software interrupts alone are not enough, you still need some language support to it. I am not sure you really know what concurrency even means...
This is another gross misconception, like an attention to manage resources from the userspace, the way JVMs or Node are trying to do by poorly reimplementing OS subsystems.
Concurrency primitives must be implemented by an OS - it will eliminate all the userspace problems - no busy-waiting, no polling, no waking up for every descriptor, etc.
Ignoring an underlying OS instead of having thin wrappers, like Plan9 from userspace does, is exactly what is called ignorance (lack of awareness of a better way).
Wow, cool to see actual speed of light mentioned as a factor, never seen that before.
He also seems to carry a little weight in the community. I'm guessing his proposal might get implemented.
That'd be a massive undertaking with potentially no payoff, but I dream.
Apple and Google made a great partnership.
It's now 2017 and iCloud/web services still kind of suck while Apple continues to make the best OS and hardware (IMO, of course). We used to be able to have both.
I'd say crazy like a fox!
There is a well-known solution to this problem,
called async/await. It is a popular programming
style that was first introduced in C#