Safe and efficient C++ interoperability via non-escapable types and lifetimes | Hacker News Reader

Safe and efficient C++ interoperability via non-escapable types and lifetimes

forums.swift.org

67 pointsmatt_d47 comments

The recent Swift updates add increasingly niche and theoretical features that are of questionable practical use.

Meanwhile the compiler still can't compile many even mildly complex SwiftUI views and expressions with the "The compiler is unable to type-check this expression in reasonable time" error.

It does not even tell you what part of code it is having issues with so you could change it. You have to comment/uncomment blocks of code to find the problematic part.

The same could be said for the similar C# and .NET features. The practical use for many low level constructs is for library authors, which is perhaps 1 developer out of 1000, but those libraries are used by the 999 others which means the "practical use" of the feature is a lot more than it would seem at first sight.

> Meanwhile the compiler still can't compile many even mildly complex SwiftUI views and expressions with the "The compiler is unable to type-check this expression in reasonable time" error.

That seems like a bug in language design? What's the story behind that? Did they inadvertently make something that was quadratic or worse and then realized it was too late or too fundamental to fix it properly so they patched it over with an error message?

Yes, it has to do with how its type inference interacts with certain language features (particularly return type overloading).

It's unfortunate but I would honestly be ok with a "strict mode" that requires extra annotations from the programmer if it would prevent this stupid error in practice.

Here's a good blog post on the issue: https://danielchasehooper.com/posts/why-swift-is-slow/

> That seems like a bug in language design? What's the story behind that? Did they inadvertently make something that was quadratic or worse and then realized it was too late or too fundamental to fix it properly so they patched it over with an error message?

This is what I would like to know, too, but you phrased it way better than I did.

Here's a good blog post on the issue: https://danielchasehooper.com/posts/why-swift-is-slow/

Ok after reading that I'm still amazed by the mistake. Surely it must have shown up before Swift was even generally available? And surely having even ONE case take several seconds for a single expression should have sent it back to the drawing board? Something still seems strange.

I can imagine people start linting to require (TypeName.Membername) instead of just (.MemberName) because they hope it can speed up compilation. And at that point it again feels like a big mistake to have allowed it from the beginning.

Yeah I have read this not long ago, it is a great post.

> A different approach to type checking is required to fix it.

I agree, are they actively working on this though?

I don't see what they can do about this in general, short of requiring sufficient explicit type annotations to mitigate the combinatorial explosion of possible combinations. But if that is an acceptable solution, you can already add such annotations to your code regardless.

C# actually has a similar issue with lambdas passed to generic functions, for similar reasons. It's just what you get when you combine overload resolution with type inference.

Those niche features are exactly what allows us to move beyond C and C++, safely.

Naturally there are those that would rather keep the status quo.

> "The compiler is unable to type-check this expression in reasonable time"

Lol this sounds funny, any other compilers that do this? Why does this happen exactly?

> It does not even tell you what part of code it is having issues with so you could change it. You have to comment/uncomment blocks of code to find the problematic part.

Awful.

> any other compilers that do this?

TypeScript has a notorious "Type instantiation is excessively deep and possibly infinite" warning that can often be unsafely ignored with a `@ts-expect-error` comment

You’re aware that the comment was specific to (I would say) reasonably complex SwiftUI expressions? How familiar are you with the context here?

So you are saying this is specifically limited to SwiftUI? This fact does not make it any less absurd, especially with the last part that has been said.

My question stands: why does this happen and how come it is still so primitive that you do not even get to know the location of the issue?

https://danielchasehooper.com/posts/why-swift-is-slow/

Thank you!

    let result: Double = -(1 + 1) + -(1 + 1) - 1

> Swift 6 takes 6.2 seconds to compile this one line. Even toy languages compile equivalent expressions faster.

Wild. :P

Note that, while this is definitely crazy, it's rare in practice if you avoid gluing a bunch of magic literals together using operators, which I already avoided as a habit. I've never encountered the exploding-compile-time issue once while writing production Swift, which I have done almost every day for a decade.

WITH THE EXCEPTION OF SWIFTUI, which relies entirely on inferring highly nested generic types, and stands out as a uniquely problematic Swift framework for that and other reasons. I don't use SwiftUI.

I was a swift developer for ~5 years before taking a rust job recently. The gap in tooling quality between swift and almost any other contemporary language is immense.

Using Xcode taught me to keep my expectations low for my programming environment. The compiler would straight up crash with shocking regularity. The debugger was utterly useless a huge majority of the time (it would just lose track of symbols and couldn’t inspect anything when you hit a breakpoint, every expression you would type to lldb would fail with an inscrutable error.) I had to keep a script around to blow away all of Xcode’s DerivedData and restart it, and I probably used it a half dozen times a day. It was terrible.

With Rust I’m using RustRover, which IIUC uses rust-analyzer which is part of the vanilla toolchain (ie. vim/neovim or vscode will be just as good as they can use it too), and I’m continually shocked at how well it works. I’ll be typing an expression that has an ambiguous type, and the moment I finish it with something unambiguous, the editor responds by filling in all the types around it within a second. And this is with tons of map/filter/iter expressions where the type inference is insanely complex. There’s no restarting the IDE because it’s out to lunch. There’s no expression that rust-analyzer can’t figure out but the compiler can (unlike swift which seems to have one type checker for editing and a different type checker in swiftc and they disagree constantly.) It’s fast, it’s reliable, and most importantly it’s correct.

Trust me, if you think Swift’s tooling issues are unavoidable reality, you need to see how good other languages have it.

I'm not defending the whole toolchain - I agree Xcode is probably the worst thing in the landscape of mature languages. I just think the infamous exponential-compile-time issue, and the "Swift is too complex now" issue, are sometimes overstated. Again, excepting SwiftUI, which is a huge exception, so I'm not exactly writing a glowing review. I just love Swift itself, and most of Apple's APIs.

I don't have an authoritative source, but I believe RustRover does not use rust-analyzer and has its own engine written in Kotlin instead.

Correct. JetBrains is allergic to LSP in general, as it directly commoditizes their core business. For example, they are only now developing a stable Kotlin analysis API that could maybe be used in a future third-party language server, despite the advanced age of both Kotlin and LSP.

The only program I’ve ever had take longer to open than Xcode is StepMania, and that’s because it has to parse thousands of simfiles.

Break up the view and you’ll see much better performance with SwiftUI.

> how come it is still so primitive that you do not even get to know the location of the issue?

The link kind of explains why this was a difficult problem, but I think it's worth a moment to consider why this particular difficult problem just wasn't solved. Why was Apple OK with just shrugging, eh, it's hard ?

This is a Quality of Implementation issue. Quality is truly important but because Quality is hard to turn into a metric you'll see management don't like to make it a goal, after all they can't really measure if they succeeded and so when it is competing with other things which are goals it's just abandoned.

Yeah, what I wanted to know is how come that Apple has not solved this issue yet, or how their decision-making goes with regarding to adding X and Y instead of fixing this absurd issue.

I am saying that it's unhelpful to miss the context, and it would be a shame if one was to misread that the Swift compiler can't even provide error locations, which it obviously can in the vast majority of cases.

Swift's compile time performance is obviously well documented. It's not great, and performance is definitely one of the trade-offs. SwiftUI leans heavily on the compiler and is very much at the sharp end of this. Most "normal Swift" code is by and large just fine. Of course discussions tend to spiral outwards, but stripping context and adding largely uninformed, quick judgements rarely adds value to any discussion.

I think Scala also does this

Swift just lets you use type inference liberally, which is nice, IMO. (Go ahead and opt-out if you disagree.)

The problem you describe is more with the design of SwiftUI, which makes massive, complex, extenuated use of the type system. The types quickly become too complex to reason about. It's a mess that hinders the developer instead of helping them.

Not what you're going for when you design an API. Apple really screwed this one up, though I doubt they will backtrack to fix it, so we're probably stuck with it.

C++ is hardly niche, nor I would say is the desire to interop with large, established languages with decades of existing code and libraries.

As someone who has worked in both Swift and C++ I’m grateful for those that choose to make an effort in this space. And of course, just because some do doesn’t take away from others that work on SwiftUI. Both can be true at the same time.

Agreed that C++ FFI is not niche, although it's also not really a core feature (C FFI suffixes for that).

Safe FFI with lifetime support is definitely lower priority though, given that FFI is usually in a thin layer of application code that can sort out safety itself.

> given that FFI is usually in a thin layer of application code that can sort out safety itself.

Apple is aiming far higher. They want [1] the ability to seamlessly replace C++ code anywhere with Swift code, to write subclasses of C++ classes in Swift, etc. without giving up performance.

See https://github.com/swiftlang/swift-evolution/blob/main/visio...

and the video linked to in https://forums.swift.org/t/video-swift-as-c-successor-in-fou..., which shows how far they were a year ago.

[1] they likely won’t get there for all C++ code, but if they get to “with a few manual annotations” without giving up efficiency, that would be a major accomplishment.

C++ interoperability is neither niche nor theoretical. I imagine it would allow them to rewrite bits of the existing compiler, written in C++, into Swift.

The type inference time explosion issue is a known and well-documented flaw. Fixing it would require an overhaul of the type system, which is possible, but I’m not sure if there is appetite for it.

Most of the time you just need to break up the view into smaller pieces. Eventually you realize you have a syntax error ;)

you think C++ interop is niche? lol

For regular Swift developers who make iOS apps, yes it's niche.

C++ interop is probably more important for Apple itself.

Swift compiler speed and error reporting are abysmal and improving them would have a much bigger impact. So far it's not getting any better at all.

If I could give up some level of type inferrence and get the build time from 3 minutes to 20 seconds, I would.

It’s not that niche at all if you have a cross platform app or you have a big part of your app that has to interface with metal kernel code, which is C++. There are a lot of reasons why you would want that code accessible from swift as well. And it’s also a big deal in games, communicating between swift and C++ is necessary unless your game only exists on iOS.

Sure, I like the C++ interop and agree that it's important. My point was that 99% of Xcode/Swift app developers will never touch it or even know it's a thing.

Well, in fairness I would probably take "error messages point to a file location" over "lifetime annotations for C++ FFI".

Apple wants to migrate away from C++, including code in the kernel, so they need a safe, low-overhead language.

Swift has originally been designed as an ObjC replacement, mostly for UI glue code, with conveniences more like a scripting language.

So now they're working on making Swift support more low-level low-overhead code.

https://youtu.be/lgivCGdmFrw

Swift was designed as C, Objective-C and C++ replacement from the start, and those statement have not yet been removed from its official documentation.

Naturally they couldn't tackle everything at once.

If that was the plan from the start I'd expect them to be more careful about dependences on reference counting and implicit allocations. It's more optimized than ObjC, but still has things like loops that iterate over a logical copy.

I take their old statements about replacing C and C++ as the same aspiration that Golang had to be a systems programming language.

Looks very similar to C#'s `ref struct` (https://learn.microsoft.com/en-us/dotnet/csharp/language-ref...).

I sometimes feel that my own lifetime is a type of non-escapable.

Haha. Now imagine cultures that have historically believed in eternal life.

My wild fantasy of a MacOS-native fork of UNetbootin (C++) has just become closer to reality.

Why is Swift interop needed for that? C/C++ are first class citizens in Apple's toolchains, and Obj-C++ interop has been a well supported thing for 25+ years.

I work like all day every day. This is just a fun project idea, so would prefer to save time with a dynamic language like Swift.

Reply on news.ycombinator.com