I write a lot of tools that depend on the TypeScript compiler API, and they run in a lot of a lot of JS environments including Node and the browser. The current CJS codebase is even a little tricky to load into standard JS module supporting environments like browsers, so I've been _really_ looking forward to what Jake and others have said will be an upcoming standard modules based version.
Is that still happening, and how will the native compiler be distributed for us tools authors? I presume WASM? Will the compiler API be compatible? Transforms, the AST, LanguageService, Program, SourceFile, Checker, etc.?
I'm quite concerned that the migration path for tools could be extremely difficult.
[edit] To add to this as I think about it: I maintain libraries that build on top of the TS API, and are then in turn used by other libraries that still access the TS APIs. Things like framework static analysis, then used by various linters, compilers, etc. Some linters are integrated with eslint via typescript-eslint. So the dependency chain is somewhat deep and wide.
Is the path forward going to be that just the TS compiler has a JS interop layer and the rest stays the same, or are all TS ecosystem tools going to have to port to Go to run well?
I've found that as of 2025, Go's WASM generator isn't as good as LLVM and it has been very difficult for me to even get parity with vanilla JS performance. There is supposedly a way to use a subset of go with llvm for faster wasm, but I haven't tried it (https://tinygo.org/).
I'm hoping that Microsoft might eventually use some of their wasm chops to improve GO's native wasm compiler. Their .NET wasm compiler is pretty darn good, especially if you enable AOT.
I'm hoping that a later version makes this possible.
https://github.com/golang/go/issues/63904#issuecomment-22536...
It sounds like it would be a great fit for e.g. Lua though.
item := *(*int)(unsafe.Pointer(uintptr(start) + size*uintptr(i)))
Usually when language reference books used to be printed, or we used ISO languages, what is there on paper, is the language.
We are only discussing semantics, if it is hardcoded primitives, or made available via the standard library, specially in case of blessed packages like unsafe which aren't fully implemented, rather magical types for the compiler.
Hence why the only thing you will see here is mostly documentation, https://github.com/golang/go/blob/master/src/unsafe/unsafe.g...
Which is nothing new since the 1960's that there are systems languages with some way to mark code unsafe, the C linage of languages are the ones that decided to ignore this approach.
I suppose that would be possible with fat-pointers that are reference+offset.
Well, for languages that use a GC. People who are writing WASM that exceeds JS in speed are typically doing it in Rust or C++.
The few cases that performed significantly better than the JS version (like >2x speed) were integer-heavy math and tail-call optimized recursive code, some cases were slower than the JS version.
What I was surprised was that the JS version had similar performance to the x64 version with -O3 in some of my benchmakrs (like float64 performance).
This was a while ago though when WASM support had just landed in browsers, so probably things got better now.
Or do you mean that there's a use case for a compilation in the browser?
If I got it correctly, they created a node native module that allows synchronous communication on standard I/O between external processes.
So, this node module will make possible the communication between the typescript compiler GO process, that will expose an “API server compiler”, and a client side JavaScript process.
They don’t think it will be possible to port all APIs and some/most of them will be different than today.
I have API use that falls into a few categories, that aren' just LSP-ish type cases:
- Transforms, which presumably there has to be some solution for, even if it's porting to Go.
- Linters, which integrate with typescript-eslint and need the type-checker.
- Codemods, which create and modify AST nodes and re-emit them.
- Static analyzers, which build us app-specific models of the code and rely on AST traversal and the type-checker.
- Analyzer libraries that offer tools to other libraries and apps that expose the TypeScript AST and functions that operate on AST nodes.
Traversing the AST over IPC is going to be too chatty, so I presume there will have to be some sort of way to get a whole SourceFile in one call, but then I wonder about traversal. You'll need a visitor library on your side of the IPC at least, but that's simple. But then you also need all the predicates. You don't want to be calling ts.isTemplateExpression() on every node via IPC.
And I do all this stuff in web workers too, so whatever this IPC is has to work there.
1. “We expect to have a more curated API that is informed by critical use-cases (e.g. linting, transforms, resolution behavior, language service embedding, etc.).”
2. “We also can imagine opportunities to optimize, use other underlying IPC strategies, and provide batch-style APIs to minimize call overhead.”
Anyway, I’ve used the compiler API a lot too, and I really enjoy its huge capabilities, making possible practically everything on the source code (EDIT: and hijack the build process too). I hope we won’t miss too much.
Why not something like Rust? Most of the JS ecosystem that is moving toward faster tools seem to be going straight to Rust (Rolldown, rspack (the webpack successor) SWC, OXC, Lightning CSS / Parcel etc) and one of the reasons given is it has really great language constructs for parsers and traversing ASTs (I think largely due to the existence of `match` but i'm not entirely sure)
Was any thought given to this? And if so what was the deciding factors for Go vs something like Rust or another language entirely?
____
Language choice is always a hot topic! We extensively evaluated many language options, both recently and in prior investigations. We also considered hybrid approaches where certain components could be written in a native language, while keeping core typechecking algorithms in JavaScript. We wrote multiple prototypes experimenting with different data representations in different languages, and did deep investigations into the approaches used by existing native TypeScript parsers like swc, oxc, and esbuild. To be clear, many languages would be suitable in a ground-up rewrite situation. Go did the best when considering multiple criteria that are particular to this situation, and it's worth explaining a few of them.
By far the most important aspect is that we need to keep the new codebase as compatible as possible, both in terms of semantics and in terms of code structure. We expect to maintain both codebases for quite some time going forward. Languages that allow for a structurally similar codebase offer a significant boon for anyone making code changes because we can easily port changes between the two codebases. In contrast, languages that require fundamental rethinking of memory management, mutation, data structuring, polymorphism, laziness, etc., might be a better fit for a ground-up rewrite, but we're undertaking this more as a port that maintains the existing behavior and critical optimizations we've built into the language. Idiomatic Go strongly resembles the existing coding patterns of the TypeScript codebase, which makes this porting effort much more tractable.
Go also offers excellent control of memory layout and allocation (both on an object and field level) without requiring that the entire codebase continually concern itself with memory management. While this implies a garbage collector, the downsides of a GC aren't particularly salient in our codebase. We don't have any strong latency constraints that would suffer from GC pauses/slowdowns. Batch compilations can effectively forego garbage collection entirely, since the process terminates at the end. In non-batch scenarios, most of our up-front allocations (ASTs, etc.) live for the entire life of the program, and we have strong domain information about when "logical" times to run the GC will be. Go's model therefore nets us a very big win in reducing codebase complexity, while paying very little actual runtime cost for garbage collection.
We also have an unusually large amount of graph processing, specifically traversing trees in both upward and downward walks involving polymorphic nodes. Go does an excellent job of making this ergonomic, especially in the context of needing to resemble the JavaScript version of the code.
Acknowledging some weak spots, Go's in-proc JS interop story is not as good as some of its alternatives. We have upcoming plans to mitigate this, and are committed to offering a performant and ergonomic JS API. We've been constrained in certain possible optimizations due to the current API model where consumers can access (or worse, modify) practically anything, and want to ensure that the new codebase keeps the door open for more freedom to change internal representations without having to worry about breaking all API users. Moving to a more intentional API design that also takes interop into account will let us move the ecosystem forward while still delivering these huge performance wins.
I don't think Go is a bad choice, though!
Cool. Can you tell us a bit more about the technical process of porting the TS code over to Go? Are you using any kind of automation or translation?
Personally, I've found Copilot to be surprisingly effective at translating Python code over to structurally similar Go code.
then when you find the correct version but you then have to install both the x86 and x64 version because the first one you installed doesn't work
yeh, great ecosystem
at least a Go binary runs 99.99999% of the time when you start it.
Is this the ultimate reason,Go is fast enough without being overally difficult. I'm humbly open to being wrong.
While I'm here, any reason Microsoft isn't sponsoring a solid open source game engine.
Even a bit of support for Godot's C#( help them get it working on web), would be great.
Even better would be a full C# engine with support for web assembly.
They did that. https://godotengine.org/article/introducing-csharp-godot/
At least some initial grant to get it started.
Getting C# working on web would be an amazing. It is already on the roadmap but some sponsorship would help tremendously for sure.
It is a huge company. They can do more than one thing. C#/.NET certainly isn't dead, but I'm not sure they really care if you do use it like they once did. It's there if you find it useful. If not, that's cool too.
I think Microsoft can find the money if they wanted to.
Personally, I would like them to never touch the game dev side of the market.
It is a powerful and robust language with great standard library, but you just cant be comfortable with it. All those boilerplate, all those sealed override virtual public protected or whatnot before each statement, those curly braces everywhere. You are always inside classes that are inside namespace, and even then you need to go deeper and have curly braces with properties and arrows in random places. Delegates and Events are ugly and unintuitive, two set of syntax for linq (and honestly for almost any somewhat new feature of the language), ref in out, you name it. It is hard to push something so inelegant.
Delegates and Events were a mistake, but that's a low-level .NET mistake that a lot of modern code can easily ignore, with Action<> and Func<> now reliably almost everywhere and WinForms easy to write off as "dead". (You can especially eliminate the need for the ugliness of Delegates and Events with System.Reactive.Linq.)
Records and Primary Constructors remove a ton of the boiler-plate of writing basic "DTOs" and/or dependency injection.
C# is pretty elegant, and a nicely evolving language. Microsoft isn't any longer trying to bet on C# as a "systems programming language" because too many people see JIT support and VMs as "not low level enough" (including apparently also Anders Hejlsberg), but that doesn't mean C# isn't "future proof".
People have been using Blazor WASM in Production for more than a year now. It's been stable since .NET 8.
I can see they do this in the future tbh, given how large their xbox gaming ecosystem, this path is very make sense since they can cut cost while giving option to their studios or indie developers
I would like MS to buy them out and FOSS the engine. Maybe if they split the ad business off into its own thing.
Unity feels like a bizarre almost abusing business relationship. They can change the terms of service at will.
The licensing is confusing. Billy is a freelancer. He makes a small game for his friend company. His friends company raises a funding round.
Depending on how much money is raised, Unity is going to call Billy up and extort him to upgrade to a higher license tier.
I don't particularly like Godot, but every few months I try and learn it again.
The game engine landscape is like picking the least worst option.
All that to be fair, Unity provided a high quality game engine for effectively nothing for over a decade to the vast majority of its users. It's time to pay the piper.
Also some of the low level improvements on C# have been done with collaboration with Unity team's requirements, regarding their Burst use cases.
C# and TypeScript are Hejlsberg's children; C# is such an obvious pick that there must have been a monster problem with it that they didn't think could ever be fixed.
C# has all that stuff that the FAQ mentions about Go while also having an obvious political benefit. I'd hope the creator of said language who also made the decision not to use it would have an interesting opinion on the topic! I really hope we find out the real story.
As a C# developer I don't want to be offended but, like, I thought we were friends? What did we do wrong???
Transcript: "But I will say that I think Go definitely is much more low-level. I'd say it's the lowest level language we can get to and still have automatic garbage collection. It's the most native-first language we can get to and still have automatic GC. In contrast, C# is sort of bytecode-first, if you will. There are some ahead-of-time compilation options available, but they're not on all platforms and don't really have a decade or more of hardening. They weren't engineered that way to begin with. I think Go also has a little more expressiveness when it comes to data structure layout, inline structs, and so forth."
For anyone who can't watch the video, he mentions a few things (summarizing briefly just the linked time code, it's worth a watch):
- Go being the lowest level language that still has garbage collection
- Inline structs and other data structure expressiveness features
- Existing JS code is in a C-like function+data structure style and not an OOP style, this is easier to translate directly to Go while C# would require OOPifying it.
It's a fascinating language, but it lacks a flagship product.
I feel the same way about Haxe. Someone created an amazing language, but it lacks a big enough community.
Realistically languages need 2 things for adoption. Momentum and ease of use. Rust has more momentum than ease, but arguably can solve problems higher level languages can't.
I'm half imagining a hackathon like format where teams are challenged to use niche languages. The foundations behind these languages can fund prizes.
And AFAIK Symmetry Investments is that dogfood startup.
Sure, AOT is not as mature in C# but is this reason enough to be a show stopper? It seems there're other reasons Anders don't want to address publicly. Maybe as simple reasons as "Go is 10 times easier to pick up than C#" and "language features don't matter when the project matters". Those would indeed hurt the image of C# and Anders obviously don't want that.
But I don't see it as big drama.
The side-by-sides that show how Go code is closer to the current TS code (visually) than C# would be are pretty compelling. He made it pretty clear they're "porting" not rewriting.
Wouldn't these things be useful if you are making an actual compiler, that would run TS? Since in this case, the runtime is JS, I don't think any of these things would get any usage, unless they are used in the existing transpiler.
That said, I must have misstated my opinion if it seems like I didn't think they have a good reason. This is Anders Hejlsberg. The guy is a genius; he definitely has a good reason. They just didn't say what it is in this blog post (but did elsewhere in a podcast video linked in the HN thread).
It obviously does because the larger open source world are huge users of Typescript. This isn't some business-only Excel / PowerBI type product.
To put it another way, I think a lot of people would get quite pissed if tsc was going to be rewritten in C# because of the obvious headaches that's going to cause to users. Go is pretty much the perfect option from a user's point of view - it generates self-contained statically linked binaries.
And there would be logistical problems. With go, you just need to distribute the executable, but with c#, you also need a .net runtime, and on any platform that isn't Windows that almost certainly isn't already installed. And even if it is, you have to worry if the runtime is sufficiently up to date.
If they used c# there is a chance the community might fork typescript, or switch to something else, and that might not be a gamble MS would want to take just to get more exposure for c#.
By who?
Nobody said anything about who likes what more, nor does that even matter in the context of the original claim that .NET doesn't have a good runtime outside of Windows.
What is this logic? "You worked on C# years ago so you must use C# for everything"?
"You must dictate C# to every team you lead forever, no matter what skills they have"?
"You must uphold a dogma that C# is the best language for everything, because you touched it last"?
Why aren't you using this logic to argue that they should use Delphi or TurboPascal because Anders Hejlsberg created those? Because there is no logic; the person who created hammers doesn't have to use hammers to solve every problem.
So it's not just that the lead architect of C# is involved in the TypeScript changes. It's also that this is under the same roof and the same sign hangs on the building outside for both languages.
If Ford made a car and powered it with a Chevy engine, wouldn't you be curious what was going on also?
If it's the latter, I think the pitch of TS remains the same — it's a better way of writing JS, not the best language for all contexts.
If the TS team is getting a 10x improvement moving from TS to Go, you might imagine you could save about 10x on your server cpu. Or that your backend would be 10x more responsive.
If you have dedicated team for front and back anyhow, is a 10x slow down really worth a shared codebase?
Maybe top ten behind MSSQL, Powershell, Excel Formulae, DAX etc.
I do love F#, but its compiler is a rusty set of monkey bars. It's somehow single pass, meaning the type checker will struggle if you don't reorder certain expressions - but also dog slow, especially for `inline` definitions (which work more like templates or hygienic macros than .net generics, and are far more powerful.) File order matters, bafflingly! Newer .net features like spans and ref structs are missing with no clear path to implementation. Doing moderately clever things can cause the compiler to throw weird, opaque, internal errors. F# is built around immutability but there's no integration with the modern .net immutable collections.
It's clearly languishing and being kept alive by a skeleton crew, which is sad, because it deserves better, but I've used research prototypes less clunky than what ought to be a flagship.
Huh? They're already implemented! It took years and they've still got some rough edges, yes, but they've been implemented for a few years now.
Agreed with the rest, though. As much as I love working with F#, I've jumped ship.
- I don't think the compiler is actually that bad, and yes - inline definitions I think once you are going on the "templating route" are going to be slower. Spans and ref structs are there - I think the design of them is more intuitive actually - the C# "ref struct" at first glance sounds like an oxymoron to me.
- modern .net immutable collections - in testing these are significantly slower than some of the F# options especially when you go away from the standard lib and use some of the other collection libraries. The algorithms within the C# immutable libs were not as optimal for some common collection types. They didn't feel modern last time I used them and I was forced to switch to the F# ones and/or others in the F# ecosystem to get the performance I needed. Immutable code felt MUCH more idiomatic with F#.
- "Doing moderately clever things can cause the compiler to throw weird, opaque, internal errors" - happened with init fields for me; can't recall another time.
Don't mind the file order bit - I thought OCAml and a few other languages also do this. Apps still scale OK, and when I was coding in it got me out of a few spaghetti code issues as the code scaled up to about the 500,000+ LOC mark.
However I do agree with you on it being kept alive by skeleton crew - I think the creators and tooling staff have moved on to the next big thing (AI and specifically Github Copilot). Which the way things are moving will raise some interesting questions about all coding languages in general potentially.
Anders Hejlsberg hasn't been the lead architect of C# for like 13 years. Mads Torgersen is:
https://dotnetcore.show/episode-104-c-sharp-with-mads-torger... - "I got hired by Microsoft 17 years ago to help work on C#. First, I worked with Anders Hejlsberg, who’s sort of the legendary creator and first lead designer of C#. And then when he and I had a little side project with others to do TypeScript, he stayed over there. And I got to take over as lead designer C#. So for the last, I don’t know, nearly a decade, that’s been my job at Microsoft to, to take care of the evolution of the C# programming language"
Years later, "why aren't you using YOUR LANGUAGE, huh? What's the matter, you don't like YOUR LANGUAGE?" is pushy and weird; he's a person with a job, not a religious cult leader.
> "If Ford made a car and powered it with a Chevy engine, wouldn't you be curious what was going on also?"
Like these? https://www.slashgear.com/1642034/fords-powered-by-non-ford-...
It's also not what anyone said.
> It's best not to use quotation marks to make it look like you're quoting someone when you're not. <https://news.ycombinator.com/item?id=21643562>
> "An indirect quote lets you capture or summarize what someone said or wrote without using their exact words. It helps to convey the tone or meaning of your source without quoting them directly." - https://www.grammarly.com/blog/punctuation-capitalization/qu...
I'm distilling and exaggerating multiple the comments to convey the tone and meaning of the bit I want to focus on. Asking "why not C#?" has the implicit framing "it should be C# by default and you have to justify why not" and calling out that bias to show it to be unreasonable is the intent.
Nope. None of those are even close to Ford + Chevy. (Ford + Mazda is well known of course).
I chose the analogy carefully.
It’s probably most common when an automaker introduces a new make and wants to save time and capital on developing and getting into production a new engine.
The Toyota matrix and Pontiac vibe used a lot of the same parts shared an engine and drivetrains if I'm not mistaken.
as you know full well, Delphi and Turbo Pascal don't have strong library ecosystems, don't have good support for non-Windows platforms, and don't have a large developer base to hire from, among other reasons. if Hejlsberg was asked why Delphi or Turbo Pascal weren't used, he might give one or more of those reasons. the question is why he didn't use C#, for which those reasons don't apply.
Go executables do not.
TSC is installed in too many places for that burden to be placed all of a sudden. It is the same reason why Java has had a complicated acceptance history too. It's fine in the places that it is pre-installed, but no where else.
Node/React/Typescript developers do not want to install .net all of a sudden. If you react that poorly, pretend they decided they decided to write it in Java and ask if you think Node/React/Typescript developers WANT to install Java.
C#: 945 kB Go: 2174 kB
Both are EXEs you just copy to the machine, no separate runtime needed, talks directly to the OS.
But in either case binary sizes are smaller and more scalable than what Go produces. The assumption that Go is good at compact binaries just does not replicate in reality. Obviously it’s nice to be able not touch it at all and opting into Rust for distributing native SDKs. Go is completely unfit for this as it has even more invasive VM interaction when you use it as dynamically linked library. NativeAOT is “just ok” at this and Go is “way more underwhelming than you think”.
I think if you're in domain of using SIMD, besides base RAM usage of 2-5MB you should not see drastic difference unless you have a lot of allocation traffic. But I guess Rust solved most of this, just wanted to note that specific memory and deployment requirements are usually solvable by changing build and GC settings.
SQL Server connections are one example where I do get .exe with the .pdb in the publish directory but the .exe won't run correctly without the "Microsoft.Data.SqlClient.SNI.dll" file.
Another example are any libraries that have "RequiresDynamicCodeAttribute" requirements.
Compare that to go. It's not even close. I see comments bickering about the size of executable files... Almost no major product cares about that within order of magnitude.
Go is a wild choice to write a compiler in. Literally in my top 10 things I never want to do. Everything else about it drove them to do it.
Memory management? Or a stricter type system?
"VS Code Go Language Extension Goes from Microsoft to Google"
https://visualstudiomagazine.com/articles/2020/06/10/go-goes...
People say this like it's a bad thing. It's not, it's Go's primary strength.
I think they missed out by not going with Rust. It seems like the social factors weighed out. Probably hard to quickly assemble a rust team within msft. Again though that makes Go a practical choice. I don't see why people are so confused by it. Go is a pretty widely used and solid choice to get things done reliably and quickly these days.
I'm not arguing saying they made a bad call. I think what they did was smart with the options in front of them and whatever budget they have. The world isn't good for idealism, but it ideally could have been written in rust in my opinion.
What exactly would that buy and would the outcome matter much?
In my opinion: pragmatism > idealism.
Golang is generally very fast and simple as well , the only problem is of memory allocation / garbage collector overhead but the benefits outweigh the loss.
Dude, just read the article being discussed, this is addressed so you can just stop making shit up.
The audacity of people like you to just keep adding speculations upon speculations on a subject without even bothering to learn what is being discussed.
Rust is really hard , compared to golang. This can increase outside contributors as well.
Golang is love , Golang is life.
They absolutely address this in the linked article, so why are we even speculating here?
> Probably hard to quickly assemble a rust team within msft.
The same MSFT that is rewriting their Windows OS in rust as we speak? I think you should stop commenting when you don't know anything about the subject.
I wrote a lot of Go code as well as Java. When people say things like this, I'm not quite sure what exactly they are referring to. No one is forcing you to write mutli-level deep inheritance hierarchies in Java/C#, and Go itself is OOP. Structural typing has its issues as well. Where does this supposed inherent productivity boost lie?
What do you see as the future for use cases where the typescript compiler is embedded in other projects? (Eg. Deno, Jupyter kernels, etc.)
There’s some talk of an inter process api, but vague hand waving here about technical details. What’s the vision?
In TS7 will you be able to embed the compiler? Or is that not supported?
So, how exactly is my app/whatever supposed to spin up a parallel process in the OS and then talk to it over IPC? How do you shut it down when the 'host' process dies?
Not vaguely. Not hand wave "just launch it". How exactly do you do it?
How do you do it in environments where that capability (spawning arbitrary processes) is limited? eg. mobile.
How do you package it so that you distribute it in parallel? Will it conflict with other applications that do the same thing?
When you look at, for example, a jupyter kernel, it is already a host process launched and managed by jupyter-lab or whatever, which talks via network chatter.
So now each kernel process has to manage another process, which it talks to via IPC?
...
Certainly, there are no obvious performance reasons to avoid IPC, but I think there are use cases where having the compiler embedded makes more sense.
Usually the very easiest way to do this is to launch the target as a subprocess and communicate over stdin/stdout. (Obviously, you can also negotiate things like shared memory buffers once you have a communication channel, but stdin/stdout is enough for a lot of stuff.)
> How do you shut it down when the 'host' process dies?
From the perspective of the parent process, you can go through some extra work to guarantee this if you want; every operating system has facilities for it. For example, in Linux, you can make use of PR_SET_PDEATHSIG. Actually using that facility properly is a bit trickier, but it does work.
However, since the child process, in this case, is aware that it is a child process, the best way to go about it would be to handle it cooperatively. If you're communicating over stdin/stdout, the child process's stdin will close when the parent process dies. This is portable across Windows and UNIX-likes. The child process can then exit.
> How do you do it in environments where that capability (spawning arbitrary processes) is limited? eg. mobile.
On Android, there is nothing special to do here as far as I know. You should be able to bundle and spawn a native process just fine. Go binaries are no exception.
On iOS, it is true that apps are not allowed to spawn child processes, as far as I am aware. On iOS you'd need a different strategy. If you still want a native code approach, though, it's more than doable. Since you're on iOS, you'll have some native code somewhere. You can compile Go code into a Clang-compatible static library archive, using -buildmode=c-archive. There's a bit more nuance to it to get something that will link properly in iOS, but it is supported by Go itself (Go supports iOS and Android in the toolchain and via gomobile.) Once you have something that can be linked into the process space, the old IPC approach would continue to work, with the semantic caveat that it's not technically interprocess anymore. This approach can also be used in any other situation you're doing native code, so as long as you can link C libraries.
If you're in an even more restrictive situation, like, I dunno, Cloudflare Pages Functions, you can use a WASM bundle. It comes at a performance hit, but given that the Go port of the TypeScript compiler is already roughly 3.5x faster than the TypeScript implementation, it probably will not be a huge issue compared to today's performance.
> How do you package it so that you distribute it in parallel? Will it conflict with other applications that do the same thing?
There are no particular complexities with distributing Go binaries. You need to ship a binary for each architecture and OS combination you want to support, but Go has relatively straight-forward cross-compiling, so this is usually very easy to do. (Rather unusually, it is even capable of cross-compiling to macOS and iOS from non-Apple platforms. Though I bet Zig can do this, too.) You just include the binary into your build. If you are using some bindings, I would expect the bindings to take care of this by default, making your resulting binaries "just work" as needed.
It will not conflict with other applications that do the same thing.
> When you look at, for example, a jupyter kernel, it is already a host process launched and managed by jupyter-lab or whatever, which talks via network chatter.
> So now each kernel process has to manage another process, which it talks to via IPC?
Yes, that's right: you would have to have another process for each existing process that needs its own compiler instance, if going with the IPC approach. However, unless we're talking about an obscene number of processes, this is probably not going to be much of an issue. If anything, keeping it out-of-process might help improve matters if it's currently doing things synchronously that could be asynchronous.
Of course, even though this isn't really much of an issue, you could still avoid it by going with another approach if it really was a huge problem. For example, assuming the respective Jupyter kernel already needs Node.JS in-process somehow, you could just as well have a version of tsc compiled into a Node-API module, and do everything in-process.
> Certainly, there are no obvious performance reasons to avoid IPC, but I think there are use cases where having the compiler embedded makes more sense.
Except for browsers and edge runtimes, it should be possible to make an embedded version of the compiler if it is necessary. I'm not sure if the TypeScript team will maintain such a version on their own, it remains to be seen exactly what approach they take for IPC.
I'm not a TypeScript Compiler developer, but I hope these answers are helpful in some way anyways.
> It will not conflict with other applications that do the same thing.
It is possible not to conflict with existing parallel deployments, but depending on your IPC mechanism, it is by no means assured when you're not forking and are instead launching an external process.
For example, it could by default bind a specific default port. This would work in the 'naive' situation where the client doesn't specify a port and no parallel instances are running. ...but if two instances are running, they'll both try to use the same port. Arbitrary applications can connect to the same port. Maybe you want to share a single compiler service instance between client apps in some cases?
Not conflicting is not a property of parallel binary deployment and communication via IPC by default.
IPC is, by definition intended to be accessible by other processes.
Jupyter kernels for example are launched with a specified port and a secret by cli argument if I recall correctly.
However, you'd have to rely on that mechanism being built into the typescript compiler service.
...ie. it's a bit complicated right?
Worth it for the speedup? I mean, sure. Obviously there is a reason people don't embed postgres. ...but they don't try to ship a copy of it along side their apps either (usually).
I fail to see how starting another process under an OS like Linux or Windows can be conflicting. Don't share resources, and you're conflict-free.
> IPC is, by definition intended to be accessible by other processes
Yes, but you can limit the visibility of the IPC channel to a specific process, in the form of stdin/stdout pipe between processes, which is not shared by any other processes. This is enough of a channel to coordinate creation of a more efficient channel, e.g. a shmem region for high-bandwidth communication, or a Unix domain socket (under Linux, you can open a UDS completely outside of the filesystem tree), etc.
A Unix shell is a thing that spawns and communicates with running processes all day long, and I'm yet to hear about any conflicts arising from its normal use.
You can get a conflicting resource in a shell by typing 'npm start' twice in two different shells, and it'll fail with 'port in use'.
My point is that you can do not conflicting IPC, but by default IPC is conflicting because it is intended to be.
You cannot bind the same port, semaphore, whatever if someone else is using it. That's the definition of having addressable IPC.
I don't think arguing otherwise is defensible or reasonable.
Having a concern that a network service might bind the same port as an other copy of the same network service deployed on the same target by another host is an entirely reasonable concern.
I think we're getting off into the woods here with an arbitrary 'die on this hill' point about semantics which I really don't care about.
TLDR: If you ship an IPC binary, you have to pay attention to these concerns. Pretending otherwise means you're not doing it properly.
It's not an idle concern; it's a real concern that real actual application developers have to worry about, in real world situations.
I've had to worry about it.
I think it's not unfair to think it's going to be more problematic than the current, very easy, embedded story, and it is a concern that simply does not exist when you embed a library instead of communicating using IPC.
Sure, some IPC approaches can run into issues, such as using TCP connections over loopback. However, I'm describing an approach that should never conflict since the resources that are shared are inherited directly, and since the binary would be embedded in your application bundle and not shared with other programs on the system. A similar example would be language servers which often work this way: no need to worry about conflicts between different instances of language servers, different language servers, instances of different versions of the same language server, etc.
There's also some precedent for this approach since as far as I understand it, it's also what the Go-based ESBuild tool does[1], also popular in the Node.JS ecosystem (it is used by Vite.)
> For example, it could by default bind a specific default port. This would work in the 'naive' situation where the client doesn't specify a port and no parallel instances are running. ...but if two instances are running, they'll both try to use the same port. Arbitrary applications can connect to the same port. Maybe you want to share a single compiler service instance between client apps in some cases?
> Not conflicting is not a property of parallel binary deployment and communication via IPC by default.
> IPC is, by definition intended to be accessible by other processes.
Yes, although the set of processes which the IPC mechanism is designed to be accessible by can be bound to just one process, and there are cross-platform mechanisms to achieve this on popular desktop OSes. I can not speak for why one would choose TCP over stdin/stdout, but, I don't expect that tsc will pick a method of IPC that is flawed in this way, since it would not follow precedent anyway. (e.g. tsserver already uses stdio[2].)
> Jupyter kernels for example are launched with a specified port and a secret by cli argument if I recall correctly.
> However, you'd have to rely on that mechanism being built into the typescript compiler service.
> ...ie. it's a bit complicated right?
> Worth it for the speedup? I mean, sure. Obviously there is a reason people don't embed postgres. ...but they don't try to ship a copy of it along side their apps either (usually).
Well, I wouldn't honestly go as far as to say it's complicated. There's a ton of precedent for how to solve this issue without any conflict. I can not speak to why Jupyter kernels use TCP for IPC instead of stdio, I'm very sure they have reasons why it makes more sense in their case. For example, in some use cases it could be faster or perhaps just simpler to have multiple channels of communication, and doing this with multiple pipes to a subprocess is a little more complicated and less portable than stdio. Same for shared memory: You can always have a protocol to negotiate shared memory across some serial IPC mechanism, but you'll almost always need a couple different shared memory backends, and it adds some complexity. So that's one potential reason.
(edit: Another potential reason to use TCP sockets is, of course, if your "IPC" is going across the network sometimes. Maybe this is of interest for Jupyter, I don't know!)
That said, in this case, I think it's a non-issue. ESBuild and tsserver demonstrate sufficiently that communication over stdio is sufficient for these kinds of use cases.
And of course, even if the Jupyter kernel itself has to speak the TCP IPC protocols used by Jupyter, it can still subprocess a theoretical tsc and use stdio-based IPC. Not much complexity to speak of.
Also, unrelated, but it's funny you should say that about postgres, because actually there have been several different projects that deliver an "embeddable" subset of postgres. Of course, the reasoning for why you would not necessarily want to embed a database engine are quite a lot different from this, since in this case IPC is merely an implementation detail whereas in the database case the network protocol and centralized servers are essentially the entire point of the whole thing.
[1]: https://github.com/evanw/esbuild/blob/main/cmd/esbuild/stdio...
[2]: https://github.com/microsoft/TypeScript/wiki/Standalone-Serv...
With a TSC in Go, it's no longer true. Previously you only had to figure out how to run JS, now you have to figure out both how to manage a native process _and_ run the JS output.
This obviously matters less for situations where you have a clear separation between the build stage and runtime stage. Most people complaining here seem to be talking about environments were compilation is tightly integrated with the execution of the compiled JS.
Porting to Go was the right decision, but part of me would've liked to see a different approach to solve the performance issue. Here I'm not thinking about the practicality, but simply about how cool it would've been if performance had instead been improved via:
- porting to OCaml. I contributed to Flow once upon a time, and a version of TypeScript in OCaml would've been huge in unifying the efforts here.
- porting to Rust. Having "official" TypeScript crates in rust would be huge for the Rust javascript-tooling ecosystem.
- a new runtime (or compiler!). I'm thinking here an optional, stricter version of TypeScript that forbids all the dynamic behaviours that make JavaScript hard to optimize. I'm also imagining an interpreter or compiler that can then use this stricter TypeScript to run faster or produce an efficient native binary, skipping JavaScript altogether and using types for optimization.
This last option would've been especially exciting since it is my opinion that Flow was hindered by the lack of dogfooding, at least when I was somewhat involved with the project. I hope this doesn't happen in the TypeScript project.
None of these are questions, just wanted to share these fanciful perspectives. I do agree Go sounds like the right choice, and and in any case I'm excited about the improvement in performance and memory usage. It really is the biggest gripe I have with TypeScript right now!
Rust and OCaml are _maybe_ prettier to look at, but for the average TypeScript developer Go is a much more understandable target IMO.
Lifetimes and ownership are not trivial topics to grasp, and they add overhead (as discussed here: https://github.com/microsoft/typescript-go/discussions/411) that not all contributors might grasp immediately.
(FWIW, It must have been a very well thought out rationale.)
Edit: watched the revenant clip from the GH discussion- makes sense. Maybe push NativeAoT to be as good?
I am (positively) surprised Hejlsberg has not used this opportunity to push C#: a rarity in the software world where people never let go of their darlings. :)
And lightly edited transcript here: https://github.com/microsoft/typescript-go/discussions/411#d...
- C++ with thousands of tiny objects and virtual function calls? - JavaScript where data is stored in large Int32Array and does operations on it like a VM?
If you know anything about how JavaScript works, you know there is a lot of costly and challenging resource management.
In a game engine, you probably aren't recreating every game object from frame to frame. But in a compiler, you're creating new objects for every file you parse. That's a huge amount of work for the GC.
Basically I'd be interested to know what the bottlenecks in tsc are, whether there's much low-hanging fruit, and if not why not.
So this might be a very different performance profile.
*edit* I had initially written "single-pass", but in the context of a compiler, that's ambiguous.
In a situation like a game engine I think 1.5x is reasonable, but TS has a huge amount of polymorphic data reading that defeats a lot of the optimizations in JS engines that get you to monomorphic property access speeds. If JS engines were better at monomorphizing access to common subtypes across different map shapes maybe it'd be closer, but no engine has implemented that or seems to have much appetite for doing so.
I really wish JS VMs would invest in this. The DOM is full of large inheritance hierarchies, with lots of subtypes, so a lot of DOM code is megamorphic. You can do tricks like tearing off methods from Element to use as functions, instead of virtual methods as usual, but that quite a pain.
Also for command-line tools, the JIT warmup time can be pretty significant, adding a lot to overall command-to-result latency (and in some cases even wiping out the JIT performance entirely!)
A few things mentioned in an interview:
Cannot build native binaries from TypeScript
Cannot as easily take advantage of concurrency in TypeScript
Writing fast TypeScript requires you to write things in a way that isn't 'normal' idiomatic TypeScript. Easier to onboard new people onto a more idiomatic codebase.
None of these things say "this is a good way to build a large compiler suite that we're building for performance".
JS is 10x-100x slower than native languages (C++, Go, Rust, etc) if you write the code normally (i.e. don't go down the road of uglifying your JS code to the point where it's dramatically less pleasant to work with than the C++ code you're comparing to).
It's kind of annoying how even someone like Hejlsberg is throwing around words like "native" in such an ambiguous, sloppy, and prone-to-be-misleading way on a project like this.
"C++" isn't native. The object code that it gets compiled to, large parts of which are in the machine's native language, is.
Likewise "TypeScript" isn't non-native in any way that doesn't apply to any other language. The fact that tsc emits JS instead of the machine's native language is what makes TypeScript programs (like tsc itself) comparatively slow.
It's the compilers that are the important here, not the languages. (The fact that the TypeScript team was committed to making sure the typescript-go compiler is the same (nearly line-for-line equivalent) to the production version of the TypeScript compiler written in itself really highlights this.)
(disclaimer: I am a biased Go fan)
Just going from ESLint to Biome is more than a 10x improvement... it's not just 1.5x because it's not just the runtime logic at play for build tools.
In JavaScript, you can't even put 8M keys in a Hashmap; inserts take > 1 second per element:
Cue rust devotees in 3, 2, ..
If you are a rust devotee, you can use https://github.com/FractalFir/rustc_codegen_clr to compile your rust code to the same .NET runtime as C#. The project is still in the works but support is said to be about 95% complete.
He clearly knows all this so the obvious inference is that the decision isn't really about features. The most likely problem is a lack of confidence in the .NET team, or some political problems/bad blood inside Microsoft. Perhaps he's tried to use it and been frustrated by bugs; the comment about "battle hardened" feel like where the actual rationale is hiding. We're not getting the full story here, that's clear enough.
I'm honestly surprised Microsoft's policies allowed this. Normally companies have rules that require dogfooding for exactly this reason. Such a project is not terribly urgent, it has political heft within Microsoft. They could presumably have got the .NET team to fix bugs or make optimizations they need, at least a lot easier than getting the Go team to do it. Yet they chose not to. Who would have any confidence in adoption of .NET for performance sensitive programs now? Even the father of .NET doesn't want to use it. Anyone who wants to challenge a decision to adopt it can just point at Microsoft's own actions as evidence.
It is especially jarring given that they are a first-party customer who would have no trouble in getting necessary platforms supported or projects expedited (like NativeAOT-LLVM-WASM) in .NET. And the statements of Anders Hejlsberg himself which contradict the facts about .NET as a platform make this even more unfortunate.
There's an interesting contrast here with Java, where javac was ported to Java from C++ very early on in its lifecycle. And the Java AOT compiler (native image) is not only fully written in Java itself, everything from optimizations to code generation, but even the embedded runtime is written in Java too. Whereas in the .NET world Roslyn took quite a long time to come along, it wasn't until .NET 6, and of course MS rejected it from Windows more or less entirely for the same sorts of rationales as what Anders provides here.
It was introduced back then with .NET Framework 4.6 (C# 6) - a loong time ago (July 2015). The OSS .NET has started with Roslyn from the very beginning.
> And the Java AOT compiler (native image) is not only fully written in Java itself, everything from optimizations to code generation, but even the embedded runtime is written in Java too.
NativeAOT uses the same architecture. There is no C++ besides GC and pre-existing compiler back-end (both ILC and RyuJIT drive it during compilation process). Much like GraalVM's Native Image, the VM/host, type system facilities, virtual/interface dispatch and everything else it could possibly need is implemented in C# including the linker (reachability analysis/trimming, kind of like jlink) and optimizations (exact devirtualization, cctor interpreter, etc.).
In the end, it is the TypeScript team members who worked on this port, not Anders Hejlsberg himself, which is my understanding. So we need to take this into account when judging what is being communicated.
no? https://github.com/microsoft/typescript-go/graphs/contributo...
First he mentions the no classes thing. It is hard to see how that would matter even for automated porting, because like you said, he could just use static classes, and even do a static using statement on the calling side.
Another one of his reasons was that Go was good at processing complex graphs, but it is hard to imagine how Go would be better at that than C#. What language feature that Go has, but C# does not supports that? I don't think anyone will be able to demonstrate one. This distinction makes sense for Go vs Rust, but not for Go vs C#.
As for the platform / AOT argument, I don't know as much about that, but I thought it was supposed to be possible now. If it isn't, it seems like it would be better for Microsoft to beef that up than to allow a vote of no confidence to be cast like this.
The question comes up and he quickly glosses over it, but by the sound of it he isn't impressed with the performance or support of AOT compiled C# on all targeted platforms.
- C# Ahead of Time compiler doesn't target all the platforms they want.
- C# Ahead of Time compiler hasn't been stressed in production as many years as Go.
- The core TypeScript compiler doesn't use any classes; Go is functions and datastructures whereas C# is heavily OOP, so they would have to switch paradigms to use C#.
- Go has better control of low level memory layouts.
- Go was ultimately the path of least resistance.
[19:14] why not C#?
Dimitri: Was C# considered?
Anders: It was, but I will say that I think Go definitely is -- it's, I'd say, the lowest-level language we can get to and still have automatic garbage collection. It's the most native-first language we can get to and still have automatic GC. In C#, it's sort of bytecode first, if you will; there is some ahead-of-time compilation available, but it's not on all platforms and it doesn't have a decade or more of hardening. It was not geared that way to begin with. Additionally, I think Go has a little more expressiveness when it comes to data structure layout, inline structs, and so forth. For us, one additional thing is that our JavaScript codebase is written in a highly functional style -- we use very few classes; in fact, the core compiler doesn't use classes at all -- and that is actually a characteristic of Go as well. Go is based on functions and data structures, whereas C# is heavily OOP-oriented, and we would have had to switch to an OOP paradigm to move to C#. That transition would have involved more friction than switching to Go. Ultimately, that was the path of least resistance for us.
Dimitri: Great -- I mean, I have questions about that. I've struggled in the past a lot with Go in functional programming, but I'm glad to hear you say that those aren't struggles for you. That was one of my questions.
Anders: When I say functional programming here, I mean sort of functional in the plain sense that we're dealing with functions and data structures as opposed to objects. I'm not talking about pattern matching, higher-kinded types, and monads.
[12:34] why not Rust?
Anders: When you have a product that has been in use for more than a decade, with millions of programmers and, God knows how many millions of lines of code out there, you are going to be faced with the longest tail of incompatibilities you could imagine. So, from the get-go, we knew that the only way this was going to be meaningful was if we ported the existing code base. The existing code base makes certain assumptions -- specifically, it assumes that there is automatic garbage collection -- and that pretty much limited our choices. That heavily ruled out Rust. I mean, in Rust you have memory management, but it's not automatic; you can get reference counting or whatever you could, but then, in addition to that, there's the borrow checker and the rather stringent constraints it puts on you around ownership of data structures. In particular, it effectively outlaws cyclic data structures, and all of our data structures are heavily cyclic.
(https://www.reddit.com/r/golang/comments/1j8shzb/microsoft_r...)
They could have used static classes in C#.
For a daemon like an LSP I reckon C# would've worked.
I doubt the ability to cross-compile TSC would have been a major factor. These artifacts are always produced on dedicated platforms via separate build stages before publishing and sign-off. Indeed, Go is better at native cross-compilation where-as .NET NativeAOT can do only do cross-arch and limited cross-OS by tapping into Zig toolchain.
I am sure it is good enough that the team decided to choose Go either way OR it is not important for this project.
> I doubt the ability to cross-compile TSC would have been a major factor.
I never said it was a major factor (I even said "I don't think it would be a deal breaker"), but it is a factor nonetheless. It definitely helps a lot during cross-platform debugging since you don't need to setup a whole toolchain just to test a bug in another platform, instead you can simple build a binary on your development machine and send it to the other machine.
But the only reason I asked this is because I was curious really, no need to be so defensive.
The gc compiler considers it a first-class build target.
Like C#, the binary tends to be on the larger side[1], which makes it less than ideal for driving a marketing website, but that's not really a problem here. Installation of tools before use is already the norm.
[1] The tinygo compiler can produce very small WASM binaries, comparable to C, albeit with some caveats (surprisingly, the extra data gc produces isn't just noise).
Where do you think Go gets those chubby static linked executables from?
That people have to apply UPX on top.
Graph for the differences in Runtime, Runtime Trimmed, and AOT .NET.
https://learn.microsoft.com/en-us/aspnet/core/fundamentals/n...
https://github.com/microsoft/typescript-go/discussions/411#d...
This is a big concern to me. Could you expand on what work is left to do for the native implementation of gsc? In particular, can you make an argument why that last bit of work won't reduce these 10x figures we're seeing? I'm worried the marketing got ahead of the engineering
Are there any insights on the platform decision?
- There is an esbuild process running in the background.
- If I look at the JavaScript returned to the browser, it is transpiled without any types present.
So even though the URLs in Vite dev mode look like they're pointing to "raw" TypeScript files, they're actually transpiled JavaScript, just not bundled.
I could be incorrect, of course, but it sure seems to me like Vite is using ESBuild on the Node.JS side and not tsc on the web browser side.
One thing I'm curious about: What about updating the original Typescript-based compiler to target WASM and/or native code, without needing to run in a Javascript VM?
Was that considered? What would (at a high level) the obstacles be to achieving similar performance to Golang?
Edit: Clarified to show that I indicate updating the original compiler.
JavaScript, like other dynamic languages, runs well with a JIT because the runtime can optimize for hotspots and common patterns (e.g. this method's first argument is generally an object with this shape, so write a fast path for that case). In theory you could write an AOT compiler for TypeScript that made some of those inferences at compile time based on type definitions, but
(a) nobody's done that
(b) it still wouldn't be as fast as native, or much faster than JIT
(c) it would be limited - any optimizations would die as soon as you used an inherently dynamic method like JSON.parse()
That is a good question.
One of the nice advantages of js is that it can run so many places. Will TypeScript still be able to enjoy that legacy going forward, or is native only what we should expect in 7+?
Will the questions and answers be posted anywhere outside of Discord after it's concluded?
Maybe I'm unique, but I prefer to not have the compiler tied up with a paryicular dependency-management/library system.
C and Rust both have predictable memory behaviour, Go does not.
(IE, as opposed to reference counting, where if you have cyclic loops, you need to manually go in and "break" the loop so memory gets reclaimed.)
It's actually pretty easy to do something like this with C, just using something like an arena allocator, or honestly, leaking memory. I actually wrote a little allocator yesterday that just dumps memory into a linkedlist, it's not very complicated: http://github.com/danieltuveson/dsalloc/
You allocate wherever you want, and when you're done with the big messy memory graph, you throw it all out at once.
There are obviously a lot of other reasons to choose go over C, though (easier to learn, nicer tooling, memory safety, etc).
Really interesting news, and uniquely dismaying to me as someone who is fighting tooth and claw to keep JS language tooling in the JS ecosystem.
My question has to do with Ryan's statement:
> We also considered hybrid approaches where certain components could be written in a native language, while keeping core typechecking algorithms in JavaScript
I've experimented deeply in this area (maybe 15k hours invested in BABLR so far) and what I've found is that it's richly rewarding. Javascript is fast enough for what is needed, and its ability to cache on immutable data can make it lightning fast not through doing more work faster, but by making it possible to do less work. In other words, change the complexity class not the constant factor.
Is this a direction you investigated? What made you decide to try to move sideways instead of forwards?
Have you considered the man-years and energy you're making everyone waste? Just as an example, I wonder what the carbon footprint of ESLint has been over the years...
Now, it pales in comparison to Python, but still...
TS currently wastes tons of resources (most especially peoples' time) by not being able to share its data and infrastructure with other tools and ecosystems, but while there would be much bigger wins from tackling the systemic problem, you wouldn't be able to say something as glib as "TS is 10x faster". Only the work that can be distilled to a metric is done now, because that's how to get a promotion when you work for a company like Microsoft
Thank you Typescript team for chasing those promotions!
From a performance perspective, I'd expect C++ and Rust to be much easier targets too, since I've seen quite a few industrial Go services be rewritten in C++/Rust after they fail to meet runtime performance / operability targets.
Wasn't there a recent study from Google that came to the same conclusion? (They see improved productivity for Go with junior programmers that don't understand static typing, but then they can never actually stabilize the resulting codebase.)