Hacker News Reader

Comment by hardwaregeek | Hacker News Reader

This reminds me of when people try to make a new Ruby implementation. They get the basics down and think "this isn't so bad! I can do this!". Fast forward a few months and they're stuck on some nasty little edge case around eval and method_missing that they need to run Rails. Or some weird aspect of C extensions.

TypeScript's compiler is an unholy, unsound mess, because JavaScript is an unholy, unsound mess. Sure the basics of type checking structs and variables is somewhat easy. But what about conditional types? What about weird generic constraints? What about indexing types? There's some really confusing stuff going on underneath the hood.

It's also a mess that *does not have a spec*. Even if you somehow manage to make a type checker that implements most of the features, there's no way to ensure it has 1 to 1 parity with TypeScript. Plus that parity can be broken at any point.

The only way I could see this happening is if the author somehow convinced Anders Hejlsberg or a significant portion of the TypeScript team to work on this new implementation. I'm not sure that's gonna happen.

View on HN

I am absolutely blown away at how good TypeScript's type checker is, given how much of an unholy mess JavaScript is, and that the TypeScript language cannot just deprecate all the awful parts.

More than once I've yelled at my screen, "HOW DO YOU KNOW THIS?!" about some inferred or narrowed type.

However, more than once I've also yelled "HOW DO YOU NOT KNOW THIS?!" at something or other :)

Just today I was triggered that this compiles.

declare foo: never[] // it can only be an empty array

foo[1] // compiles

TypeScript is a nice type system, it's surprisingly expressive and flexible, like e.g. in Haskell I was surprised that there's this strange notion that for a given type you can only have one `Eq` or `Ord` instance for some type A.

But in TypeScript you can have as many Ord interfaces as you want. I may want to order some `User` type by last login, first login or whatever I want, but in Haskell you need to overcomplicate it and create a newtype.

> you can only have one `Eq` or `Ord` instance for some type A.

This is a good thing - it stops you from using one Ord instance when putting As into a Map or Set, and then trying to pull them back out using another. Newtypes let you provide "additional" instances in a safe way.

Why shouldn't I be able to use as many Ord instances as I want?

I'll give you a trivial example I have some data type representing a TvShow, why should there be just one way to sort a collection of TvShow? Maybe I want to sort by year, maybe by length, maybe by category.

It's trivial to do in fp-ts, you can have as many Ord instances as you wish.

They just said why it's important to explicitly track which instance is in effect and newtypes do that.

Ord is for the natural (obvious and hopefully uncontroversial) ordering of a given type.

If your type doesn't have a natural ordering (as your TvShow doesn't) then you would use sortBy to supply an arbitrary compare function (just like what you would define in your Ord instance).

the argument is that usually you do not want that, it is a Haskell typeclasses vs ocaml modules discussion

You seem to be a bit confused because indeed there's no such thing as "natural, uncontroversial, obvious ordering".

Ordering (and more specifically a total ordering) in mathematics is a set and a binary relation that has the transitive, irreflexive and connected properties.

Not even natural numbers have "uncontroversial natural ordering", as I guess even you can think of at least two different binary relations (<= and >=) that form two different orderings for N.

There's a lot of orderings more, here's one that sorts first odd and even numbers: "0 < 2 < 4 < 6 < ... < 1 < 3 < 5 < 7 < ...". And there's others like this.

https://en.wikipedia.org/wiki/Gray_code

Natural numbers in fact have (or better: form) an aleph one types of orderings. None is "special, uncontroversial or natural" just because we're used to think about the default <=.

The concept of ordering requires two things: the data type and the binary relation. It's a pair of things we can represent as (X, compare) but which compare you choose isn't implicit, natural or magical.

When talking about Haskell, it has the concept of type classes, where types and their behavior are bundled together. That's a design choice of Haskell which has pros and cons, but there's no mathematical foundation for a data type to have one preferred ordering.

Ocaml, Scala and most pure fp libraries I know do not have the limitations of Haskell when it comes to the concepts of ordering and equality, in fact in most of those when asking for an ordering ask you for a data type and a compare function. Exactly what the mathematical definition requires you to.

Not a typeclass where those are bundled together for some design decision.

Hope I clarified you that there's no such thing as "uncontroversial natural obvious" ordering, because there's no such things in mathematics.

I wouldn't expect never[] to mean 0 length array, tbf. It's a variable length array of _type_ never. It should be something akin to:

    declare foo: never[0];

I think the correct solution is:

    declare foo: []; // fixed-length array (tuple) without any items

JS is a strict language, so an array containing one or more bottoms is a bottom. The only legal value is the empty array, at least in theory.

foo[1] is undefined (literally the undefined symbol), in JavaScript index-out-of-bounds is ok.

Although i don’t know if TypeScript asserts that x[i] is T | undefined if x is T[] and the length is non-trivial. In strict mode this would be really useful as you can just do x[i]!, but it requires you to think a bit about index-out-of-bounds or at least throw on the undefined case.

> i don’t know if TypeScript asserts that x[i] is T | undefined if x is T[] and the length is non-trivial

There's a flag for that: https://www.typescriptlang.org/tsconfig#noUncheckedIndexedAc...

I believe the type you're looking for is `[]` (a tuple with no elements) instead of `never[]`.

https://www.typescriptlang.org/play?#code/CYUwxgNghgTiAEYD2A...

Yes I know, but `never[]` should also be unaccessible by index since it can never have any value.

variable length arrays are accessible at all indexes,

const a = foo[1]

should result in a bring of type `never|undefined` or `undefined` depending on flow analysis

Just a note that TypeScripts type checking & compilation are two separate systems. It's never a compilation error because it always compiles.

In TS this is controllable. If noEmitOnError is true, then any error (including type errors) will stop it emitting JS output.

> But in TypeScript you can have as many Ord interfaces as you want. I may want to order some `User` type by last login, first login or whatever I want, but in Haskell you need to overcomplicate it and create a newtype.

I don't think that's accurate. Typescript only has one instance of Ord per type and it's inconveniently implemented through the method toString.

To usefully sort anything other than strings in Typescript, you are using a function where you provide your own comparator at runtime. That's more analogous to Haskell's `sortBy :: (a -> a -> Ordering) -> [a] -> [a]` or `sortOn :: Ord b => (a -> b) -> [a] -> [a]` which don't put any type bounds on the type you're sorting.

I've managed to get the compiler to typecheck

  const x: Thing = exprA;
  const y: Thing = exprB;
  const z = [x, y];

but somehow not:

  const z: Thing[] = [exprA, exprB];

although exprA and exprB were pretty nasty typing wise.

> but somehow not: const z: Thing[] = [exprA, exprB];

Doesn't

    const z = [exprA, exprB] as const;
    type GnarlyThingTuple = typeof z;

or something like this help?

Where you trying to create a tuple? Does `const z: [Thing, Thing] = [exprA, exprB];` work?

I'm the opposite. I was continually perplexed by trivial stuff that TypeScript just couldn't figure put. Maybe that has changed in the last year but I doubt it.

Do you have examples?

In my 5+ years introducing TypeScript everywhere that my team has to use JavaScript, this is almost always because JavaScript is an insane language that allows many counterintuitive things.

TS is sometimes weird because it is a superset of JS and has to be compatible.

> This reminds me of when people try to make a new Ruby implementation. They get the basics down and think "this isn't so bad! I can do this!". Fast forward a few months and they're stuck on some nasty little edge case around eval and method_missing that they need to run Rails. Or some weird aspect of C extensions.

I think that's exactly why the author wrote this:

> Eventually, I realized that rewriting a massive project such as tsc would be extremely difficult to continue. But the performance gains were exciting. This led me to try another route: porting instead of a complete rewrite. So, I started looking at the TypeScript source code.

It will require ongoing work to keep up with changes to upstream, but by matching upstream's structure as closely as possible (given the language difference), the path to supporting these nasty little edge cases should be relatively clear.

Interesting, in theory you could make a rough js to golang compiler given the relatively strict subset of typescript that I recall the typescript compiler using (not that the strictness means a fully sound implementation is easy, but that you can use the regularity to cheat text transforms)

What is the difference between porting and complete rewrite? I thought they are the same?

> What is the difference between porting and complete rewrite? I thought they are the same?

As with so many terms in computing, there aren't universally agreed definitions. But in the author's post, I read "porting" as switching languages while changing as little as possible vs "complete rewrite" as redesigning/rearchitecting it along the way.

To add to that, I would say that simple forms of "porting" can be done without really understanding the overall architecture. You just need to understand each line (or a few lines around it) at a time. I understood the post to mean something like that, which sounds like a good plan here, to minimize the risk of adding bugs.

And in comparison, a "complete rewrite" would require a full understanding of the big picture design of the original code, come up with a perhaps different one in the rewrite, and keep the differences in your head as you go (or, forget about the original design entirely and solve it from scratch - and keep that in your head as you go). Far more work and risk, and it would have been a bad choice here.

If that's the case, it seems unlikely that they'll be able to get the hoped for performance improvements

I think here “rewrite” means completely changing the internal structure of the code (while preserving the functionality) whereas “port” means copy the structure of the internal code with as little modification as possible. And it’s hard to do the latter in Rust since Rust is not garbage collected.

Porting can involve a spectrum of changes. If you are porting to a different language, often for bug-for-bug compatibility you try to maintain the same rough code and class/object structure as the source codebase. This might give you some non-idiomatic code at first, but once you have good test coverage you can refactor. In this case you might maintain some of the upstream code structure to better match future changes.

If they focus on the core features and make a significant enough improvement, perhaps the core TS team could be convinced to pull it in as a native extension and make calls to it for the operations it covers? If something like 62x improvement is really achievable, it's hard to imagine the team not being interested at all.

I would personally be very happy to use a significantly slimmed down subset of TypeScript if it provided a 62x performance improvement. Slow type checking is currently the main drawback of TypeScript development in large projects.

To this point, many teams are already using subsets of TypeScript to improve compilation times. Often times it’s things like always declaring return types on functions to avoid the compiler having to infer it (especially between module boundaries). Calling this a “subset of typescript” might be strong wording, but it does suggest there is some desire in that general area.

It seems like depending on how you configure typescript (e.g, in tsconfig), typescript is already something like an ensemble of mini-languages with different dialects and semantics. Much more so than other languages. But I agree, restricted typescript that has to do less work (data or control flow analysis) would probably be on the orders of magnitude faster.

> To this point, many teams are already using subsets of TypeScript to improve compilation times. Often times it’s things like always declaring return types on functions to avoid the compiler having to infer it (especially between module boundaries).

I hadn't heard of this trick - how much improvement does it make? Seems like it might be good for readability, too?

I haven’t personally benchmarked it, but I know it’s available as an eslint rule (all exported functions must have declared return types).

I always thought this is because some people prefer this style.

This makes sense for semantic reasons though. If you accidentally return a `number` when you returned a `string` in some earlier branch, Typescript will happily unify them and now your function silently returns `string | number`. The linting rule prevents this.

I haven't found any large scale analyses but here's an example of a simple type annotation halving compilation time: https://stackoverflow.com/questions/36624273/investigating-l...

The Typescript compiler performance wiki might also be of interest: https://github.com/microsoft/TypeScript/wiki/Performance

The issue here is that if you're running in the JS ecosystem you'll definitely want to use other people code (npm package or internal lib), if the subset breaks JS compatibility then you can break a significant amount of code without realising, if it is "only" a TS subset, then you need to make sure that each lib/package you import are compatible. Anyway this does not seem like a good solution.

to be fair, even within the TypeScript world that can be a problem. If typescript versions don't line up, or if your various strict options are too strict, you can get into really weird situations. It -generally- works, but when it doesn't, it's hell.

I'm not saying this is impossible but I think you'd have to work rather carefully to not have to parse the code twice and effectively run two type checkers in parallel. Which, on second though, you could run the fast one then the slow one. Perhaps that's an option.

> I would personally be very happy to use a significantly slimmed down subset of TypeScript if it provided a 62x performance improvement. Slow type checking is currently the main drawback of TypeScript development in large projects.

We all think that until the slimmed down subset doesn't work with our favorite libraries. Sure, there may be people who use no dependencies, but I'd bet that there's sophisticated types in some very popular libraries that even low dependency programmers use.

62x improvement is not achievable. 6.2x might be.

edit: I see they are running it with 8 threads, in which case yes 50x is achievable. In any sizable codebase though, you should probably split into modules and run with some parallelism via e.g. `wsrun` or `turborepo`

esbuild is > 125x faster than Webpack, doing the exact same job. It's not some theoretical microbenchmark, you'll see it when converting a project from one to the other. If a software hasn't been designed with speed in mind, and I don't think tsc has, there can be massive performance improvements possible if you create a new implementation with performance as a top goal.

Webpack does way more than esbuild, including running a typechecking compiler instead of just transpiling, running compilers able to downlevel emit to ES5 and providing a deep plugin architecture allowing you to hook into any bit you like. But yes, it hasn't been designed with speed in mind - it has been designed for maximum extensibility instead. Its the same reason why Babel is slow compared to sucrase (written in JS, currently faster than SWC and esbuild but doing somewhat less - https://github.com/alangpierce/sucrase)

tsc has in fact been designed with speed in mind (I've been following the project since before it moved to GitHub, which is when they redesigned it for speed). Going beyond 1 order of magnitude performance improvement while implementing the full feature set is highly unlikely.

62X is still one order of magnitude.

I would bet on a port eventually being done and hitting at least 10X.

Well we have some intersection then, I think it will be 10x at most :)

Isn’t that already available by running in “transpile only” and having the type checking happen in the background?

> I would personally be very happy to use a significantly slimmed down subset of TypeScript if it provided a 62x performance improvement.

I'm just guessing but I think a lot of Typescript's really complex stuff is to support third party libraries with really complex types - at least they often refer to libraries they've improved support for when they discuss complex seeming additions. So I wonder how much of the JS ecosystem you'd have to cut off to get to such a subset. A particular example would be that I've often seen a chain of 20+ dependencies leading to Lodash, and I'd guess that Lodash is really complex to type. If you lose everything that's eventually dependent on something that's really complex to type, do you have much of NPM left?

As a maintainer for Redux, Redux Toolkit, and React-Redux, I can tell you that we would _love_ to have several additional features in TS that would make our library TS types much easier to maintain. Like, say, partially specified generics.

Tanner Linsley (author of React Table, React Query, and other similar libs) has expressed similar concerns.

(I'm actually starting to work on a talk for the upcoming TS Congress conference about "lessons learned maintaining TS libs", and this will be one of the things I point to.)

Even if it’s not official i could use a slimmed-down, faster TypeScript. Even if it has slightly different semantics. I always use skipLibCheck so TS only checks my code.

It’s a form of linting, not semantics, so it doesn’t have to be exact.

One of the main requirements of TypeScript is that the compiler has to work in a browser. The monaco editor is the prince jewel of TypeScript applications.

So the only possibility I see is a webassembly alternative.

Not saying that the TypeScript team should actually do this, but Rust does have best-in-class WASM support.

No spec, but the golden files that check that the compiler is working as intended are very comprehensive. If you can generate the same ~38,000 files in this directory from the test cases, you're pretty much on par with tsc: https://github.com/microsoft/TypeScript/tree/main/tests/base...

But that only works for a bug-for-bug compiler, no? If you want to build something like a linter, you couldn't use them to validate you are covering the fully syntactic range. Though I guess it would help look at coverage if your linter threw any errors, but it wouldn't validate correctness.

This is true of many programming languages projects. It's not uncommon for someone to write a language in a weekend, and then marvel at how their very simple toy language is faster than C! And look how quickly it compiles! Thus begins a several years-long journey, where with every feature added the runtime and compiler get slower and slower and slower.

There is no specs so TypeScript's own impelemtption might not be accurate to what the team imagined on how things should work.

That means if this implementation passes all TypeScript's tests, it is as good as TypeScript.

Each new issue that is opened in either repo that shows discrepancy can be a new unit test added to the TypeScript codebase.

Good thing is that the only thing it does is type-checking. So if you miss some edge case it won't influence at all how your ts code works.

The only thing is it may not catch some type error, which JS already does none of and TS can skip wherever you want or even implicitly and people are fine with that.

If I get 98% of TS typechecking 10 times faster. It's a huge win.

You're vastly underestimating the advanced type system features TypeScript requires to typecheck even the most basic JavaScript code. It would be more like 10% rather than 98%. Intersection and union types are no joke, and are notoriously difficult to make performant.

I just saw an example where the upcoming 4.6 will improve the clever use of recursive types done by some libraries.

Give the people a turing complete type system and they will write a complete application with it.

My experience with a project like this is unless you do reach fabled 1:1 parity, the new tool is orders of magnitude less useful or even actively harmful.

Saving a minute every cycle sounds very attractive, but when a person spends a morning hacking and then finds out the normal tool chain doesn't accept their work when the optimized one did, they will quickly distrust it in the future. It'll also make them very reluctant to share it with their team.

The only reason my project was successful was we ultimately reached a set of compromises with the team that owned the woefully underspecified tool chain and upstreamed large swaths of changes. I would probably never do something like that again.

I think it's okay not to get near 1:1 parity given the amount of unchecked type-casts (foo as unknown as X) and ts-ignores that happen anyway in most code bases. The other advantage is that the typechecker is advisory, meaning the code will run (and might explode) but it will run even if the types are wrong since compiling typescript is mostly just throwing away the type info.

The good thing is you have a ton of code on the Internet to use to test it. Unlike a dynamic language reimplementation, a static language transpiler can check if the results produced are identical to `tsc`

Yeah, the lack of specification for Typescript is quickly turning into the Achilles heel for the web tooling ecosystem. Several projects got the parsing part done, but a lack of specification and the relatively fast release cycle means it isn't really feasible to invest into actually implementing the meat of the TS compiler, so everyone is stuck with that last remnant of critical tooling running slowly.

> It's also a mess that does not have a spec. Even if you somehow manage to make a type checker that implements most of the features, there's no way to ensure it has 1 to 1 parity with TypeScript. Plus that parity can be broken at any point.

You need a test suit for this not a spec.

One way to collaborate would be to write a spec as you go, along with an associated test suite that can be run against multiple implementations.

Presumably TypeScript has a test suite already that might serve as a starting point?

I really like dart actually. It's typed, close enough to JavaScript, and yet I don't think is an unholy mess ?

But in the meantime he gets to work on a fun project, learn a lot and get paid for it at the same time ;)

Reply on news.ycombinator.com