Extreme explorations of TypeScript's type system | Hacker News Reader

Extreme explorations of TypeScript's type system

learningtypescript.com

You can do some truly silly things with sufficiently ridiculous uses of typescript. I built a typecheck-time spell checker[0] in it such that:

  import { ValidWords } from "./spellcheck";
  
  // Typechecks cleanly:
  const result: ValidWords<"the quick brown fox."> = "valid";
  
  // Throws a type error
  const result: ValidWords<"the qxick brown fox."> = "valid";

[0] https://github.com/kkuchta/TSpell

So does spellcheck contain a gigantic array of the English language?

the source code to answer your question is directly above your question

I'm inclined to cut someone a bit of slack for not following the link to the implementation if the context suggests deeply nested template meta programming golf is involved. It's surprisingly much less complicated than I would have expected.

Yes. It looks like this:

    export type ValidWords<T extends string> = T extends ""
      ? "valid"
      : T extends `the${infer Rest}` | `of${infer Rest}` | `and${infer Rest}` | ...

This is great!

How long does this take to compile?

Something like 40 seconds for a 9-word sentence. Would not recommend in production. :)

so that's great and I know this wasn't your point but ....

I use this VSCode extension

https://marketplace.visualstudio.com/items?itemName=streetsi...

To spell check my code. It's surprisingly useful. It doesn't check at compile time but it does check camelCase and snake_case and even in typescript code I've found it highlight various actual issues.

What do you do when a colleague has made the spelling error in the pasta? Open a PR?

Your GitHub account is a rabbit hole of crazy interesting "monstrosities" like this. I love them! My favorite ones, apart from this typechecker, are css-only-chat[1] and tabdb[2]

1: https://github.com/kkuchta/css-only-chat 2: https://github.com/kkuchta/tabdb

When doing fancy things with typescript types, be really careful - it's possible to accidentally construct typescript types that will increase your tsc compile times by multiple seconds and the tooling for troubleshooting this is nonexistent. A tiny change to one codebase I work on made compile times go from 300ms to something like 7 seconds and it took me something like 14 hours of grepping and manually bisecting source code to find the cause - tsc was O(N * N * N) trying all possible types for a string literal to determine whether any of them were valid matches, and someone had defined a very fancy string literal type.

When this happens, typescript language integration (like in vs code or sublime text) will suddenly fall over and stop working correctly, and it'll be near impossible to figure that out too.

Our build uses rollup to invoke tsc and as it happens their profiling system doesn't actually measure how long tsc takes to run - the time is unaccounted :) So in general, be aware that 'typescript is taking a long time to compile' is a blind spot for this whole ecosystem and if you hit it you're going to have to work hard to fix it.

This doesn't even require a type system anywhere near as fancy as what TS has. C#, for example, has a similar problem with how type inference in lambdas interacts with overload resolution, once you start nesting those lambdas:

https://docs.microsoft.com/en-us/archive/blogs/ericlippert/l...

> If you do find a need to use type operations, please—for the sake of any developer who has to read your code, including a future you—try to keep them to a minimum if possible. Use readable names that help readers understand the code as they read it. Leave descriptive comments for anything you think future readers might struggle with.

Also, as you start getting complicated logic in your types, you need to test your types; make sure they admit things they should admit and reject things that they should reject. Ideally these tests can also serve some role as examples for your documentation.

@ts-expect-error is useful for this

We do a _lot_ of this in the Redux library repos (examples: [0] [1] [2] ). We have some incredibly complicated types in our libraries, and we have a bunch of type tests to confirm expected behavior.

Generally, these can just be some TS files that get compiled with `tsc`, but it helps to have a bunch of type-level assertions about expected types.

I actually recently gave a talk on "Lessons Learned Maintaining TS Libraries" [3], and had a couple slides covering the value of type tests and some techniques.

[0] Redux Toolkit's `createSlice`: https://github.com/reduxjs/redux-toolkit/blob/9e24958e6146cd...

[1] Reselect's `createSelector`: https://github.com/reduxjs/reselect/blob/f53eb41d76da0ea5897...

[2] React-Redux's `connect`: https://github.com/reduxjs/react-redux/blob/720f0ba79236cdc3...

[3] https://blog.isquaredsoftware.com/2022/05/presentations-ts-l...

Hey Mark, I’m actually currently looking at a similar problem.

I’m writing a HTTP client based on composition. The exact details aren’t important, but one of the goals is to have a strong type system for describing a valid pipeline of things like response parsers. Imagine something like

Doing “type tests” alone isn’t too hard - we can just use conditional types and the extends keyword. If the code compiles, fine.

But the harder part is negative type tests. “Given this code, the developer should get this error from TSC”. But this is just as important a part of the API; your types are there to convince the consumer that they can call a type-checked API with confidence.

In theory it should be plausible to run TSC programmatically. The issue is that TypeScript’s ScriptProcessor API really wants to be called with files on the filesystem rather than source text. So I am having to do some bodging. If I can get something sorted I may write a repo to demo it, I think it is a common problem.

I use @ts-expect-error for testing the negative case, but if there was a practical way of testing the actual error reported that would be wonderful.

My solution to this was to run tsc from Jest tests and do snapshot tests for the error messages

https://github.com/noppa/get-optional/tree/master/tests/typi...

Hmm. While it may not be the immediate answer to your question, my Redux teammate Lenz Weber ( @phryneas ) wrote a Remark plugin that parses TS codeblocks out of Markdown and actually runs them through the TS compiler. As part of that I know he generates a bunch of "virtual files", including some parsing that lets you add an extra section of the codeblock representing another file to be compiled along with the actual example. The source for that may at least help give you some examples of how to use TS programmatically:

https://github.com/phryneas/remark-typescript-tools

I lost it when at my previous job I found a 20 multiline super complex type defined by another dev, asked him to describe it because I was in a tight deadline and had not time to parse whatever he was defining. He starts with "it's pretty simple" and then used like 10 minutes to describe me what he meant while writing on paper the various pieces getting confused two times. At the end of the day it could be simplified in a one line union type of a few strings which would cover 99% of the usecases and the other 1% was something we had never used and would never use.

I really wish people would focus more on keeping types as simple as possible instead of using that complexity just because the language allowed it.

It isn’t unusual that a complex working solution can be simplified over time. Isn’t that part of eliminating tech debt or something?

Typescript is just one of those languages where things can get carried away but then reigned in again with a nice complexity gradient.

>I lost it when at my previous job I found a 20 multiline super complex type defined by another dev, asked him to describe it because I was in a tight deadline and had not time to parse whatever he was defining. He starts with "it's pretty simple" and then used like 10 minutes to describe me what he meant while writing on paper the various pieces getting confused two time

sorry about that

There are probably newer tools for this, but we use `eslint-plugin-expect-type` [0] for this:

https://github.com/cypress-io/cypress/blob/develop/cli/types...

[0]: https://github.com/JoshuaKGoldberg/eslint-plugin-expect-type

To try and limit one's use of operations on types, as suggested in the article, is not really great advice in my opinion. Sure, you would not want to actually implement and use a VM in types, but distilling rules about a program into types and then deriving the actual interfaces and signatures from those rules with operations on types? That's quite powerful.

TypeScript's type annotations are really a DSL embedded into JavaScript. And they can, and, depending on the problem at hand, should be treated as such.

I think it depends how far you go: if you start encoding rules into the type system that are undecidable then you can quickly run into trouble.

To me its about readability. I agree that operations on types generally are a good thing because I find them more readable. Not using operations I've found leads to many very specific types which are hard to read and understand from the devs perspective.

> TypeScript's type annotations are really a DSL embedded into JavaScript. And they can, and, depending on the problem at hand, should be treated as such.

I think this is the key. If treated as you describe, meaning the advanced types are well-written, well-documented, and well unit-tested as if they are "true" code, then using them shouldn't be too much of an issue.

However, I think people often just assume that the types aren't "real" code and thus the normal concepts of good software engineering don't apply and type monstrosities which nobody can understand result.

Imagine if this code[0] wasn't well-documented, fairly clearly written, and also tested. It would definitely be a liability in a codebase.

In addition, the rules of how advanced TypeScript concepts work can be quite nuanced and not always extremely well defined, so you can end up in situations where nobody even _really_ understands why some crazy type works.

[0]: https://github.com/sindresorhus/type-fest/blob/2f418dbbb6182...

Yes, while that is true, TS errors can sometimes be really clunky. And sans a debugger, it is not uncommon for me to be spending stretches of 20-30 mins almost every week trying to unravel complex type errors that span multiple pages. I recently traced a very weird error to TS changing what keyof never evaluates to in a minor version.

So yeah, using discriminated unions, branded types, mapped types etc. in moderation can substantially reduce the surface area of errors - more so than other mainstream nominally typed languages. However, trying to model and prevent every invalid state at type level can lead to a serious drain in productivity. And, I am not really sure how to draw a line between.

> You have to wonder whether you could implement TypeScript itself in that language...

I also wonder if you could compile TypeScript to TypeScript types? After all, you want your type manipulation code to be typesafe.

This is a great list. I feel I'm only scratching the surface when it comes to Typescript, and it would be awesome to have a place where we can see advanced examples of Typescript usage like this.

I've seen many projects where the typing is done so well that it can infer and include all the data I've fed into the TS-defined functions / classes, which is great for IDE autocompletion.

What are some of the projects have done typing that well?

If you're a GraphQL developer, Pothos is the best example - all your user-defined types just fits in it like a glove 99% of the time. It definitely makes the most use of TS generics.

https://pothos-graphql.dev/

(I'm a bit sleepy, so this is the main one I can think of at the moment that I really enjoy using.)

Not really a "project", but TypeScript's own type definitions for the DOM are a great example. Eg. document.createElement returns different subtypes of HTMLElement depending on the string argument (HTML tag name) it gets called with.

purify-ts is my favourite currently.

Some other type-only TS projects:

- RegExp matching through types: https://github.com/desi-ivanov/ts-regexp

- Lambda calculus through types: https://github.com/desi-ivanov/ts-lambda-calc

- Brainfuck through types: https://github.com/susisu/typefuck

These are wonderful, thank you!

I did some fiddling around building a graphql layer with a bunch of complex types. Basically this was trying to encode all the various GraphQL rules into the type system itself e.g. if a resolver takes arguments, ensure that a schema of the correct type is provided as an object etc. I also built a client that would take a schema and ensure you used it correctly at compile time.

Example of the code: https://github.com/pj/typeshaman/blob/main/packages/graphql/...

Documentation is incomplete, unfortunately I had to get a job. I started working on encoding all of SQL as well.

These are some of my "Typescript type system" based projects:

Wordle: https://codesandbox.io/s/wordle-typescript-d4srgi?file=/src/...

Anadrome(Anagram Palindrome): https://codesandbox.io/s/anagram-palindrome-7u14xr?file=/src...

Candy Crush 1D: https://codesandbox.io/s/candycrush-u2v5pr?file=/src/index.t...

Are there (m)any other languages with type systems as flexible and powerful as Typescript's?

I would guess that all languages with Turing complete typesystems would be technically equally powerful. I think Typescript supports more useful behaviors out of the box.

Equally computationally powerful, but not necessarily equally powerful as type systems. "X is Turing complete" means that it can compute the same set of computations as any other Turing complete system, but it does not mean that it can accept inputs and produce results in an encoding relevant to what you're trying to do.

Yes there are. For instance Idris (https://www.idris-lang.org/) has a way more powerful typesystem than Typescript.

If you are looking for more practical and less academic languages, then Scala would be one of the languages that technically has a more powerful/generalized typesystem but at the same time is harder to use compared to Typescript's and cannot do some things that Typescript can do.

I really like the intro page for Idris.

> In type-driven development, types are tools for constructing programs. We treat the type as the plan for a program, and use the compiler and type checker as our assistant, guiding us to a complete program that satisfies the type. The more expressive the type is that we give up front, the more confidence we can have that the resulting program will be correct.

Great stuff.

Other examples are Agda and Lean.

I believe that typescript type system is so flexible, powerful and complex just because it had to be adapted and built around the shortcomings and limitation of javascript. It makes no sense to have something like it if you build a language from the ground up (or if you could just scrap backward compatibility in a bad designed one)

I think I disagree (though happy to be corrected). It's common to have dervied types (example Base = Shape, Derived = Circle, Rectangle, Hexagon). Often you have a collection of Shape and often there is some runtime code that can go from a Base type to a Dervied type based on some key or value but the relationship between the key and the Dervied type is rarely directly expressed in the type system where as in typescript it can be.

I think an example is how typescript can know, based on the first argument to a listener, what the Event type coming in will be

    elem.addEventListner('mousedown', (foo) => {...});
    elem.addEventListner('keydown', (bar) => {...});

typescript knows foo is a MouseEvent and bar is a KeyboardEvent

Of course you could argue that `addEventListener` is just bad design but I feel like there are legit uses to being able to associate an enum or string with type and I haven't seen that feature in other languages I've used.

Keeping in mind that it can only associate compile-time-known values, there's no practical advantage over something like a tagged union.

I've had to work with some really complex data types and I have really appreciated the utility types, conditional types, modifying data type recursively, template literal types etc. I was wondering how many other types languages are able to do all that stuff.

No, TypeScript has these type system features because they are necessary to accurately type the programs people actually write using JavaScript.

This isn't because of the shortcomings of JavaScript, it's because of the shortcomings of simple type systems.

Simple type systems (like those of Java or Go) restrict the kinds of programs you can easily write while satisfying the type checker. Normally, the alternative is resorting to languages without static type checkers, like JavaScript.

TypeScript allows mostly idiomatic JavaScript even with complex runtime invariants to be given quite accurate types at compile time. It has features I miss all the time in Java.

If you're going down the rabbit hole of writing complex types, check out Dan Vanderkam's "The Display of Types" post[0]. It goes into how types show up in editors and error messages and such, and has a bunch of tricks for improving type readability. I really wish I read it sooner!

[0]: https://effectivetypescript.com/2022/02/25/gentips-4-display...

TypeScript's type system is Turing Complete: meaning it has conditional branching (conditional types) and works with an arbitrary huge amount of memory. As a result, you can use the type system as its own programming language complete with variables, functions, and recursion. This blog post is a starting list of a bunch of the shenanigans TypeScript developers have pushed the type system to be able to do.

It also makes static checking undecidable in pathological cases. I know Idris has this problem as well; they work around it by skipping partial functions in types, so you end up with "total" and "partial" programs where only the former are completely typechecked proofs.

That's not the meaning of turing completeness, just an implication of the space hierarchy theorems. There are systems that also have branching and unbounded memory, but are not turing complete. Context free grammars for example.

Turing completeness means for TS that for every computable function, there is a TS type that can compute that function (if TS wouldn't limit the recursion depth).

Does anybody have a document that describes all type constructions in typescript? The "official" handbook doesn't seem complete. Some time ago, I tried to use tuples in types, but I couldn't figure out the correct syntax. I found some vaguely similar examples on stackoverflow, so it seems some people did get that information.

It helps to read the release notes to keep up to date. https://www.typescriptlang.org/docs/handbook/release-notes/t...

Thanks. Looks like there are a few release notes to catch up.

Advanced type systems are fun to play with. But unfortunately some people get carried away and build a mountain of unnecessary complexity that other developers then have to deal with. A bit like Lisp macros. It is fun to implement your own type system and DSLs in Lisp. But the result is likely to be completely unmaintainable by anybody else and yourself a year later when you have forgotten how it works. I have seen the same happen with templates in C++. Developers that spend weeks having fun building a mountain of template hell to solve a problem that could have been solved in a few hours without template magic. As with everything else, keeping an eye on the benefit/cost ratio is key.

I'd expect Lisp macros to be easier to debug, since they are written in Lisp themselves (and not in a template language) and the usual interactive debugging tools apply.

There are lots of Lisp DSLs using macros which have been maintained by different people over several decades.

Very very cool. Thanks for sharing. SQL Database engine source code exceeded my expectation.

Reply on news.ycombinator.com