Speeding up the JavaScript ecosystem – Isolated Declarations | Hacker News Reader

Speeding up the JavaScript ecosystem – Isolated Declarations

marvinh.dev

45 pointsrealshadow54 comments

whilst reducing the time to create type definition files from minutes, sometimes even hours, down to less than a second.

WTF. What kind of project would take long to compile from TS? Windows 11 (if it were TS)?

I had a long time personal project that was several megs of code with several hundred types/interfaces. It took 13 seconds to compile. If your application takes hours to compile and is less than a petabyte of source code you have some catastrophic mistakes in your approach.

Author here. That's a good point and maybe a matter of perspective. I've worked on projects where using tsc to generate .d.ts files took close to an hour. Even 13s would be too long in my opinion. It should be instantaneously. I guess it depends on where one's personal tolerance threshold is. I have a bit of a lower one as I easily get distracted and loose flow.

You seem confused about what .d.ts files are. They are type definitions to make using JavaScript code in typescript nicer. They're not regular typescript code.

Just think if you had to take a big JavaScript library, say the size and level of dynamism of jQuery, and manually write annotations for it yourself.

It would indeed take you hours to cover everything.

I am aware. I only use d.ts files to store statements starting with keywords type or interface.

BTW, jQuery is pretty small. With good lint rules you could probably manually type everything without missing anything in less time than you think. Whatever time is spent doing this manually apparently is worth the cost savings comparing to long compile times if takes more than a few seconds. You only need to write the types once and then again on major refactors. If your builds just take seconds you can run them dozens of times a day. If your compile time is hours then whatever automation you imposed to write your types or d.ts files for you is entirely self-defeating as its costing you dramatically more than just doing it manually one time.

When you have a large project types are written as the code they describe are written, so its little more effort than typing on the keyboard in a separate file.

I think they are referring to the manual creation of type definitions. As they mention it in the article.

But then they assume the code is authored in Typescript... Who is authoring in Typescript and then simultaneously writing their d.ts manually?? It makes no sense.

I write all my d.ts files manually. Maybe that’s the problem: people don’t know how to write TS, expect some software to do it for them, and then end up waiting hours on a build that should take 10 seconds.

This confuses me. You either write the d.ts manually before build, which only makes sense if you wrote the original code in JavaScript (not Typescript) or you write the source in Typescript and enable the declaration option in your tsconfig and the d.ts pops out in seconds along with your final JS output.

Unless you are using transpileOnly, generating declarations will not make a few second build become a few hours-- Typescript is already type checking your code.

I write TypeScript projects in TypeScript from the start. If I have an existing JS project that needs to port to TypeScript then I will rewrite it entirely in TypeScript. Yes, that takes some small amount of time, but it’s never a complete rewrite. It’s mostly just minor syntax updates as the actual runtime should remain the same as validated by test automation.

I type everything I declare whether a primitive or not. My complex types like data structures, objects, and modules get typed as interfaces. I write them as I need them. I also have lint rules that check for missing type declarations.

Don’t over think any of this. None of this exists as an exercise in code masturbation. There are two immediate values to TS: warning on unintended type coercion and predictive rapid refactoring. So, just keep plan your software appropriately and keep everything simple.

But why would any of that need to be done manually in a .d.ts file? You can just write your type definitions in ts files

Types are meta data for the compiler. They are not code that is compiled to JavaScript, so I isolate my type definitions to d.ts files. That is the only reason I have d.ts files at all.

Types are also documentation for the developers reading and editing the code.

Maybe if writing .d.ts-es by hand was a proper way to write TS, people would know about that from the docs. Don’t sell your fetishes as some sort of a common knowledge please, it doesn’t help anyone.

Writing types yourself may or may not be the best approach. Everybody is entitled to their opinions. However, builds that take more than 30 seconds (and that is extremely forgiving) is ridiculously wrong.

So, my very best recommendation is to turn that frown upside down and measure things. This is called total cost of ownership (TOS). If the cost of manually writing types is less than the total cost of running builds in a week you then it is absolutely the most correct approach.

https://en.wikipedia.org/wiki/Total_cost_of_ownership

As far as fetishes go I try to avoid logical fallacies like you recommended for me: https://en.wikipedia.org/wiki/Argumentum_ad_populum

Author here. Unfortunately, I think worded that a bit confusingly in the article. I'm used to working with runtimes that run TS natively and didn't have to care about .d.ts files myself for a long time. From that perspective creating them by running the tsc compiler feels like a manual step for me.

There are cases where folks write them by hand, but that's not what I had in mind when writing the article. I've rephrased that part a bit to hopefully make it less confusing.

Thanks for the clarification.

The fact that there are developers out there that are willing to accept hour long compile times from the typescript compiler is disgusting. I’ve seen slow recursive types here and there, but you can analyze your slow types using tsc and fix them yourself.

I hope that slow typescript compiling is a niche issue for massively large/complex codebases and not a common problem.

The blog doesn't explain how to take advantage of isolated modules without using JSR, or how JSR transforms published packages.

How would a project configure node and typescript to use a npm module that has exported .ts files?

You wouldn't, you would use JSR.

I despise this recurring trend in the JS-adjacent world of "cutting edge" people making their published packages intentionally incompatible with what everyone else is doing.

Well, if they want no one to use their libraries, then fine. You would think that as a person publishing library code to a public registry, you would be incentivized to ensure your code is broadly usable. At least in general.

I know this is in the context of JSR where this isn't an issue, but I suppose some people just want to just use a registry as a personal code repository. Id just hope they don't pollute the top level namespace in that case.

Who am I kidding, sadly for NPM that ship sailed years ago :*-(

The site's down for me, here's an Internet Archive link:

https://web.archive.org/web/20240706220437/https://marvinh.d...

I've really enjoyed this author's series on optimization techniques explored through non-trivial yet still small case studies, the other articles linked near the top are all worth checking out too!

Oh no, I just woke up and saw this comment. It seems to be up again. Apologies for the inconvenience. Looks like the HN hug of death is real!

Happy to hear that you like the series!

This article is kind of a messy read, isolated declarations feels like it's being used as a mcguffin to advertise the jsr registry. Both may be neat things on their own, but I feel the connection is a bit misleading.

Isolated declarations should allow parallelization and faster type checking with tooling that supports it. This shouldn't be changing your build/release process or what you export in your package.json.

If you also happen to use the jsr registry to publish your package, it sounds like you can update your package.json to export TS files and they'll compile/inject the JS into your release artifacts on publish. Not sure if this feature requires isolated declarations though.

Author here. Thanks for sharing that feedback. I think I missed the mark by being so used to working with runtimes that natively run TS files, that I forgot that this is not the default for everyone. The article was mostly written from that perspective.

For Node users the isolated declaration feature won't do much other than speed up the creation of the .d.ts files a little. For runtimes that can run TS natively like Deno (disclaimer: I work for Deno) the benefit is much more apparent as you always want the original TS sources, but can generate .d.ts files on the fly for faster type checking.

EDIT: Formatting

This doesn't really seem to address my most common packaging pain points in the JS/TS ecosystem (which is the inability to easily use ESM in legacy commonjs applications) but maybe I'm just working in the wrong area?

I think the author seriously exaggerates the difficulty of creating type declaration files. It's literally one boolean setting in your tsconfig. I wish they would have given more time to proving the alleged pain points with some examples rather than just telling us how great this new feature is. I personally don't see how this changes my life at all, and I'm a full time JS/TS dev.

The article suggests that you ship typescript so the app would compile down to esm or cjs.

Node 22 has an experimental flag that supports requiring esm as long as there's no top-level await

Author here. I should've worded that better in the article. The takeaway should not be to publish only TS sources to npm. The whole npm ecosystem is based around the assumption that you ship .js files and doing anything else would break it. I've updated the wording to hopefully make it less confusing.

From what I understand, the author is persuading library authors to stop compiling TS->JS, and instead ship your TS files to the users over npm. I like the idea of it, but will it actually work in practice? What if some dependency only compiles with some version of tsc? What if it needs different tsconfig-s?

> but will it actually work in practice?

No. It seems like they are assuming the user is using JSR, which handles generation of JS and d.ts for you. But then none of this would even be a problem.

The NPM registry doesn't do any of that. You could absolutely publish only .ts files to NPM and then use a post install script to generate JS and d.ts files, but that is a pretty terrible idea. Even if the process is super fast, it's still an operation done potentially millions of times, versus an operation which can happen once at publish time. You could also assume the user will have their own mechanism for compiling TS or consuming TS directly, but this is even worse.

You can specify different exports based on environment in the package.json so you could ship TS files… alongside JS files, so it feels a little redundant.

Yes, I recommend shipping TS source alongside compiled JS+d.ts even if you don't support a TS native environment, because you then have the potential of a source mapped debugging experience within that library (though currently doing that is a bit painful in some debuggers).

That can be nice, but do keep in mind that shipping more files leads to a larger tarball downloaded from npm.

Of course. A few more kilobytes is a good tradeoff for debuggability and the ability to read the original code during development. Web apps that get bundled won't include it, ESM web apps won't request them, and if a server side application is sensitive to deployment size, it's trivial to remove source files from all dependencies during build. As an application developer I'd much rather an upstream package provide it and let me strip it out than the opposite.

Author here. I should've worded that better. The recommendation is not to stop compiling TS->JS for npm users. The whole npm ecosystem is built around the assumption that you ship JS and doing anything else would break it.

In Deno (disclaimer: I work for Deno), which supports running TS natively, packaging outside of npm has always worked by shipping the TS files directly. Transpiling them down to JS files is only something you ever had to do when publishing on npm.

So far the problem that a dependency only compiles with a certain version of tsc hasn't popped up since Deno was launched. But that may have been because Deno always ships with latest TS and Deno users don't hesitate to update as we haven't done any breaking changes so far.

JavaScript runtimes are full of optimizations that execute conditionally on specific scenarios. If you don't follow the rewarding path of optimization you then go to the deeply unhappy path of deoptimization.

It's easy to get started with JavaScript, but to squeeze every bit of performance in JavaScript will quickly get you into a very complex topics that are not documented and change may change from version to version as they are often not "contracts".

The only way to learn those obscure topics is by reading the source code for your JavaScript implementation, their bug tracker, etc. This means if you have not been reading C++ or executed a non-JS profiler to make your JS faster then you probably have not gone far enough.

There are unexpected things that you would never have suspected if you had not read the implementation.

I feel like the point is skipped in the post: if you ship TS sources, it means you need the TS compiler on the installed side, right? And specifically the right TS version for each installed package... which in some cases may mean conflicting requirements?

(If you own code is in JS, that is)

Author here. True, I think I should've worded it better. I work for Deno and it being a JS runtime that is able to run TS natively makes this less of an issue. The problem that a dependency needs a specific TS version hasn't come up so far. But that might be because Deno users tend to update to the latest version pretty frequently as we haven't done any breaking changes so far.

The TypeScript compiler is, in most or even all cases, backward compatible. You just need to have the latest version installed.

They may not be frequent, but they do happen https://www.typescriptlang.org/docs/handbook/release-notes/t...

And given enough public modules, we will find some conflicts in the dependency graph. YOLO just use the latest version is rarely a full solution.

You would be probably better off just running esbuild or similar - it will happily strip types from any version of TS.

I don't know what an isolated declaration is. Do we write TS source differently?

I believe the intention behind isolatedModules is to make compilation faster by forcing you to define types that require no inference from other files (thus, being isolated).

Not… whatever this is.

Please do not start publishing raw Typescript into the NPM registry thinking that all the reasons we have not done that in the past do not continue to apply.

I shouldn't have to say this, but at least 27 people upvoted this article, so here we are.

isolatedDeclarations is most useful for large codebases and other situations where performing a full Typescript compilation is prohibitively expensive. This is not the vast majority of cases. It is the declarations analog of transpileOnly, which skips all type checks and simply strips Typescript's special syntax.

The author seems to assume use of JSR, which is an alternative to the NPM registry which automatically compiles Typescript. Not being terribly familiar with it, it's possible this is more relevant to users of that registry, but it's not clear why this is an issue when the registry itself is handling transpiling and declaration generation.

EDIT: To be clear, I think it's totally fine to publish your TS source in addition to your JS and d.ts outputs, just don't publish TS only packages, please.

Author here. Agree! The whole npm ecosystem is based around npm packages shipping JS files and doing anything else would break it. I've updated the article to hopefully make it less confusing. The takeaway should definitely not to only publish TS files to npm.

I mainly work with Deno which can run TS files natively (disclaimer: I'm employed by them). This changes the parameters a little as all registries for Deno could always ships the TS sources directly. But when publishing to npm you always have to publish the JS sources to work with runtimes like Node that don't support running TS natively.

> that all the reasons we have not done that in the past do not continue to apply.

Your plea would be better if you mentioned what the actual reasons are.

Sure-- the first issue is that NPM is a JavaScript registry, and it's bad form to restrict usage of your library to Typescript projects, or require non TS users to go through hoops to use your code. Additionally, if your own project executes as JavaScript (the vast majority of cases other than sort of Deno), your project will need to do special work to have JS files produced for the TS only library-- by default this won't happen so imports to the library will fail at runtime. When you do add node_modules/some-package to your TS project, this will change the structure of the files emitted into your output directory (assuming you use one, which is highly recommended), creating weird pathing like dist/src/main.js. Also the module would end up in dist/node_modules/some-package, which would be fine as long as the package isn't using non-code files within its module, which it likely is. And of course, you'd be missing all export information within that folder, which is likely to break imports, so you'll need to copy the package.json....

This is all a massive breach of encapsulation.

Is it possible to consume a TS only library distributed via NPM in a Node.js Typescript project? Yes, certainly. Is it ergonomic for consumers? No, certainly not.

Overall, it will be a bad experience for a lot of environments where JS files are expected to be present but aren't. Deno is probably the only environment that can consume NPM packages where it could conceivably work.

Id love it of npm added support for additional artifacts — separate tarballs for source, code, binaries, source maps, docs etc

Fair, but if you don't fall into the pit of despair by consuming micro libraries (ie "is-true" or "leftpad"), the overhead of having source and source maps present on a developer environment or even a server deployment is not terrible. If you are bundling for web delivery, those files will be omitted automatically by nature of the bundling process.

While it would be nice to have native support for package components, personally I think the best path is to ship all of it (save perhaps documentation, where there's currently no ergonomic way for a consumer to make use of it) in the main package.

The above is assuming "docs" mean some kind of external documentation, you absolutely should not strip out your docblocks when you package your code. Ditto for minification: Just don't. If the end user needs to optimize for size, they will be minifying anyway, all packaging minification does is create painful debugging and code reading experiences for consumers.

...wait.

Is the entire article based on the assumption that everyone has Typescript installed in every project? Otherwise, how would publish Typescript files work at all?

> We only ever ship build artifacts, the compiled JS and the relevant .d.ts files in a package.

Isn't this because anyone, regardless of whether they have use the project in JS or TS, can import the library without caring out how the library code is implemented (in JS or TS)? It is not perfect but has worked for almost everyone. How would the author's project layout help people who only ever write code in JavaScript and never installed the Typescript package? Does Typescript have to be a dev dependency of any project that has an upstream package written in Typescript? That seems a bad idea.

I write almost all my JavaScript projects with Typescript now (it's a bad sentence but you know what I mean), but I don't think we should ever make the life harder for those who only write in JavaScript.

I think most bundlers can handle TS nowadays so you don't actually need tsc.

Technically you still need tsc or an alternative TypeScript compiler like swc, but bundlers sometimes include them in their own dependencies so you don’t have to think about it.

I've heard of `isolatedDeclarations` in TS 5.5, but this post left me with more questions than answers because it conflates so many distinct things (TS 5.5, Deno, JSR).

I'll try my best to break it down succinctly:

1) Historically, .d.ts generation is slow because it requires the full TypeScript type checker in order to correctly handle all cases (e.g. exported functions with inferred return types)

2) However, if types were to be fully explicit, then .d.ts generation could be performed without a full type checker (i.e. by a faster compiler via mere syntax transformation)

3) The isolatedDeclarations flag causes the TS compiler to trigger errors when any exports would require inference in order to generate .d.ts

4) Thus, the isolatedDeclarations flag can be used to guarantee compatibility with faster, simpler .d.ts generation using syntax-based transformation

5) A consequence of simpler .d.ts generation is that it can be trivially done in parallel, which can be helpful for large monorepos

Author here. That's good feedback. Looks like that part of my article is a bit confusing and I've updated the phrasing a little to hopefully make it less so.

You're spot on with all of your points. The isolated declarations feature forces your code to be written in a way that creating the .d.ts files is a mere exercise of stripping syntax and can be done easily without invoking the tsc compiler.

For runtimes like Deno which support running TS natively (disclaimer: I work for Deno) you never had to care about creating .d.ts files when publishing your package to any of the Deno registries: previously /x, now JSR. In the background though we've always tried to feed the tsc compiler in Deno something like .d.ts files though as it's quicker for type checking purposes.

This is all fully correct, which is why isolatedDeclarations is a nice feature... for authors of large Typescript projects. The tradeoff is to be more explicit on your public API surface so that you can reduce build times. That's great.

It really doesn't do the things the author thinks it does.

Reply on news.ycombinator.com