Deno 1.28: Featuring 1.3M New Modules
deno.com
deno.com
But hey, that’s pragmatism for you, sometimes you just have to let go of ideals even if it hurts a bit.
I’m certainly looking forward to what happens next, even if more from the sidelines.
But on the other hand, that's how we end up with a programming language landscape where every language is mostly the same, except for community conventions and very slightly different syntax.
I'd love it if more languages where strongly controlled like Clojure and alike, where there is a unified vision that is well kept across time. Not that Clojure is perfect, but probably the best example of a language where the community is welcome to suggest things but unless the person in control approves it, it won't make it into the core language and instead will/could be implemented as a library. In contrast to Rust which seems to base language additions/changes based on popularity in the community.
I don't want a different language, I want multitude of different languages. And not a multitude of different Fortrans/C-like language.
If you'll permit me to be a cynic: this is VC funding for you. Deno took on a ton of funding and they need results. The barrier of NPM incompatibility simply can't be allowed to get in the way of them getting marketshare.
The dreaded npm ecosystem is still full of value.
I tried Deno a few months ago, because I wanted to avoid the hassle with setting up a package.json, yarn lockfile, adding dependencies, etc.. just to run a TS script that generates k8s manifest YAMLs via https://github.com/cdk8s-team/cdk8s-plus
But very quickly I realized it's not there yet.
Search this transcript for "any mistakes so far": https://changelog.com/podcast/443
Meanwhile configuration and npm support was something that initially (pre and at 1.0 times) was something considered a bad thing. Now, not so much.
I would lie if I said that such a hard stance on these things wasn’t what initially drew me to the project.
Because as much as there are loud voices on the internet decrying packages, most people are not so ideologically focused. They just want to get their code written. Packages help and so they use packages. Plus most of the loud voices neglect to offer any solution other than "packages are bad!!!"
It doesn't matter how fast a type checker is at that point at runtime.
The only reason to bring back type checking at runtime would be if V8 et al ever also started using type hints as JIT hints and you want to triple check you are sending the right type hints and not accidentally stomping on your own runtime performance. (Which is intentionally not in that Stage 1 proposal mind you: no one is expecting runtime type hints in the near future.)
I'm not familiar with Deno, but have skimmed a few posts about it here and on Reddit. In any case, my point of view here is general enough that it doesn't matter if it's literally about the Deno project or some other new language/runtime project.
When you say,
> And this is a great move, help people to switch from "node" to "deno" while they keep using existing tools/libraries they use. And slowly, people can then switch to deno standard/3rdParty modules.
my main thought is that compromising whatever benefits Deno envisioned to "attract" Node developers actually gives Node developers LESS reason to switch.
I'm no psychologist, but I can't help but believe that part of the reason Rust got such a cult-like following (myself included) is because a bunch of us spent years writing C++ and then tried Rust and it wasn't smooth. Perhaps that's paradoxical, but for me, it really showed me how much safer and more robust my code could be with this new tool. I don't think that Rust would be as popular today if it had some kind of unsafely-call-C++-mode enabled by default.
It's a balance, obviously. If you want your language/runtime to be the best quality, you don't compromise much; if you want it to be popular, you make it easy to get in to. Often those two have some tension.
Just my two cents as someone who's a bit more on the idealist/academic side of things and is super disappointed by the compromises in languages like Kotlin and TypeScript.
I prefer Deno packages.
Deno is better than nodejs but nodejs is “good enough”.
Competing against “good enough” is extremely hard.
If Deno becomes a better choice than node through npm compatibility them that’s a huge win for Deno.
If I have to get an entire application’s types perfect before even running the code that’s a huge DX slowdown.
Types should pass, strictly, before merging.
Personally speaking, isn't that exactly the value of having a type system in the first place? If it still type checks after the refactor it ideally* is still working. Unless of course some system boundaries changed or there is some dynamic component.
I suppose we both come from different philosophies here. I write out the types for a program first, then the behavior follows through. If I cannot properly determine the types I escape with dynamism.
You seem to dynamically write the system up and then determine the types, do I interpret that right?
The current setup might be a better experience then, but I think the default matters here.
In my little bubble I've encountered more libraries with broken typing since type checking has been reduced, so I perceive it as a net-negative. You end up having to explain that you need to take manual precautions to actually get type checking.
Of course this could just be due to the growing user-base and the higher probability of hitting a "wrongly typed" library, so it will remain to be seen how much impact it has.
I'm used to the --check flag by now :)
* How ideal depends on how expressive the type system is.
I am still not using Deno as my main production runtime but at some point I might make the switch.
Personally I'll use Deno if I can use it with Next.js which with this npm development I can see Next.js support coming soon. Next.js and Deno are both focused on executing code on the edge so it's just natural for it to happen.
Great work Deno team! You deserve all the credits!
I still like the framework, but it probably targets a more specific segment of the market. I think aleph.js[1] is more like next and then there is the esoteric ultra.js[2] which kind of tries to do something similar and be super bleeding edge.
[0] https://github.com/denoland/fresh/issues/403#issuecomment-11... [1] https://github.com/alephjs/aleph.js [2] https://ultrajs.dev/docs
This is definitely interesting. It will be nice not having to deal with that massive directory in each project.
Still, the end-goal is not the same, IMHO. Outside exceptions (e.g. some developers not packaging their applications and making users install through `pip`/`npm install -g`, which I also really dislike), your typical end-user would not interact with language specific package managers. They're most typically used for development/deployment dependencies, not end-user installation.
Rubygems is usually global.
The granddaddy of them all, CPAN and its cpan install tool, defaults to installing globally, in practice, on most systems.
Pip, as you noted, does it.
I think Maven's typically user-scoped, so not really global, but also not project-scoped like npm. Similar to Go.
You can get around this in most or all cases—sometimes with options or config, sometimes with extra effort or some aliases or scripts, sometimes environment variables, sometimes with 3rd party tools—but the simplest or default mode is global installation, or at least project-global if not user-global in Go's or Maven's cases. And many of those solutions (e.g. rvm) still don't scope installation per-project by default, like npm does.
At the time npm got started, especially, it was definitely an outlier in its default scope. Some others have taken its approach since, and maybe a majority of total language-specific package managers even do, these days, but adjusted for actual use in the wild, I bet NPM's by far the most-used one that operates that way, and that most of the other popular ones scope more broadly—including, often, globally—by default. That is, a language-specific package manager one runs into in the real world, in 2022, usually isn't going to scope packages like NPM does without some extra effort. Global or per-user scoping (the latter being functionally identical on any single-user machine) are more likely.
Even the --node-module-dir flag just creates a symlink to your home directory cache so you can't do per-project package fixes (besides forking and rehosting the package)
I'm also not sure how/if it supports packages that require compiling binaries like sqlite or playwright without post-install hooks
https://www.npmjs.com/package/patch-package
You commit your changes as a diff text file and add the patch-package command to your post-install hook so it runs after every install
My main use for it is fixing other packages' package.json exports field and Typescript definitions
Where should one put the text files in the code base? Do you create some root level "patch" directory?
That’s also how I would share any patches: Just apply them when needed during the build process. NPM doesn’t care about what changes inside packages, so you can hack them all you want.
A popular choice if you need some pr's that aren't (going to be) merged.
You can do a patch package by doing something like the following (and then you can move this into a `deno task` https://deno.land/manual@v1.28.0/tools/task_runner when launching your app to ensure it happens):
deno cache --node-modules-dir main.ts
deno run --allow-read=. --allow-write=. scripts/your_patch_script.ts
deno run --node-modules-dir main.tsAnd while the ‘1.3M modules’ headline is more scary than exciting (for me), I’m glad they locked it behind a special “npm:” classifier and have their own way of dealing with node_modules so things stay explicit, while allowing people to swap over somewhat seamlessly.
I wonder what impact this will have on Deno and NPM long term.
When ES6 came out but wasn't yet widely supported it became pretty common to add a build step to transpile the backend code to. This brought a whole bunch of issues that transpilation brought with it but was seen as a temporary evil worth the tradeoff because the new language features brought so many productivity improvements, but stuck for so long it became pretty standard. And when TypeScript became popular, it felt like we'll just never get rid of this complexity explosion that build systems bring. And then Deno came.
Just imagine the amount of config files you'll be able to delete once you no longer need to build your backend codebase.
[1] https://github.com/solidjs/solid/discussions/332
deno run npm:install-malware
┌ Deno requests write access to /usr/bin/.
├ Requested by `install-malware`
├ Run again with --allow-write to bypass this prompt.
└ Allow? [y/n] (y = yes, allow; n = no, deny) >
I assume it is because of the permissions feature of Deno. If so, at what extent(s) does Deno provide security if I have a deep, deep [npm] dependency that, for instance, reads file, i.e. needs file system permission? Do I have to specify every single permission for each dependency? Does Deno have some kind of "dependencies file" that allows to specify dependencies' permissions?
Signup on https://app.windmill.dev -> New Script -> Next, that's it you can now play with deno and get a feel of the language and auto npm imports.
1. downloading an executable 2. opening an editor 3. writing and saving a file 4. running the executable
Is somehow easier than opening a website writing some code and pressing run?
Faster than opening website - no.
Faster than opening website, signing up, verifying email, saving password - absolutely.
```
import namespace npm
import { chalk } from "chalk"
import { assert } from "test"
…
import namespace local
import { chalk } from "mychalk.ts"
import { assert } from "test"
// who needs extensions anyways?
```
b) The above introduces non-standard JS syntax and semantics, which also goes against Deno's philosophy
a) that's fine. In fact namespacing helps with segregation, because once there is some "npm for deno" one would just set the appropriate Deno.namespace("default") without having to touch a million import "npm:package_xyz".
Explicitly repeating full paths such as
import { createRequire } from "https://deno.land/std/node/module.ts";
over and over again is just an ugly antipattern.
Another thing that helps is that in Deno, it's common to have a deps.ts file that imports and then exports all the third-party libraries in use, for the rest of the project to reference. This is mainly done to make sure everything is using the same versions of everything, but it helps with brevity too. You could even approximate your own namespacing mechanism using this pattern
You create an import map JSON file that tells Deno that maps human useful "short names" and "short paths" to full URLs (including `npm:package@version` URLs).
[1] https://deno.land/manual@v1.28.0/basics/modules/import_maps
not fully supported in WebStorm but the future is near
Interesting!
Interesting indeed that you can remap whole URLs:
{ "imports": { "https://www.unpkg.com/vue/dist/vue.runtime.esm.js": "/node_modules/vue/dist/vue.runtime.esm.js" } }
"… instead grab the one from the local server."
This will surely result in much power and confused human debuggers;)
Changing import maps will also be a prime target for some class of hacks.
require() and .mjs are specific to Node (Deno does away with them), import() is actually a web standard. Not sure how npm: fits into things, though if you're importing an NPM dependency directly you're almost certainly pulling in some non-web stuff anyway, so maybe they just decided that was a lost cause
But "web standards compatible" is one of Deno's stated goals. A whole lot of the Deno code out there (or at least a much larger portion than Node code) can be imported directly, with no preprocessinng, by browser scripts. That's a super exciting goal to work towards.
Which is a bigger change then Deno, as that is basically node with some more stuff included (same engine, lead developer, similar attitude towards standard library etc.), whereas Bun is using the JavaScript Core engine from Safari.
[1]: https://bun.sh
There's still a whole lot of confusion around which is actually faster. Both have published counter benchmarks to prove that they are faster than their rival.
Please share any feedback jarred@oven.sh
Also, at the moment, npm specifiers aren't supported with `deno compile` (https://deno.land/manual@v1.28.0/tools/compiler), but in the future that will be one way to have a self contained executable with everything ready to go.
This way the execution is almost garanteed to me the same and the edge network doesn't have to support the runtime at all, just allow binaries.
Pragmatism prevails.
https://www.npmjs.com/package/binaryan returns a 404.
The package you try to import must exist on npm.
https://deno.land/manual@v1.27.0/getting_started/permissions
Node developers be like: "Oh that function I just wrote is so beautiful, let's make a module!"
this is not a flex.
It's those random lib authors who take the philosophy too literal that fail the community
The philosophy is beautiful on the surface, which is exactly how far beauty extends.
Write some load-bearing software using a litany of software all connected by pipes throwing untyped text streams around to various handles. If you do this, and you do not experience problems due to composability, one of the following is true: 1) you aren't doing real work, 2) your software exists in an environment which never changes, which is very rare for most people (meaning that you have fixed your environment enough to wallpaper over problems with composability.)
statically compiled stuff and strongly typed data is the only way to longevity that is worth the time you need to put in to a system that must work.
For some things, yes. For ad-hoc stuff, composability is amazing. Once in awhile I need to use a wrench to fix my car, and I have a whole array of wrenches to accommodate most needs I have with that car. I would never drive the car with a wrench still in place holding something together, and that's what the Unix philosophy promotes: take all these small tools which each do their thing and create some larger "grander-purpose" software with them.
That works if you need to do a one-time (or at least infrequent) analysis of some raw text, but I have learned over and over to not rely on the passing of plain text between programs for anything that you need to work later on. Something always changes which breaks it. Always. Maybe not for a year, but some update happens which subtly changes the output of a command which breaks stuff down the line.
With typed data checking and compiled applications (the opposite of passing untyped text between individual tools) I can easily account for errors before they happen, and rely on tests which exercise these things, and gain significant performance improvements over piped single-purpose tools, in less time overall.
The issue is when each function/command is a separate package that needs to tracked separately and depends on 10 additional packages.
I would like to see the typical dependency tree of a linux distro, I guess it is shallower and has a smaller fanout than a JS application.
Each additional dependency you add is another vector for a supply chain attack. If you keep your dependencies to the large libraries (say, use lodash instead of a hundred tiny libraries), there are at least fewer people in your supply chain. But if you are depending on left-pad for its 10 lines of code (that still managed to have bugs), how many other random tiny dependencies did you bring in? How many individuals are in your supply chain? All it takes is for one of them to go rogue.
(Version pinning can help, but npm's default behavior is to accept minor version upgrades instead of pinning to the exact version that was live when you installed it. Lock files usually just get updated and recommitted without very much thought, so they don't help much either.)
Automated builds should use ‘npm ci’ to use the exact versions.
If a developer goes crazy and push broken updates, it has been reversed and made the front page of most IT news websites in the past.
The supply chain of software rely on so many things. It’s impossible for anyone to trust everything. I don’t trust OpenSSL for security, the Linux kernel still doesn’t have unit tests as far as I know, my intel CPU is a black box with proprietary microcode that is known to allow network access to anything , so many tiny libs in C are half maintained,…
left-pad being offline for a few hours many years ago or fakers doing an infinite loop in one promptly removed release last year are not preventing me to sleep.
This is the perspective of a relatively detached observer who has played with both a little and kept up with their development somewhat but hasn't done a serious project with either.
Bun also adds many runtime APIs like a builtin websocket server, FFI, Bun.mmap, SQLite, Bun.Transpiler and more.
`bun dev` let’s you use bun’s transpiler for frontend code
`bun install` is an npm client you can use with Node and it installs packages 20x - 100x faster than npm/yarn/pnpm
`bun run` let’s you run package.json scripts really fast
(I work on Bun)
It makes 100% sense for Deno to do this from a business and marketshare standpoint: they need those modules in order for people to make the kind of projects they're making with Node. But I was really hoping that Deno would be a reboot: flush out all the awful NPM modules out there you don't even know you have a dependency on and create a JS module ecosystem worth its salt. In some ways "1.3M new modules" is more scary than impressive.
But alas, here we are. We've got the JS ecosystem we deserve.
Clean reboots are always hard, usually don't work out. Just look at the Py2-Py3 transition. Also there's a Perl6 story somewhere here. (Relevant xckd[1], relevant c2 entry[2] )
This kind of view is very prevalent in the web community, and in my opinion it's often the failing of the ecosystem. Good engineering takes time, there's almost no project out there that nailed everything on day one. It is through iteration and progressive improvement we achieve big things. Anyone with a crash course in programming can publish a library in a week. But it takes proficiency and hard work to maintain an interface over years or even decades. There is also no such thing as a perfect solution. Every solution makes tradeoffs and being able to adapt to tradeoffs relevant to your problem is a huge aspect of engineering.
Deno going the scorched earth way would mean pulling back the ecosystem by at least 10 years. There is no guarantee they will get things "perfect" this time around. In all likelihood, they eventually would've come up against the same set of constraints the original ecosystem did, make similar tradeoffs and there would be a conversation 10 years from now about needing a reboot.
The right way to do is to embrace legacy and build solid tooling where the legacy gives you rough edges. Another big aspect of engineering, in my opinion, is not being fanatical about aesthetics. Function over form, every time. I'm glad Deno is taking this route, I was previously critical of their choice to fork core packages like dotenv, semver, base64 etc, but hopefully this will move the community towards consolidation rather fragmentation.
I do think if Node has a successor, it won't be a future version of Node - it will be Deno. There is too much that Deno does better to ignore once all the compatibility is there to migrate.
https://www.joelonsoftware.com/2000/04/06/things-you-should-...
TLDR: Starting from scratch is rarely a good idea. (just read it though, it's not that long).
The fact is that there are lots of good npm packages that have had tons of time and effort put into them, and years of running in production to prove their worth. Yeah there are some dumb packages and micropackages seem like a mistake, but there's no reason to throw it all out.
That's exactly what they tried, by settting up a Deno-only registry.
But there is just way too much work and momentum happening in the NPM ecosystem. Many library maintainers wouldn't want to bother maintaining separate packages for a barely-used ecosystem, and users will be less likely to switch because they can't use the existing (huge!) ecosystem.
It's the same language after all, "just" a different standard library (which is very impactful for lot's of server-side code) and packaging/distribution mechanisms.
This move was probably inevitable.
1. The legacy language baggage of different syntaxes, bad Node APIs, non-standard import styles, etc
2. The existing ecosystem of actual packages and their capabilities
It's a good thing that Deno isn't a reboot of #2. Despite the recurring narrative on HN, the scale of the NPM ecosystem is a good thing. It's one of JavaScript's greatest strengths. I don't understand all the people complaining that they're given too many powerful and free packages to choose from. Whenever I've brought Deno up to people at work or otherwise, their major source of hesitancy for using it on real projects is (rightly) that it's lacking Node's vibrant package ecosystem: "Does it have a mature auth library? What about an AWS SDK? GraphQL server? Client? Redis library? What if we need something later that we don't know we need yet?"
As for #1- Deno is still a reboot, just a softer one. It still gives people lots of reasons to write new packages in the well-formed, web-standard-based way that Deno directly supports. NPM compatibility is a bridge to get us there gradually (because that ecosystem is so valuable, and can't be rebuilt overnight).
It was always a bold move to try and go scorched-earth, and especially with the new pressures from Bun, I think it was the right call to make this exact level of compromise.
However, my main gripe with Deno is that it's tied to one company and it won't solve issues that it doesn't have. As an example of this, a version of nodes cluster module is not supported so there is no way of running one deno process per cpu which is very bad if you're hosting it yourself and want to utilize the full potential of your hardware.
It also means that if your app is bound to some CPU heavy action, like generating a large excel-file or similar, your app will go down since no requests will be processed due to the single thread nature. Of course, such actions can be solved with a web worker but what if you make a coding mistake which renders an unhandled exception? In this case the app would probably go down and if the end user for example does the same action over and over again (which end users tend to do in frustration), the app will go down again and again.
Deno as a company has probably little interest in solving this as the solution is to run it on their paid service. You could of course have several servers hosting the app, but that gets expensive real quick especially if you are a small shop. Another example is to run the entire app process as web workers but then you need to spin up many processes that all have their own ports which you need to add a load balancer in front of it. This is kind of advanced and adds unnecessary complexity to the app IMO.
Also, if Deno the company company fails, what then will happen to Deno the project?
> what if you make a coding mistake which renders an unhandled exception?
AFAIK unlike node, Deno's API is largely promise-based and therefore all unhandled exceptions should become unhandled rejections withou crashing the process - so this should be less of an issue than it is with node (Of course, using older node libraries makes crashes more likely)
Perhaps you're right, I have not tried this out in Deno-land so I cannot say wether this actually applies to Deno the same way.
I just know that I got burned by this exact issue in node and I think this can be a security issue that people usually don't really think about. If you know that some api is a node api, if you manage to find a bug that is crashing the process one could probably script it to trigger it to crash over and over again. Even if it restarts quickly, usually it takes a second or and in that time one could bring the entire api down.
I am not aware of any instance that this have been used against some api in practice but most other languages handle this much better than node does and the javascript way of crashing on error is one of the things that made me question using javascript for backend services at all.
Regarding the kubernetes... no thanks. I would not go into that beehive unless forced. I didn't like the complexity increase of adding a load balancer and that is like a drop in the ocean of the complexity that is kubernetes.
You can, of course, make it more complex by piling a ton of stuff on top of it, and most cloud provider offerings do that. But its not required.
This option is also quite good for deployments as you can have instances stop reading from the port while client traffic is still being served from other instances on the port. This works well for HTTP requests, but less so for something like gRPC.
int sfd = socket(domain, socktype, 0);
int optval = 1;
setsockopt(sfd, SOL_SOCKET, SO_REUSEPORT, &optval, sizeof(optval));
bind(sfd, (struct sockaddr *) &addr, addrlen);
Edit: oh sorry, you meant when systemd is opening the port for you. It looks like you can set ReusePort=yes in your configuration? https://www.freedesktop.org/software/systemd/man/systemd.soc...