seldo

For agentic use cases, where you might need several round-trips to the LLM to reflect on a query, improve a result, etc., getting fast inference means you can do more round-trips while still responding in reasonable time. So basically any LLM use-case is improved by having greater speed available IMO.

We are planning to move our blog off of Medium (we've been busy!), but this post is public so you can actually just click through the nag screen if you see one.

Retrieval-Augmented Generation, where you ask an LLM to answer a question by giving it some context information that you have retrieved from your own data rather than just the data it was trained on.

LlamaIndex lets you attach metadata to Nodes which are basically chunks, although that fact is poorly documented! Will fix.

This really isn't news, folks. It happens every week. I was just grumpy this morning.

Further clarifying: npm will revoke all tokens issued before 2018-07-12 12:30 UTC. If you rolled your tokens after that time you will not need to re-issue them.

https://status.npmjs.org/incidents/dn7c1fgrr7ng

Update: (this is not the post-mortem, this is just more detail) http://blog.npmjs.org/post/169432444640/npm-operational-inci...

We're working on a full post-mortem now. Until then we don't want to give out misleading/partial information.

Absofuckinglutely. It's being done as we speak.

Hi folks, npm COO here. This was an operational issue that we worked to correct. All packages are now restored:

https://status.npmjs.org/incidents/41zfb8qpvrdj

I think you should probably re-examine your priorities in life, but that's just me. You do you.

Yarn does not run a mirror of the registry. registry.yarnpkg.com is a pass-through domain to the npm registry. It allows them to collect stats about yarn usage but is not a mirror.

L1 visas are easier for the company to get, because there is no cap on the number issued. The employee on an L1 is just as qualified as an H1, but less free -- they cannot switch jobs, and if they get fired they must leave the country within 15 (!) days.

I'm not sure where you've interviewed but everywhere I've worked we always ask what you've done, and at my current company we have no whiteboard interviews. Tech is learning, slowly, how to give proper interviews.

Funny you should mention a managed relational database service; Instapaper uses one of those and had more than 12 hours of downtime this week: http://blog.instapaper.com/post/157027537441

No database solution is totally reliable. If storing data is my primary job, like it is GitLab's, I'd like to have as much control of it as possible.

A few points: as @chrisfosterelli noted, if a package is being squatted we'll give it to you, no problem.

As for why we don't automatically/proactively handle squatting: it's a very thorny problem. Whatever minimum standard we applied to count as "not squatting" could be trivially discovered and gamed, eventually resulting in people who wanted to squat on a name just publishing a copy of `express` or something to that name as a placeholder.

Relatedly: you can report offensive, or deliberately confusing package names ("typosquatting") and we will take those package names down permanently.

How would you like us to better publicize it? I agree it's a few clicks away from the home page, but as others have noted not a lot of people need to enter this process, so giving it prominence on e.g. every package page would be overkill.

How does one contact the moderators to let them know that a comment is racist/sexist/whatever? I don't see any "flag" UI at the comment level, and I don't know who the mods are by name.

Reminds me of this great presentation: http://lunar.lostgarden.com/Rules%20of%20Productivity.pdf

Especially this graph of productivity over time: https://slides.com/seldo/makersquare-6-stuff-everybody-knows...

TLDR: if you crunch for 4 weeks it will take you so long to recover that it'll be as if you never crunched.

Check out my detailed answer a few comments down: https://news.ycombinator.com/item?id=12861180

99.9% of our requests are handled by the CDN. The CDN doesn't cache 404s, so 404s are handled by our origin servers, which are much fewer in number and therefore quite easy to overwhelm.

You're right that our handling of 404s was naive, and that's definitely something we'll be improving as a result of what we've learned from this incident.

More efficiently handling 404s, which as many have pointed out we were handling quite naïvely.

I'm sorry my response looked like I was blaming them, that wasn't my intention. Like I said, it was an honest mistake: these things happen, and they handled it well.

Once we determined 404s were the problem we put mitigation in place that worked fine, but the problem of request volume remained: the 10% figure I gave was at a 5% rollout of VSCode. A full rollout would therefore have meant the registry became 3x bigger overnight and two thirds of that would have been 404s to VSCode users. At that point the issue is financial, not technical, which is another reason the rollback happened.

I was a bit vague :-) India's about 10% of total requests on any given day. VSCode was 10% of requests for a couple of hours.

A VSCode person can (and probably will) answer in more detail, but at heart it's simple: if you want to add type-checking goodness to a library that isn't itself written in TypeScript, you can create a thing called a declaration file: https://github.com/DefinitelyTyped/DefinitelyTyped

Microsoft publishes a list of known good declaration files for popular npm packages to npm, under the scope @types: https://www.npmjs.com/~types

The 1.7 release of VSCode helpfully tries to automatically load type declarations for any npm package you use by requesting the equivalent declaration package under @types. When the package exists this is fine, because it's cached in our CDN.

What they forgot to consider is that most CDNs don't cache 404 responses, and since there are 350,000 packages and less than 5000 type declarations, the overwhelming majority of requests from VSCode to the registry were 404s. This hammered the hell out of our servers until we put caching in place for 404s under the @types scope.

We didn't start caching 404s for every package, and don't plan to, because that creates annoying race conditions for fresh publishes, which is why most CDNs don't cache 404s in the first place.

There are any number of ways to fix this, and we'll work with Microsoft to find the best one, but fundamentally you just need a more network-efficient way of finding out which type declarations exist. At the moment there are few enough that they could fetch a list of all of them and cache it (the public registry lacks a documented API for doing that right now, but we can certainly provide one).

CDNs don't usually cache 404s. VSCode was looking for @types packages for any and every npm package its users were using. Packages that had a type description caused no issue, but most packages don't, so we had a > 1000% spike in 404s. Our workaround before MS did the rollback was to cache 404s for @types packages specifically, and it was effective enough that the registry never really went down.

I'd just like to say on behalf of npm that Microsoft's handling of this incident was A+. As soon as we alerted them to the issue they were all hands on deck and did a rollback.

We've been really pleased that Microsoft chose to put their @types packages into the npm registry rather than a separate, closed system, and in general happy with Microsoft's support of node and npm. We're confident we can make the new features of VSCode work, we just need to work with Microsoft to tweak the implementation a little.

This was an honest mistake on their part, and we caught it in time that there was very little impact visible to any npm users.

Fun fact: at its peak, VSCode users around the world were sending roughly as many requests to the registry as the entire nation of India.

You make a good point. In practice, we've not found a reliable way to weed out the false positives that result (I would not characterize it as an "explosion", but there are definitely some). However, we've stuck with this process because we firmly believe that the benefits of having the excellent people we've hired who would not have passed a standard interview outweigh the costs of getting rid of the others.

Doing what you're doing will probably work, in that you'll get decent candidates. But this approach only works for one kind of good developer -- the kind who can survive this kind of interview. Silicon Valley's diversity problem is made worse by interview styles that give false negatives for excellent candidates who lack confidence or just don't think the same way. My thesis is that this test, by giving frequent false negatives, is resulting in a worse pool of final hires than a broader process.

It is possible to disable install hooks at install time by running npm install with --ignore-scripts.

You can also make this the default, with npm config set ignore-scripts true (and then --ignore-scripts false at install time if you wish to run them).