nimski

Yes, although the folks that said "this is the worst it will ever be" have some evidence in their court. The best models now are much better than the best models a year ago.

Some big gnarly problems remain (e.g. continual learning), but I think the fact of AI existing and being better than humans at cognition no longer seems like something out in the far future.

Because I'm guessing your passion was for the craft and not just the result.

Before there were factories, if you wanted a chair you needed an artisan to make one for you. When factories pumped out chairs, there was no need for the artisan to make chairs. But building factories that made chairs was a craft in itself. If you wanted a factory to make chairs, you needed an artisan to set one up for you. A craft became redundant, but enabled another to exist.

It's natural to mourn a craft that becomes redundant. But such an event is not unique.

I do think there is something different about AI, though, in that it attacks the craft of cognition itself. At some point there's no room for a human artisan to build on top. The required faculties become too great for any human mind.

nkeivan.com

bravo

This has been the thesis behind our product since the beginning (~3 years), before a lot of the current hype took hold. I'm excited to see it gain more recognition.

Chat is single threaded and ephemeral. Documents are versioned, multi-threaded, and a source of truth. Although chat is not appropriate as the source of truth, it's very effective for single-threaded discussions about documents. This is how people use requirements documents today. Each comment on a doc is a localized chat. It's an excellent interface when targeted.

Karpathy recently tweeted about the importance of being close to the data when training LLMs, and I think a similar level of rigor and transparency is essential when judging evaluations. Thanks for bringing us this insight from the data.

Building hierarchical abstractions on top of code is IMO the only way to truly enable AI to write code beyond better autocomplete. Kevin's work with DreamCoder shows that hierarchical abstractions can be built automatically in the code domain.

Once these abstractions exist side-by-side with natural language (essentially a code-natural language world model), it'll enable arbitrarily complex code generation from descriptions of the outcomes/results.

This is what we're doing. I think it's not only possible, but also necessary for applications beyond trial-and-error generation.

Is that really that much of a barrier? Off the top of my head: you could start with a prompt like "write a summary of book XYZ, followed by a summary of each chapter". Then dive deeper into each one from there using the same prompt recursively, etc.

I appreciate the example, but here's where I think it differs as an analogy of what LLMs do:

A summary doesn't have infinite or variable depth. If you read the summary of a non-fiction (I'll limit my argument to that, as another poster pointed out) book, and either aren't convinced, or want to learn more about the matter, you'd have to purchase the book.

An LLM that has ben trained on the book, if somehow designed not to hallucinate, would be able to answer any question you have about the book at any depth, seamlessly blending in material from other books to answer a question or explain a concept. That seems like an entirely better experience than reading the book from start-to-finish. I don't see how the original can compete.

You're right. I think it's fair to carve out fiction from my argument. For that, I would surely go to the source material until the point where the LLM was coming up with better long-form fiction de-novo. But for non-fiction, which I would guess is the economically and intellectually more important category to protect, the effects may be devastating.

I also agree that attribution can't be solved easily in the current paradigm. Perhaps, during training, one could deduce how much of the net gradient on a particular weight was derived from the batches covering some book, and then during inference, assign attribution based on the effect of that weight on the output. All of this is very expensive to do, and I don't have strong intuitions for whether the resulting attributions would be in any way meaningful.

To your point about hallucinations, if there's not a solution to that, then perhaps the whole point is moot when, after a while, the hype dies down. But if somehow hallucinations are solved (I don't see a technical way this can happen now, but who knows?), then I think we'll need to address attribution for non-technical material.

"The difference, when it comes to AI, is one of scale. ChatGPT can “read” more published words in a few seconds than I could in several lifetimes and, unlike me, that data isn’t immediately replaced in my human-limited short-term memory by whatever I’m thinking of next."

I think this misses the point. The issue of scale isn't on the ingest side, it's on the output side. Once you train an LLM on a book (however long that takes), then the LLM can be the interface to that book for an unlimited number of users. That scales very differently to, say, a person reading a book and writing something influenced by it.

In the case of the LLM, it's a complete interface to the contents of the book. It lets you "talk to the book". If that exists, why would anyone buy the book? If I could ask ChatGPT to "summarize the new book by XYZ", then spend an hour or two asking the questions _I_ have about the book from it, then buying the book would be a net negative.

If we don't solve attribution (like BMI solved for music), then the financial upside of publishing might be majority-captured by whoever trains LLMs on the copyrighted material.

The addition of many remote positions since COVID has been a great hiring filter for us. For the type of work we do (AI, R&D), and the culture that we find most productive and enjoyable (enthusiasm because we love the work, a sense of working in a team), remote was a real downgrade when we tried it.

We advertise the job as on-site only, and because of that the applications self-select for those that want in-office work. It's made our interviews more focused on technical ability.

I think this is a better equilibrium overall. Those on either side of the remote/on-site preference can find the right respective jobs and work cultures.

It is possible to build technical solutions that take the work and luck out of social organization by shouldering the heavy lifting, but then get out of the way once people meet IRL.

The best tool for this right now is calendar and email. There's a lot of room for improvement.

Please PM me if you're interested in a deeper discussion. I've thought a bit about what this tool might look like and how to get it off the ground.