ibestvina

> ... until we at the very least give LLMs a direct way to interact with lab equipment and train them using RL on it.

Which is exactly what is being done.

https://www.reuters.com/world/anthropic-quietly-sets-up-biol...

On a related note - would it be easier, instead of doing a benchmark sweep across the whole NxN set of start-end pairs for which layers to modify, to instead measure cross-correlation between outputs of all layers? Shouldn't that produce similar results?

I think that "hovering the code" and "broken rhythm" are correlated, but still separate issues. There are parts of my code (e.g. vibe-coded mockups) that I don't look at the code of at all, and I don't care about it. There are others where I check everything in detail (same as if I were to review someones PR), and I think I have a very good grasp of that code.

But the broken rhythm problem persists regardless, and I find that issue to become more and more serious as LLMs are able to work for longer and longer on their own.

It might be that what we're experiencing now is just an uncanny valley, where they're not yet good enough for us managing them to work in similar ways as with other developers, but are good enough to allow us to switch our attention away from them while they work. But that attention span is mostly wasted, as the time between interactions isn't enough to e.g. work on something else, or read a book.

It's a stupid analogy, but currently it's similar to having a bathroom break every couple of minutes, and if this continues, most developers will probably start doomscrolling more and more.

I was wondering recently if there are some productive activities that might fit well into this rhythm, but I haven't found any yet. I guess sourdough baking is one such example, but there's only so much bread you can eat...

Exactly! I now feel bad for not thinking of that example, thank you.

There's a whole industry of "illusions" humans fail for: optical, word plays (including large parts of comedy), the Penn & Teller type, etc. Yet no one claims these are indicators that humans lack some critical capability.

Surface of "illusions" for LLMs is very different from our own, and it's very jagged: change a few words in the above prompt and you get very different results. Note that human illusions are very jagged too, especially in the optical and auditory domains.

No good reason to think "our human illusions" are fine, but "their AI illusions" make them useless. It's all about how we organize the workflows around these limitations.

This is very "distant" suggestion if you enjoyed Antimemetics, but The Unconsoled by Kazuo Ishiguro is another one of my favourites, and it too explores this idea of unreliable and inconsistent memories, although from a completely different angle.

First, that's quite a sad view of incentives structures. Second, you can't be serious in thinking that "worker worried they might be fired" puts the person in charge closer to the "materials" and more "hands on" with the project.

I don't see why his involvement, explaining to his team how exactly to build a piece, is any different from a developer explaining to an LLM how to build a certain feature, when it comes to the level of "being hands on".

Obviously I am not comparing his final product with my code, I am simply pointing out how this metaphor is flawed. Having "workers" shape the material according to your plans does not reduce your agency.

This makes no sense to me. There are plenty of artists out there (e.g. El Anatsui), not to mention whole professions such as architects, who do not interact directly with what they are building, and yet can have profound relationship with the final product.

Discovering the right problem to solve is not necessarily coupled to being "hands on" with the "materials you're shaping".

Gates of Heaven (1978)

https://www.imdb.com/title/tt0077598/?ref_=ext_shr

I always send myself messages as bookmarks, notrs, and general "here's an idea I might want to come back to later". It's an awful system when it comes to discoverability (I do it in Messenger so search is... bad), but I still do it as it's the most convinient option at the moment when I need to store that something somewhere.

So I built a simple Telegram bot which automatically stores anything I send as a text embedding into a vector database, and allows me to search over it in that same chat (same process that powers the AI Q&A assistants these days).

If I post a link, it automatically scrapes it and stores text as chunks for better search, extracts text from youtube videos (still wip), turns images into text with the visual models, etc.

One thing I'm unhappy about is not being able to easily edit any notes I search for later, but it's miles ahead of my previous "system". Hopefully I can open source this when I clean it up - if anyone is interested, let me know.

How do you look at hiring "experienced people" vs. "enthusiastic interns" on something like this? More generally, how quickly do you think the team will grow, and what the ratio should be between the "old" and the "young"?

Could you give an example of content where such critical distilation would be useful to you?

Interested in this as well - do you already work on this somewhere out in the open?

As someone who daily sends messages to myself as a form of notetaking, for everything from todos and shoplists to ideas and bookmarks, I've thought about this a lot. The only conclusions I came to are that I keep using messages because (1) they are convinient and synced across devices, (2) app is already installed and frequently opened so it rarely needs to load up making notetaking faster, and (3) I do not have to think about organization.

I believe the last point is most important, and is where your app is misaligned with my reasons for using messages - you seem to have spearate folders and notes, prompting me to think "where does this go?" before writing, which is something I really want to avoid.

Great work! Any plans to open up an API?

I'm always drawn to these types of initiatives, and your whitepaper looks (at least with my very limited knowledge of the domain) interesting.

What I am always wondering, and maybe you can give some details here, is the following: isn't the fact that regulations are in natural language, with all its ambiguity, a necessary requirement to have the system operate without being fully specified?

In other words, wouldn't any kind of strict DSL force us to think through all the edge cases that might possibly arise, instead of dealing with them when they do arise, which is basically what the judiciary is for? And isn't that a price too high to implement these kinds of systems?

So if the product from OP used Azure OpenAI, it would be okay? You say "companies happily pay up", but the pricing is exactly the same (source: my company is paying for both APIs).

It's been quite clear for some time that, between OAI and MS, they very neatly split their market: OAI handles the early development and direct customers, and MS handles enterprises. It would require OAI to be a much bigger company than it is right now to properly handle enterprises, and MS already has all that infrastructure (legal, support, etc.). Seems like a sensible setup to me, and I don't see the need for enterprises to run open source models themselves (in this context - of course I see the value in all the other respects about lock-in and specialization), especially if they are already on Azure.

That (LLM -> SQL -> Pandas using PandaSQL) was my approach with Datasloth [1] some time ago. It then grew into Qluent [2] which I now work on full time, along the lines of what the sibling comment wrote about managers being able to analyze their data without coding.

[1] https://github.com/ibestvina/datasloth

[2] https://qluent.com/

Thank you for the information. I'm sure there are a lot of people who'd be interested in the full 1 hour talk, if you ever find time to film it.

What an amazing person. I first learned about Clifford when he did his TED talk, and I was immediately a fan. The quote from the Hayes Hall Bell Tower resonated, and stayed with me, for all the years since. I'm deeply grateful that people like him exist, sharing their experiences and knowledge while still being simply kind and approachable. If you're reading this, thank you!

On the flipside, it might create more demand for some engineers (or skills in general) as the bottlenecks move "upward" on the skills ladder (if we define that ladder as "how hard is it for AI").

E.g., faster and cheaper compute created more demand for software developers as that demand was no longer capped by the compute bottlenecks. Similarly, faster and cheaper "basic" software development might create more demand for software architects, and so on.

Incredible work, and great explanation, thank you. Could you comment more on the cost of running this (e.g. per query or per hour, or however it's set up)? Where are you running the model from?

Also, could you provide more details on the cross encoder used for reranking?

Btw, if you already have all these song embeddings, it would be very interesting to be able to pick a song, and get all the similar ones in a playlist (sort of like "Song radio" on Spotify)!

We are currently building something similar at Qluent [1], and are considering expanding into the usecase you described. If you find it interesting, I'd love to talk about your exact requirements. You can find me at ivan (dot) bestvina (at) qluent (dot) com.

[1] https://qluent.com

Great work, and a really interesting application of GPT3. Some time ago I developed Datasloth [1] which might be a nice complementary feature to Sketch. Ping me if you're interested to bounce ideas :)

[1] https://github.com/ibestvina/datasloth

Here are the stats :)

   - Found the optimal path 504
   - Found a (non-optimal) path 293
   - Correctly reported no path exists 131
   - Found a solution when none existed 19
   - Ended on wrong node 0
   - Used edges that don't exist 53
   - Started with the wrong node 0
   - Incorrectly reported no path exists 0
   
   - Total:  1000

Here is the code: https://github.com/ibestvina/gpt3-graph-search

Great writeup! I made a couple of changes in your code and it seems it gets ~90% of examples correct (inside the first three categories you have) with aroung 65% having either optimal path or correctly saying that there is no path.

What I changed was:

1. Used code-davinci-002 (codex)

2. Instead of using an explanation of how these tasks work, I changed your code prompts slightly and instead gave it 3 examples (two with paths, and one to show it what to output when there is no path present).

3. Changed the output so that it has to tell me which edges, instead of which nodes, it is traversing (this helps GPT to avoid using nonexistent edges it seems).

Here is an example: https://pastebin.com/D7Hn95VC Last "Problem" is the real problem we want to solve of course, everything else is static.

I'll post my code when I finish testing :)

EDIT: I used Codex not because I think it's better suited for this, but because it's currently free. I don't have enough credit ATM to run 1000 iterations with text-davinci, so I have no idea what difference using Codex made.

Thanks! I found this project[0] recently that does what you described, but it is not open sourced.

Maybe if I get around to it, I might build a UI.

[0] https://www.askedith.ai