I don’t know, somehow the pace seems different, these transitions felt more like the last generation retiring, and the next generation doing something different, instead of it happening to everyone at the same time.
I think these are different things. The source code, it’s tests, it’s documentation, it’s data. They are different perspectives to approach a given body of work while together building an overall understanding. It doesn’t matter at all where the files live but when you approach the work from that perspective you know how to find it, and I don’t think next to the source but away from the others is a good combination.
This particular example aside, I don’t think it being derivative and simplified is necessarily bad. Libraries that are popular today were written for humans and reinforced by LLMs via training. It’s unlikely they represent the ideal interaction surface for an agent.
There was a study recently that LLms prefer resumes written by LLMs rather than by humans. Stands to reason they would prefer apis written by LLMs.
This is probably the early days of such intentionally simplified agentic semantic primitives like “DAG Workflow” where the answer for why not Temporal is that LLMs prefer different things than humans.
Is the difficulty that in high entropy situations, you can’t really tell whether it’s because the model is uncertain, or because of the options are so semantically similar that it doesn’t matter which one you choose? Like pure synonyms.
I’ve been experimenting with LiveStoreJS which uses a custom SQLite WASM binary for event sync, so for simplicity I’ve also used it for regular application data in browser and found no issues (yet). It surprised me that using a full database engine in memory could perform well vs native JS objects at scale but perhaps at scale is when it starts to shine. Just be wary of size limits beyond 16-20mb.
Of all the interface modalities available, CLIs seem like the most natural for copilots to work with. Lots of examples in the training data, universal interface for help, maps well to the sequential nature of token generation, similar syntax for different OSs… I can see them replacing skills and MCP et al from the model’s perspective.
How do they prove this? It sounds like the plaintiffs basically claimed they were rejected a bunch of times and since the resume had recognizable indicators of protected classes they must have been discriminated against?
Don’t get me wrong, I do this work, and Workdays statement of “we don’t use protected classes” instead of “we test our models to prove they are unbiased when given recognizable indicators of protected classes” is pretty telling. Because it’s hard and if you solved it you would be proud. If you don’t control for it it WILL discriminate. See Amazon’s experiment a decade ago.
I’m just really curious how all this plays out in front of a judge.
This is a great use case for using an algorithmic difficulty ramp where it can really dial in that curve to solve for getting people to play longer over multiple sessions.
I’ve built lots of pre-LLM data processing pipelines like this and the more I read people putting “agents” into this kind of context the less they resemble agents like the Anthropics of the world defines and the more they just resemble functions. I wonder if eventually there won’t be a distinction and it’ll just be a way to make processing and branching nodes in a pipeline less deterministic when you need more flexibility than pure code-rules can give you.
Isn’t this an arbitrage opportunity? Offer to pay a fraction of the cost per token but accept that your tokens will only be processed when the batch window isn’t big enough, then resell that for a markup to people who need non-time sensitive inference?
What kind of customers are using this API? I’ve had many similar thoughts but I get hung up on the idea that customers are “developers” from a marketing standpoint, because those developers are developing something and that something is probably a bigger driver of utility that a truly generically developer tool like Cursor.
If I’m understanding this correctly, if you set off one bomb the explosion travels at a certain speed, but if you put a bunch of bombs in a circle and set them off one at a time, the very last explosion will be going a lot faster than the first one?
There’s a trope where the best white hat is a former black hat because they can recognize all the tricks, I wonder if training an LLM to be evil and then fine tuning it to be good will produce more secure code than the opposite?
FWIW I’ve been doing a lot of work with text to SQL and in that space verbosity in naming tables and columns matters a lot because it adds additional context about what data is in the table and how it can be used. Think “subs.id” versus “subscriptions.stripe_customer_id”.
Hi. Maybe anecdotal, Brother MDC-J480DW, had it about 3 years, always bought third party ink, two weeks ago I had to replace the color ones and it started saying there was no ink even though ink was clearly visible, it failed on two different colors from two different sources, and the three different black cartridges, then bought legit from Walmart and worked perfectly. I can’t speak for everyone’s experience, but mine was definitely changed recently to always say third party ink was empty, and I’ll junk it once this ink runs out.
I’ve been working on some tooling to create better views of analytics based on the idea that people don’t like to write joins so why not create logical views that do all possible joins. Appreciate any feedback. https://github.com/eloquentanalytics/pyeloquent
I think a lot of times this is not about skill sets but more that data engineers don’t build datasets with UX in mind. The examples in this piece are not what show up when a leader browses a real database with hundreds of tables stuffed with abbreviations and numeric/short codes. If you want your leaders to use your data, you have to design your data to be used by leaders, not teach them SQL and statistics.
I often wonder if you can create databases like this using the data in the training datasets of LLMs. Generate the list of spices by asking for categories, countries of origin, etc and then asking about a list of properties for each. You could use these kinds of Wikipedia lists as a validation mechanic.
Enhancing the comments on the existing data model seems to be the most common approach for sure. I'm implementing this as a data architecture at several clients and I've found creating a whole new logical structure designed for the LLM is really effective. Not being bound by the original data model lets you solve several problems related to the "n-hops" question, avoiding needing the comments, and the semantics of how data engineers define columns. Some more details here [1], but obviously you can implement this totally yourself by hand.
I’ve been doing a lot of work on semantic data architecture that better supports LLM analytics, did you use any framework or methodology to decide how exactly to present the data/metadata to the LLM context to allow it to make decisions?
I often wonder if NL-SQL tasks would benefit from an intermediate query language that is more compatible with the next-logical token approach that is used to generate the code. Obviously there is less of this in the training set, but if it transpires in a testable way, you could generate training data yourself from known good sql queries? Are there any languages that have been designed specifically for this?