https://lord.io/spreadsheets/ is one of my favorite technical blog posts of all time. Highly recommend everyone check it out!
1,474 karma · joined February 2, 2012
You’re right that there’s a voice alert. But TCAS also has a map of nearby planes which is much more “HUD”! So it’s a combo of both approaches.
(Interestingly it seems that TCAS may predate Weiser’s 1992 talk)
Recent coding interfaces are all trending towards chat agents though.
It’s interesting to consider what a “tab autocomplete” UI for coding might look like at a higher level of abstraction, letting you mold code in a direct-feeling way without being bogged down in details.
In the section on dynamic documents towards the end of our essay, we show several of our lab’s own takes on this category of tool, including an example of integrating AI as an optional layer over a live programmable document.
To answer your question: although we use Patchwork every day, it’s currently very rough around the edges. The SDK for building stuff needs refinement (and SDKs are hard to change later…) Reliability and performance need improvement, in coordination with work on Automerge. We also plan to have more alpha users outside our lab before a broader release, to work through some of these issues.
In short, we feel that it’s promising and headed in a good direction, but it’s not there yet.
Coauthor here -- did you catch our section on AI? [1]
We emphatically agree with you that AI is already enabling new kinds of local software crafting. That's one reason we are excited about doing this work now!
At the same time, AI code generation doesn't solve the structural problems -- our whole software world was built assuming people can't code! We think things will really take off once we reorient the OS around personal tools, not prefabricated apps. That's what the rest of the essay is about.
[1] https://www.inkandswitch.com/essay/malleable-software/#ai-as...
AFAIK, atproto is primarily designed to support multiple distinct clients over shared data, but I also wonder if it could help with composing more granular views within a client. I previously worked on a browser extension for Twitter, and data scraping was a major challenge - which seems easier building on an open protocol like atproto.
Sorry we didn’t mention — it is on our radar but we ran out of space and had to omit lots of good prior art..
I should also mention btw that Bluesky user-configurable feeds is a perfect example of a gentle slope from user to creator!
On that note, Robin Sloan has a beautiful post about software as a home cooked meal…
https://www.robinsloan.com/notes/home-cooked-app/
That said, I think talking about cars may be stronger ground for the argument you’re making. Mass production is incredible at making cheap uniform goods. This applies even more in software, where marginal costs are so low.
The point of our essay, though, is that the uniformity of mass produced goods can hinder people when there’s no ability to tweak or customize at all. I’m not a car guy, but it seems like cars have reasonably modular parts you can replace (like the tires) and I believe some people do deeper aftermarket mods as well. In software, too often you can’t even make the tiniest change. It’s as if everyone had to agree on the same tires, and you needed to ask the original manufacturer to change the tires for you!
You make a fair point! Ease of use matters. We all want premade experiences some of the time. The problem is that even in those (perhaps rare!) cases where we want to tweak something, even a tiny thing, we’re out of luck.
An analogy: we all want to order a pizza sometime. But at the same time, a world with only food courts and no kitchens wouldn’t be ideal. That’s how software feels today—-the “kitchen” is missing.
Also, you may be right in the short term. But in the long run, our tools also shape our culture. If software makes people feel more empowered, I believe that’ll eventually change people’s preferences.
A couple reasons we've decided not to share the demo widely: 1) it's research software, not developed to the quality standards of a commercial product, so we don't want people to get confused or disappointed by that, 2) the prototype heavily uses the paid Google Maps API.
We've also publicly released demos of some related work, like Potluck, an interactive medium built on text notes:
Claude Code did great and wrote pretty decent docs.
Codex didn't do well. It hallucinated a bunch of stuff that wasn't in the code, and completely misrepresented the architecture - it started talking about server backends and REST APIs in an app that doesn't have any of that.
I'm curious what went so wrong - feels like possibly an issue with loading in the right context and attending to it correctly? That seems like an area that Claude Code has really optimized for.
I have high hopes for o3 and o4-mini as models so I hope that other tests show better results! Also curious to see how Cursor etc. incorporate o3.
Yes, it's absolutely great to have a "low floor" -- common use cases should be easy to do, without needing to learn a ton up front. But, hiding the structure is not the only way to achieve that!
For example: a microwave could have presets like "we recommend cooking a potato with this power for this time. If it's undercooked, try higher power, but avoid max power because XYZ."
The simple use cases should guide the user while building up a coherent mental model. If there are fancier sensors being used, those could be explained and exposed to the user directly.
Otherwise you end up with no path to further learning. If I have no idea how potato mode works, then I don't know when it applies or doesn't apply, I don't know how to adjust when it doesn't work well, and I don't know how it relates to the other modes at all.
https://en.m.wikipedia.org/wiki/Choreographic_programming
I’m curious to what extent the work in that area meets the need.
One reason: while I still think there's great potential in browser extensions, I ultimately felt limited by scraping and fetching bits of data from a server, and grew more interested in a deeper overhaul of how we build applications in order to locate more data on the client. Eg: check out this demo [1] of a Wildcard-esque data manipulation but in an app that stores all its data in a client-side database. I'm still continuing this work today, exploring "local-first" architectures at the Ink & Switch research lab, in support of malleable customizable interfaces.
It would be interesting to revisit Wildcard now that LLMs exist—I suspect that many of the scraping parts (building "site adapters" in Wildcard parlance) would be far more possible to automate now. A few years ago we explored some semi-automated programming-by-example interactions for scraping [2] but the friction was always really noticeable without any AI support.
[1]: https://riffle.systems/essays/prelude/#data-centric-design-e...
[2]: https://www.geoffreylitt.com/wildcard/Joker-LIVE-2022.pdf
1) AI
AI is rapidly getting better at coding. Current AI is often bad at high-level architecture but is capable of making small local tweaks. Seems like a good fit for the kind of code you need to write a browser extension!
I'm exploring this direction; wrote more about it in "Malleable software in the age of LLMs" [1]
2) Security
Having talked to people who worked on various extension platforms including the browser extensions API, I see more clearly than I did five years ago that security is often the key bottleneck to deploying extension platforms meant for mass adoption. Anytime you want everyday computer users to be installing invasive extensions to important software from untrusted third parties, it's gonna be challenging to protect them.
That said, I still think that conversations around extensions tend to focus too much on security at the expense of all else. Customizability is important enough that it may be worth prioritizing it over security in some cases.
I also think there are many reasonable paths forward here. One is to exchange extensions with trusted parties -- e.g, coworkers or friends -- rather than installing from random people on the internet. Another might be to only build your own extensions; perhaps that'll become more viable with AI-assisted programming, although that introduces its own new security issues. And finally, I've met a few people who have smart ideas for architecting software in a way that helps resolve the core tensions; see [2] for an example.
3) Backend access as a key limitation
I've increasingly realized that the fact that browser extensions can only access client code in a fairly server-centric web means that many deep customizations are out of reach. Perhaps you can't read the data you want, or there's not a write API to do the thing you need.
While I'm optimistic about what extensions can do within the boundary of the client, this is an inherent limitation of the platform.
At Ink & Switch (the research lab I now work for), we're working towards local-first [3] software: collaborative software where the data and the code lives on your device. Among other benefits like privacy, we think this is the right foundation for more powerful extensions, since your data and the app code aren't locked away on a server.
[1] https://www.geoffreylitt.com/2023/03/25/llm-end-user-program...
[2] https://www.wildbuilt.world/p/inverting-three-key-relationsh...
I’m excited about the ongoing push towards cross-browser standards in this area; Safari has become way more compatible recently as well.
Ultimately, in the long term, I think the filesystem probably provides the wrong abstraction for this use case though. The API we really need is "make these changes", (w/ changes represented thoughtfully in a mergeable way) not "here's the new final state." For now, diffing filesystem states is a reasonable workaround.
Yjs is an impressive project and probably the most production-ready text CRDT library out there today.
One of our goals was to write down a clear spec for how a rich text CRDT should behave separately from any particular algorithm, as a resource for people working on tools in this space.
In the process we also found some examples where we felt Yjs didn’t maximally preserve intent. One of them is documented in the paper in the section about control characters, and Peritext handles that case differently.
Also, as an implementation note, our prototype sends incremental changes to the Prosemirror editor, whereas I believe yjs sends changes that replace the entire document, which can break the behavior of other plugins [1].
[1]: https://discuss.prosemirror.net/t/offline-peer-to-peer-colla...