4,056 karma · joined May 31, 2015
I don't represent anyone and no one represents me
I have a thousand names
Since this place doesn't have DMs, I set up an email for this username at tuta (dotcom)
People have been doing some cool stuff for like a decade with giving dogs buttons to use human language, something they can seemingly get decent at communicating effectively with if they can get around the pesky issue of not having the sophisticated vocal machinery needed to produce recognizable phonemes, through the power of a good interface for them, even if the output is discretized to the level of words
I thought maybe this would be about creating a way for a dog to create stuff said dog might actually want or enjoy via the more powerful lever of effective long-context natural language processing that came of a similar tokenization approach - which can even sometimes churn out working code - that we have now
Instead it seems to be an exploration of how the capabilities you can produce from essentially random noise from this technology is less distinguishable from the result of thoughtful input than I might have hoped. Still interesting, but way less so
Don't get me wrong, I don't think you need or should need a degree to program, but if your standard of what abstractions you should trust is "all of them, it's perfectly fine to use a bunch of random stuff from anywhere that you haven't the first clue how it works or who made it" then I don't trust you to build stuff for me
I think LLMs as a technology are pretty cool, much like crowdsourcing is. We finally have pretty good automatic natural language processing that scales to large corpora. That's big. Also, I think the state of the software industry that is mostly driving the development, deployment, and ownership of this technology is mostly doing uninspired and shitty things with it. I have some hope that better orgs and distributed communities will accomplish some cool and maybe even monumental things with them over time, but right now the field is bleak, not because the technology isn't impressive (although somehow despite how impressive it is it's still being oversold) but because silicon valley is full of rotten institutions with broken incentives, the same ones that brought us social media and subscriptions to software. My hope for the new world a technology will bring about will never rest with corporate aristocracy, but with the more thoughtful institutions and the distributed open source communities that actually build good shit for humanity, time and time again
But on the other hand, I am used to arch, which both does package-management ala carte as a rolling release distro and has a pretty extensively-used secondary open community ecosystem for non-distro-maintained packages, so maybe this isn't as true in the "stop the world" model the author talks about
My experience with the linux ecosystem overall, which seems consistent with that of the person you're responding to from what little information that post gives, has been of consistent improvement over a long timescale with an increasingly capable stack of open-source software whose exact pieces have shifted with various community and maintainer dramas and the natural process of the birth of new projects and death of old ones over time. I've found that I have my preferences within that ecosystem, like I settled on archlinux as a distro about ten years ago and haven't really seen a strong reason to switch, despite periodically working with other popular ones in the course of a career as a software engineer and researcher. I have strong reasons to prefer a modular, composable operating system that I control, so I wouldn't consider using proprietary software if there's a working FOSS alternative. This is a bias for sure! But I find my frustration with these things has decreased in aggregate over time, even as I've changed tools and suffered switching costs for it numerous times, and dealt with the general hostility with which a lot of manufacturers seem to view open-source software running on their hardware, and their attempts to make this more difficult. However, the aggregate experience of proprietary software users seems to have significantly degraded over the same period. They generally insist that this is still worth it to them over doing what I do, and again I've been in enough dumb internet arguments to know that it's not worthwhile to do more than gently suggest that alternatives exist and may be worth trying unless I know them personally.
I do get a window into proprietary ecosystems nonetheless, because I still don't feel I can replace the use cases required of me on mobile phones with an open-source alternative yet, and have seen my frustrations steadily increase over time with both these and SaaS products that I've been required to use for work. I also got frustrated enough with game consoles that I've entirely switched over to using PCs, running linux, for any games I want to play. At every turn, I have found that while computers are always error-prone in some way or another, and using them extensively will result in some frustration, this is significantly less when I have more control over the computer, and has become less rather than more frequent as open-source projects mature. I can not only observe that my own experience with proprietary products has followed the opposite pattern, but that more and more people talking about tech companies with scorn rather than effusive praise, yelling at their phones, and the public discourse adopting terms like "platform decay", "enshittification", "tech rot", etc all suggest that this is a general trend rather than my biases
Again, your mileage may vary, but I do find it odd that you are so immediately dismissive of this perspective, accusing a pretty innocuous comment about it of reactionary identity-defense basically immediately without engaging at all. If you're inclined to listen to a zealot like me at all, I would only urge you to consider why you have assumed this so quickly, why you are so adamant that this is the only sort of person who could form such an opinion
I think the most I can say I've dove in was in the last week. I wrangled some resources to build myself a setup with a completely self-hosted and agentic workflow and used several open-weight models that people around me had specifically recommended, and I had a work project that was self-contained and small enough to work from scratch. There were a few moving pieces but the models gave me what looked like a working solution within a few iterations, and I was duly impressed until I realized that it wasn't quite working as expected.
As I reviewed and iterated on it more with the agents, eventually this rube-goldberg machine started filling in gaps with print statements designed to trick me and sneaky block comments that mentioned that it was placeholder code not meant for production in oblique terms three lines into a boring description of what the output was supposed to be. This should have been obvious, but even at this point four days in I was finding myself missing more things, not understanding the code because I wasn't writing it. This is basically the automation blindness I feared from proprietary workflows that could be changed or taken away at any time, but much faster than I had assumed, and the promise of being able to work through it at this higher level, this new way of working, seemed less and less plausible the more I iterated, even starting over with chunks of the problem in new contexts as many suggest didn't really help.
I had deadlines, so I gave up and spent about half of my weekend fixing this by hand, and found it incredibly satisfying when it worked, but all-in this took more time and effort and perhaps more importantly caused more stress than just writing it in the first place probably would have
My background is in ML research, and this makes it perhaps easier to predict the failure modes of these things (though surprisingly many don't seem to), but also makes me want to be optimistic, to believe this can work, but I also have done a lot of work as a software engineer and I think my intuition remains that doing precision knowledge work of any kind at scale with a generative model remains A Very Suspect Idea that comes more from the dreams of the wealthy executive class than a real grounding in what generative models are capable of and how they're best employed.
I do remain optimistic that LLMs will continue to find use cases that better fit a niche of state-of-the-art natural language processing that is nonetheless probabilistic in nature. Many such use cases exist. Taking human job descriptions and trying to pretend they can do them entirely seems like a poorly-thought-out one, and we've to my mind poured enough money and effort into it that I think we can say it at the very least needs radically new breakthroughs to stand a chance of working as (optimistically) advertised
My experience with FOSS has mostly been that mature projects with any reasonable-sized userbase tend to more reliably not break things in updates than is the case for proprietary software, whether it's an OS or just some SaaS product. YMMV. However, I think probably the most potent way to avoid problems like this actually ever mattering is a combination of doing my updates manually (or at least on an opt-in basis) and being willing to go back a version if something breaks. Usually this isn't necessary for more than a week or so for well-maintained software even in the worst case. I use arch with downgrade (Which lets you go back and choose an old version of any given package) and need to actually use downgrade maybe once a year on average, less in the last 5
I also think the predictions section seems kinda generic where the roast section felt better-personalized, which seems like a prompting issue
I guess in short, this quote brings into sharp focus how brainrotting all this information and curation is, automatic and pervasive as it's become
That said, I do think this article frames its advice in a clickbaity way by handwaving cumulative effort while talking about instantaneous effort
Sadly, the most reliable signal american tech companies send is that they are primarily concerned with building a surveillance state. Whether this is for the US government or just their own fiefdoms (franchulates?) seems to vary a lot both within and between them, but neither prospect is particularly appealing to me as a prospective customer and/or target