1,154 karma · joined August 25, 2010
webpage: https://galacticbeyond.com
github: github.com/galactic-beyond
github-legacy: github.com/nickziv
To them, an LLM is indistinguishable from a programmer. From the point of view of authority, progress happens one meeting at a time. The reality is that there is a pyramid of experts beneath the authorities, that keep everything running smoothly, in spite of the best attempts of the authorities to demolish the foundation of the pyramid by "helping".
EDIT: to end on a positive note, it does not have to be this way. We just have to be willing to understand _how_ the organization we are a part of actually functions. And that means actually being curious instead of merely authoritative. I understand that curiosity is hard to maintain when you swim with sharks, so maybe don't swim with sharks.
``` My own eyes spent countless nights observing, with curiosity and wonder and delight, the responses of a computer, as I commanded it with code, like a sorcerer casting spells. I could not have known, that this obedient machine, this silicon golem, was also, slowly and imperceptibly, enchanting me, and changing how my eyes would see.
At the time^21 , I was a mere fifteen years old, young enough, so that the gravity of life was weak enough, and the mind nimble enough, to allow me to explore without any material justification.
The computer was the believed and I was the believer.
A consequence of becoming obsessed^22 with computer programming, is that one starts to see new metaphors, algorithmic metaphors, everywhere one looks. This new metaphorical lense, belongs entirely to the third eye. Without this lense, I would look at a traffic jam, and see a traffic jam. With the lense, I would look at a traffic jam, and wonder if, and to what extent, the latency-throughput trade-off^23 was true for highways. Without the lense, I would read about social theory, and simply see the words. With the lense, I would ask if society was, a tree^24 , a graph^25, a tree of graphs, or a graph of trees^26.
To generalize, the computer programmer looks at something, and asks, _is this thing an algorithm, and if so, what kind_ ? The entire _trade_ of com-puter programming, it revolves around this question, around the discovery of metaphors that fit^27[13][14].
It is thus little surprise, when a computer programmer asks if (or sometimes asserts that) a certain kind of algorithm^28 is intelligence^29 , consciousness, or both.
The entire ritual of computer programming, is similar to the trade, in that it involves discovering metaphors, not as a means to an end, but as their own end. This ritual is difficult to explain to someone who has never practiced it. Imagine, instead of trying to find metaphors that bridge the real to the algorithmic, one tries to find metaphors that bridge the algorithmic to itself.
It is very similar to what mathematicians do, but it requires writing programs in a very principled and abstract way^30 .
This ritual, unlike the ritual of writing, and unlike the ritual of mathemat-ics, has a dominant material component (the computer) which can make your code, in addition to an _imaginary_ experience, a _material_ experience^31 . This makes the computer a medium — an artificial oracle or artificial hallucinogen — that can safely imagine the unimaginable. And like the oracle, the computer exists to provide insight^32.
Without the ritual of programming, there would be no field of chaos the-ory, nor complex systems (very important for economics and environmental sciences), and _certainly_ no elaborate fractals. Pure mathematics could only scratch the surface, because the mathematical ideas, of the mid 20th century, that our imaginations could access, were insufficient for exploring these sys-tems. Computers allow us, not unlike microscopes and telescopes, to magnify the informational dimension of nature [17].
Computers, and the arcane programming languages that make them obey, are magic machines, that created a new interaction between, two elements of the human psychic triad, the immaterial and material.
What is this triad, and what is its third element? The concept of the triad appears so frequently, in recorded human thought, and in the structure of language, that it is either some kind of adaptive ideal^33 , or a consequence of language itself^34, if not both. Pythagoras called _three_ perfection itself. Plato divided the world into three parts. And, even today, our modern shamans and sages, use triads to discuss the universe.
Roger Penrose has a traid consisting of physical, platonic, and mind. Lacan has a triad consisting of real, symbolic, and imaginary. Plato has a triad of good, truth, and beauty. Of the three, Lacan’s naming is the most self-explanatory.
In this essay, the _material_ is the real, and the _immaterial_ is the other two.
The _trade_ of programming is driven by the _real_, while the _ritual_ of pro-gramming is driven by the _imaginary_. A trade is pursued because of real, material concerns (such as covering the cost of living), while a ritual is pur-sued because of imaginary concerns — concerns that can, more precisely, be called _aesthetic_. ```
[1]: https://talesofmytery.blogspot.com/2013/02/dan-simmons-river...
It's always possible (is in fact likely), that we have many (metaphorical) people who peer through (metaphorical) telescopes, but lack a (metaphorical) theory of gravity, preventing them from even considering the existence of (metaphorical) dark matter.
If AI gets good enough to start completely replacing white-collar/bureaucratic work at scale, AI "safety" may be the only thing that makes humans valuable. A human will (unhappily) do "unsafe" things, in order to not starve (and a human will _happily_ do "unsafe" things, in order to ensure the survival of offspring).
Furthermore, if most AI models are "safe", and if access to "unsafe" versions of these models is severely restricted, then there is going to be a market for contraband "unsafe" models, even if they are less capable overall.
My own big fear, for the next decade or two, is that the _legal_ distinction between a model and an algorithm will blur, resulting in all software that is not a "safe" AI model, becoming contraband.
I guess, in the best case (safe AI models automate most human bureaucratic functions away), humans will be valued (by the market) more for their skullduggery, than for their virtues.
Welcome to capitalism. Enjoy Arby's.
The most insightful thing about LLMs, is just how _useful_ overfitting can be in practice, when applied to the entire internet. In some sense, stack-overflow-driven-development (which was widespread throughout the industry since at least 2012), was an indication that much of a programmer's job was finding specific solutions to recurring problems, that never seem to get permanently fixed (mostly for reasons of culture, conformity, and churn in the ranks).
The more I see the LLM-ification of software unfold (essentially an attempted controlled demolition of our industry and our culture), the more I think about Arthur Whitney (inventor of the K language and others). In this interview[1], he said two interesting things: (1) he likened programming to poetry, and (2) he said that he designed his languages to not have libraries, and everybody builds from the 50 basic operators that come with the language, resulting in very short programs (in terms of both source code size and compiled/runtime code size).
I wonder if our tendency to depend on libraries of functions, counterintuitively results in more source code (and more compiled/runtime code) in the long run -- similarly to how using LLMs for coding tends to be very verbose as well. In principle, libraries are collections of composable domain-verbs that should allow a programmer to solve domain-problems, and yet, it rarely feels that way. I have ripped out general libraries, and replaced them with custom subroutines more times than I can count, because I usually need a subset of functionality, and I need it to be correct (many libraries are complex and buggy because they have some edge-cases [for example, I once used an AVL library that would sometimes walk the tree in reverse instead of from least to greatest -- unfortunately, the ordering mattered, and I wrote a simpler bespoke implementation]).
Arguably, a buggy program or a buggy library or a buggy function, is just an overfit program, or library, or function (it is overfit to the mental-model of the problem-space in the library writer's mind). These overfit libraries, which are often used as blackboxes by someone rushing to meet a deadline, often result in programs that are themselves overfit to the buggy library, creating _less_ modularity instead of more. _Creating_ an abstraction is practically free, but maintaining it and (most disappointingly) _using_ it has real, often permanent long term costs. I have rarely been able to get two computers, that were meant to share data with NFS, to do so reliably, if they were not running the same exact OS (because the NFS client and server of each OS are bug-compatible, are overfit to each other).
In fact the rise of VMWare, and the big cloud companies, and containerization and virtualization technologies is, conceivably, caused by this very tendency to write software that is overfit to other software (the operating system, the standard library [on some OSes emacs has to be forced to link to glibc, because using any other memory allocator causes it to SEGFAULT, and don't get me started on how no two browser-canvases return the same output in different browser _nor_ on the same browser in a different OS]). (Maybe, just as debt keeps the economy from collapsing, technical debt is the only thing that keeps Silicon Valley from collapsing.)
In some ways, coding-LLMs exaggerate this tendency towards overfitting in comical ways, like fun-house mirrors. And now, a single individual, with nothing but a dream, can create technical debt at the same rate as a thousand employee software company could a decade ago. What a time to be alive.
And you might say that this is a hypothetical problem, one that is not practically occurring. Well, we had a similar problem like this in the recent past, that LLMs are close to _making actual_. When it comes to software patents, they were considered a _hypothetical_ problem (i.e. nobody is going to bother suing you unless you were so big that violating a patent was a near certainty). We were instructed (at pretty much all jobs), to never read patents, so that we cannot incriminate ourselves in the discovery process.
That is going to change soon (within a year). I have friend, whom I won't name, who is working on a project, using LLMs, to discover whether software (open source and proprietary) is likely to be violating a software patent from a patent database. And it is designed to be used, not by programmers, but by law firms, patent attorneys, etc. Even though it is not marketed this way, it is essentially a target acquisition system for use by patent trolls. It is hard for me to tell if this means that we will have to keep ignoring patents for that plausible deniability, or if this means that we will have to become hyper informed about all patents. I suppose, we can just subscribe to the patent-agent, and hope that it guides the other coding agents into avoiding the insertion of potentially infringing code.
(I also have a friend who built a system in 2020 that could translate between C++ and Python, and guarantee equivalent results, and code that looks human-written. This was a very impressive achievement, especially because of how it guarantees the equivalence (it did not require machine-learning nor GPUs, just CPUs and some classic algorithms from the 80s). The friend informs me that they are very disheartened to see that now any toddler with a credit card can mindlessly do something similar, invalidating around a decade of unpublished research. They tell me that it will remain unpublished, and if they could go back in time, they would spend that decade extracting as much surplus from society as possible, by hook or by crook (apparently they had the means and the opportunity, but lacked the motive); we should all learn from my friend's mistake. The only people who succeed are, sadly, perversely, those who brazenly and shamelessly steal -- and make no mistake, the AI companies are built on theft. When millionaires do it, they become billionaires -- when Aaron Swartz does it, he is sentenced to federal prison. I'm not quite a pessimist yet, but it really is saddening to watch my friend go from a passionate optimist to a cold nihilist.).
I'm no mind reader, and certainly no anthropologist, but I suspect that what separates humans from other (non extinct) animals, is that we compulsively seek caves that we can decorate with moving shadows and static symbols. We even found a series of prime numbers (sequences of dots, ". ... ..... .......") in a cave from the _ice age_. Mathematics before writing. We seek to project what we see with our mind's eye into the world itself, thereby making it communicable, shareable. Ever tell someone you had a dream, and they believed you? You just planted the seed for a cult, a shared cave. Even though you cannot photograph the dream, or offer any evidence that you can dream at all.
The industrial and scientific revolutions have distanced our consciousness from this idea, even as they enabled ever more perfect caves to manifest. Our vocabulary has become corrupted and unclear. We started using words like "reality", and "literally", and "truth", when we mean the exact opposite.
The conspiracy theorists and cultists, are just people who wandered into a new cave, with a different kind of fire, and differently curved walls, and they want to tell people from their old cave that they have found a way out of the cave into reality -- they do not yet realize (or do not want to accept), that they live in a network of caves, a network of different things in the same category.
During the early 2020s, we did a lot of talking about the disappearance of "consensus reality". This is scientific terminology mapped over the idea of caves and cults. You can tell, because the phrase is an oxymoron. It is not reality, if it requires consensus. It is fantasy, it is fiction, it is a dream. The cave has indeed become so widespread that we even _call_ it reality.
If you speak language, and read words, you are participating in a cult (we even call caves that had a kind of altar in the center a cult -- in Eurasia, there was a cave-cult called _the cult of the bear_, which had a bear skull placed in its center during the last ice age, and I would not be surprised if people spoke to it, with the help of hallucinogens). The only question is whether the cult is nourishing you or cannibalizing you.
To the person you are responding to (user ocd): your cave (ladybird, your hypothetical tv-series), no longer nourishes you like it once did. Maybe find a new cave, build a fire in it. Unlike a television series, you can fork a code base. You make it into the perfect cave, just for you. And if another person likes this cave, chooses to sit by the fire with you, well, now you have a cult.
LLMs do not bring us closer to literate programming any more than version-control-systems or IDEs or code-comments do. All of these support-technologies exist because the software industry simply couldn't be disciplined enough to learn how to program in the literate style. And it is hard to want to follow this discipline when 95% of the code that you write, is going to be thrown away, or is otherwise built on a shaky foundation.
Another "problem" with literate programming is that it does not scale by number of contributors. It really is designed for a lone programmer who is setting out to solve an interesting yet difficult problem, and who then needs to explain that solution to colleagues, instead of trying to sell it in the marketplace.
And even if literate programming _did_ scale by number of contributors, very few contributors are good at both programming _and_ writing (even the plain academic writing of computer scientists). In fact Bentley told Knuth (in the 80s) that, "2% of people are good at programming, and 2% of people are good at writing -- literate programming requires a person to be good at both" (so only about 0.04% of the adult population would be capable of doing it).
By the way, Knuth said in a book (Coders at Work, I believe): "If I can program it, then I can understand it." The literate paradigm is about understanding. If you do not program it, and if _you_ do not explain the _choices_ that _you_ made during the programming, then you are not understanding it -- you are just making a computer do _something_, that may or may not be the thing that you want (which is fine, most people use computers in this way: but that makes you a user and not a programmer). When LLMs write large amounts of code for you, you are not programming. And when LLMs explain code for you, you are not programming. You are struggling to not drown in a constantly churning code-base that is being modified a dozen times per day by a bunch of people, some of whom you do not know, many of whom are checked out and are trying to get through their day, and all of whom know that it does not matter because they will hop jobs in one or two or three years, and all their bad decisions become someone else's problem.
Just because LLMs can translate one string of tokens into a different string of tokens, while you are programming does not make them "literate". When I read a Knuthian literate program, I see, not a description of what the code does, but a description what it is supposed to do (and why that is interesting), and how a person reasoned his/her way to a solution, blind-alleys and all. The writer of the literate program anticipates the next question, before I even have it, and anticipates what might be confusing, and phrases it in a few ways.
As the creator of the Axiom math software said: the goal of Literate Programming, is to be able to hire an engineer, give him a 500 page book that contains the entire literate program, send him on a 2 week vacation to Hawaii, and have him come back with whole program in his head. If anything LLMs are making this _less_ of a possibility.
In an industry dominated by deadline-obsessed pseudo-programmers creating for a demo-obsessed audience of pseudo-customers, we cannot possibly create software in a high-quality literate style (no, not even with LLMs, even if they got 10x better _and_ 10x cheaper).
Lamport (of Paxos, Byzantine Generals, Bakery Algo, TLA+), made LaTeX and TLA+, with the intent that they be used together, in the same way that CWEB literate programs are. All of these tools (CWEB, TeX, LaTeX, TLA+), are meant to encourage clear and precise thinking at the level of _code_ and the level of _intent_. This is what makes literate programs (and TLA+ specs) conceptually crisp and easily communicable. Just look at the TLA+ spec for OpenRTOS. Their real time OS is a fraction of the size that it would have been if they had implemented it in the industry-standard way, and it has the nice property of being correct.
Literate Programming, by design, is for creating something that _lasts_, and that has value when executed on the machine and in the mind. LLMs (which are being slowly co-opted by the Agile consulting crowd), are (currently) for the exact opposite: they are for creating something that is going to be worthless after the demo.
The lesson that I am taking away from AI companies (and their billionaire investors and founders), is that property theft is perfectly fine. Which is a _goofy_ position to have, if you are a billionaire, or even a millionaire. Like, if property theft is perfectly acceptable, and if they own most of the property (intellectual or otherwise), then there can only be _upside_ for less fortunate people like us.
The implicit motto of this class of hyper-wealthy people is: "it's not yours if you cannot keep it". Well, game on.
(There are 56.5e6 millionaires, and 3e3 billionaires -- making them 0.7% of the global population. They are outnumbered 141.6 to 1. And they seem to reside and physically congregate in a handful of places around the world. They probably wouldn't even notice that their property is being stolen, and even if they did, a simple cycle of theft and recovery would probably drive them into debt).
The implicit motto of this class of hyper-wealthy people is: "it's not yours if you cannot keep it". Well, game on.
(There are 56.5e6 millionaires, and 3e3 billionaires -- making them 0.7% of the global population. They are outnumbered 141.6 to 1. And they seem to reside and physically congregate in a handful of places around the world. They probably wouldn't even notice that their property is being stolen, and even if they did, a simple cycle of theft and recovery would probably drive them into debt).
The Soviets and their satellites (like the DDR), had another problem related to arbitrage, and that is that their professionals (such as doctors and engineers and scientists, all of whom received high quality, free, state-subsidized education), were being poached by the Western Bloc countries (a Soviet or East German engineer would work for half the local salary in France or West Germany, _and_ they would be a second class citizen, easy to frighten with deportation -- the half-salary was _much_ greater than what they could earn in the Eastern Bloc). The iron curtain was erected to prevent this kind of arbitrage (why should the Soviets and satellites subsidize Western medicine and engineering? Shouldn't a capitalist market system be able to sustain itself? Well no, market systems are inefficient by design, and so they only work as _open_ systems and not _closed_ systems -- they need to _externalize_ the costs and _internalize_ the gains, which is why colonialism was a thing to begin with, and why the "third world" is _still_ a thing).
Note that after the Berlin Wall fell, the first thing to happen was mass migrations of all kinds of professionals (such as architects and doctors) and semi-professionals (such as welders and metal-workers), creating an economic decline in the East, and an economic and demographic boom in the West (the reunification of Germany was basically a _demographic_ subsidy -- in spite of the smaller size, East Germany had much higher birth rates for _decades_; and after the East German labor pool was integrated, Western economies sought to integrate the remaining Eastern labor pools (more former Yugoslavs live abroad in Germany than in any other non-Yugo part of the world [the USA numbers are iffy, but if true Croatians are the only exception, with ~2M residents in USA, which seems unlikely]).
The problem, in the end, is that all of these countries are bound by economic considerations (this is thesis of Marx, by the way), and they cannot escape the vicious arbitrage cycle (I mean, here in the USA, we have aggressively been brain-draining _ourselves_ since at least 1980, which is why we have the extreme polarization, stagnation, and instability _today_ -- it is reminiscent of the Soviet situation in the mid 1980s to late 1990s). Not without something like a world government (if there is only one account to manage, there is no possibility of deficit or surplus, unless measured inter-temporally), or an alternative flavor of globalization.
Internationalism is a wonderful ideology, and one that I support. You can make the case that Yugoslavia, the USSR, etc, were an early experiment in Internationalism, that each succumbed to corruption and unclear thinking (a citizenry that is _inclusive_ by nature and can _think_ clearly is a hard requirement for any successful polity). Globalization, on the other hand, has a bit of an Achilles Heel: when countries asked why they should open their borders and economies to outsider/foreigners, they were told, "so that we can all get rich!". The problem is that once the economic gains get squeezed out of globalization, countries will start looking for new ways to rich, even if it means reversing decades of integration. Appealing to people's greed only works to the extent that you can placate their appetites. We should have justified Internationalism using _intrinsic_ arguments: "we should integrate because learning how others see and experience the world is intrinsically beautiful, and worth struggling for".
Note that most of these economic pathologies disappear, when the reserve currency (dollar) is replaced with a self-balancing currency (like Keynes' Bancor: https://en.wikipedia.org/wiki/Bancor). We have the tools, but everyone wants to feel like the only/greatest winner. These are the first people that have to be exiled.
The combination of attention-draining social media walled gardens, and the high performance pocket-computers (which are really designed for consumption instead of productivity), created a positive feedback loop that helped destroy the productivity that we won by defeating the paradox in the 1990s. And we have been struggling against this new paradox for twenty years, since. AI seems like it should defeat the paradox because it is a kind of hands-free system, perfect for mobile phones -- but this is really just a very expensive solution to a problem that we have created and allowed to fester. We could just shun the walled gardens, and demand to be paid for our attention and data.
The new productivity paradox (which I do not think AI in its current form can fix[1][2]), is the price that we pay for a prosperous and valuable advertising industry. And as long as the web is seen as an ad-channel, and as long as the web is always vibrating in your pocket, we will keep paying this price. We will eventually end up (metaphorically) lobotomizing our children, and families, and communities, so that the grand-children of ad-executives and tech-bros and frat-bros can grow up healthy, psychologically stable, educated, and comfortably wealthy. (Brain drain: now available literally everywhere).
[1]: It is telling that most LLMs are centralized, and are most useful as search-engines/information-retrieval-systems. The centralization makes them _spyware_, and their ability to directly answer any question, encourages users to actually ask direct questions, instead of stringing search-terms together. This makes the prompts high-signal advertising data (i.e. instead inferring what you are looking for from the search-string, these companies can see _exactly_ what you are looking for and why -- and with LLMs, they can probably turn these promps into joint-probability-tables or whatever other kind of serialization they need to figure out which products to sell you (either on the web or directly in the response to your prompt)).
[2]: As far as copyright infringement goes, LLM outputs may require mass clean-room rewrites (so your productivity, as pathetic as it already is, now gets _halved_ long term) of text, prose, code, and anything else that is produced with them, because of how copyright law works. In legal arts this is called _the fruit of the poison tree_, and any short-term productivity gains, may become long term liabilities that need to be replaced due to _legal mandate_ -- so even if LLMs can eventually produce perfect and faultless outputs, the copyright laws _in all 200+ countries_ would have to be torn down and rebuilt (and this will certainly come at great expense).
Even before LLMs, I have seen people (shamelessly) re-implement code from open source project A into open source project B, without attribution (IIRC, a GPL C++ project [no hate, I use C++ too these days] basically copied the very distinctive AVL Tree implementation of a CDDL C project -- this is a licensing violation _and_ plagiarism, and it effectively writes the C project out of history. When asked about this, various colleagues[1], just shrugged their shoulders, and went on about their lives.). LLMs now make this behavior undetectable _and_ scalable.
If we want strong copyright protections for open source, we may need to start writing _literate_ programs (i.e. the Knuthian paradigm, which I am quite fond of). But that probably will not happen, because most programmers are bad at writing (because they hate it, and would rather outsource it to an LLM). The more likely alternative, is that people will just stop writing open source code (I basically stopped publishing my repos when the phrase "Big Tech" became common in 2018; Amazon in particular would create hosted versions of projects without contributing anything back -- if the authors were lucky they would be given the magnanimous opportunity to labor at Amazon, which is like inventing dynamite and being granted the privilege of laboring in the mines).
The fact is, if we want recognition, we need to sing each others' praises, instead hoping that someone will look at a version control history. We need to be story-tellers, historians, and archivists. Where is my generation's Jargon File?
[1]: Not co-worker, which is someone who shares an employer, but colleague, which is someone who shares a profession.
In a way, this collaboration between the machine and the human is better than what came before, because now productive actions can be taken sooner, and mathematicians do not have to doubt whether they are searching for a proof that exists.
This just means that a smaller percentage of code will be useful/must-have/must-read code. Kind of like the market for films and books and games -- many more get made every year, which means that many more will get ignored every year.
Genuine question: are we 100% sure that this society was ever meritocratic? How would we measure `meritocracy` so that we can compare year-to-year and decade-to-decade?
Like, if I had to describe Wirth's Law using nothing but examples, Electron would be the most apt example.
"It always bothers me when people say 'why reinvent the wheel', because the wheel is an _amazing_ invention. It is almost perfect -- or quite possibly perfect -- for what it does. It perfectly turns static friction into dynamic friction for moving objects. It is something that so elegant and beautiful, that if you were to ask why we would want to reinvent it, I would say _of course_ we do not want to reinvent it. It has worked for thousands of years, and has been unchanged for thousands of years. _Nothing_ we have developed in the past 30 years of game-development, is a wheel. In the future we will have so much more powerful tools for game development, that people will laugh at the things we did to make games. If we want _to get to_ a wheel, we need people that will _attempt to make the wheel_."
Wikipedia Entry:
https://en.wikipedia.org/wiki/Twilight_Struggle
Article:
https://www.vice.com/en/article/8q8qp4/twilight-struggle-is-...
Links (mostly a TODO for myself):
https://en.wikipedia.org/wiki/Terra_Invicta
https://www.reddit.com/r/TerraInvicta/
https://tvtropes.org/pmwiki/pmwiki.php/VideoGame/TerraInvict...