Against the Dark Forest
wrecka.ge
wrecka.ge
To the degree that we've collectively built a dark forest, it's been in part due to this lack of multi-disciplinarianism.
Humans are amazingly adept at rationalization.
But depressingly few of my peers seem to be true general "artisan scientists" as I see it, understanding engineering to be not only a job but a craft, a discipline, an art form, a power and a responsibility.
So much is possible today. Individuals in developed countries, and even many in underdeveloped countries with access to tech, have more velocity at their fingertips than almost any individual in history.
We've built critical systems which rapidly create value with the slightest gesture. But few really take the time to sit down and connect enough dots to see enough of a bigger picture to make meaningful and considerate modifications to the status quo. Few push back against what they're told, few take the time to deeply appreciate even a fraction of the complex systems which create modern society.
We have to do better about educating ourselves, our peers, and our offspring. We need to encourage generalists, facilitate access to multi-disciplinarian education, and hold ourselves and others to higher ethical standards which are themselves informed by a broad perspective.
This certainly comes at the cost of one's personal time. However, I think a majority of us are overentertained and could be better about conditioning ourselves to routinely seek out and learn new things.
I'm sorry to rebutt your very first assertion but we had been in a cultural and scientific Renaissance and the last 15 years have been the slow unwinding of that. We got lucky that the internet explosion overlapped with the tail end of publicly supported cultural and scientific production.
I hung out with physics majors in college, all of them smarter than us compsci chuds, and uniformly they are absolutely struggling to survive as post docs or in industry. One of my college buddies has worked at nasa for 4 different firms and has had to move to Texas, Kansas, and Maryland for these gigs and has once again landed on a project where the funding got cut and is looking for a new job. Another works in nano scale semiconducting and had to move to Finland to get project funding from the EU since the US has made basic research funding so scarce. And after several post doc roles he is leaving the field after his last grant wasn't renewed, with not an ounce of negative feedback. Just, sorry we don't have any money anymore. He's now going to go into failure analysis for a mobile phone manufacturer to pay the bills.
The woes of cultural production have also been well documented.
We are in a cultural and scientific collapse
https://academia.stackexchange.com/questions/25910/what-does...
I would love to choose to be an artisan, a craftsman, a changer of the world, a multi-discipline Renaissance Man, (or just an over-entertained couch potato) but I need to make my mortgage payment next month.
For the most part, our station in life is something we constantly wage war against from the moment of birth. Coming from a very poor and abusive background, I was homeless by 16 and left with a choice of fully committing to boring technical work, or taking advantage of already being at rock bottom and surgically improving my skills over years until I've become as well-rounded as I would like. It's a lifelong journey, though.
I do wish you luck and I hope you do find time to accomplish some of your desires. You will definitely have to create time and space for it though, free time likely won't magically appear within your current routine.
But yeah, people should be curious and willing to try things, even things that doesn't seem to be the thing for them.
Now I say that stem is just a tool that does no good by itself. It's just as harmful as beneficial. An engineer doesn't like to think of themself as a tool, but tough shit. Without the humanities an engineer is exactly that, merely a tool that others wield, and others decide how to wield and to what ends.
Stem tells you how to build a bridge or a bomb. The humanities tells you whether/when/why to build a bridge or a bomb.
We need a field that is able to systematically cartograph the limitations our brains impose on us, society and technology. We need a world map of blindspots, of disabilities l. Not 15 years ago we congratulated ourselves on plans to degrowth , and look at what happened during the pandemic.
While I agree that disdain is not the right reaction, neither is misdescribing what these disciplines are. And the most important fact about them is: they are not sciences.
We do not have a science of how people work, how groups of people work, how collective action works. Dealing with people is an art. It requires applying non-repeatable human judgments to non-repeatable human situations, and doing so constantly, day after day, year after year, realizing that there is never going to be a stable state of things, there is never going to be a time when you have dealt with all the conflicts and solved all the problems and can sit back and relax.
Engineers are understandably uncomfortable with such a situation because it goes against everything we (I say "we" because I'm an engineer myself) are taught. We are taught to engineer solutions to problems, solutions that keep working, if not forever, at least for a long enough time that we aren't constantly having to manage them. A well built bridge does not require constant engineer attention once it is finished.
But there is no such thing when it comes to humans and human relationships. They do require constant attention in order to survive and flourish. But we have no science of how to do that, the way we have a science of how to build bridges. All we have is our fallible human judgment, and if we're lucky, some rules of thumb that, while they aren't science, are hopefully better than nothing. Unfortunately, I think what rules of thumb we have have been mostly ignored in the process of building systems like social media.
To that degree, studying enough sociology to be able to make the statement, "Compartmentalization and hierarchy of academic subjects contributes to software engineers being unwitting used to replicate social structures," is not dependent on sociology being a science. It is true without relying on an arbitrary epistemology.
Findings don't replicate, data is used to tell lies, policy built on scientific findings is mostly unrelated to the findings, focus is primarily guided by culture instead of discovery or curiosity. I read a lot of sociology, and the general level of critical thinking and concern for the truth in journal articles is much lower than biology, engineering, or even economics.
Cite something or don't @ me.
Yes, we do, but their predictive power, if it exists at all, is very poor. Science is about building models with good predictive power. If you don't have that, you don't have a science, at least not in any way that matters.
And there are social science hypotheses that can predict outcomes.
It's true that the most egregiously wrong prediction in the Federalist papers, that the US would not have political parties because it was too large, was the work of Madison, not Hamilton. But AFAIK none of Hamilton's predictions turned out very well either.
Many times these practitioners of these ”soft sciences” are cloistered away in the ivory towers of academia and away from regular people whom they actually look down on because they don’t match their theories.
This isn't a new concept, but the compartmentalization of subjects in school is one of the factors at play along with a hierarchy of subjects and students. You can observe how a lot of computer science folks hold mathematics and physics in high regard, and biologists, and sociologists in low regard. It's no coincidence that this maps to relative possession of the habitus valued by the ideological hegemony.
Again, it works to reinforce social statuses rather than dismantle them, often in ways that are detrimental to those actively, albeit unknowingly, contributing to the same systems.
My issues with the "soft" sciences, is chiefly their inability to replicate findings, and the paradoxical authority these so called scientists seem to gain despite not actually following the scientific method. It doesn't have to be this way either.
I'm not saying everything in these fields are total bunk or anything but I've been deceived by them enough by now that they have lost my trust. I'm sure there are plenty that actually do good science but I'm not capable of telling them apart from the rest unfortunately.
Even more concerning is this pretentious attitude reminiscent of what I'd expect from a religion or political movement than a field of science. Are we starting to treat universitys like cathedrals now? A scary thought.
Qualitative precision matters. HN crowd presumably loves true social network academics.
Dark forest Vs real could probably be described by networks.
Also Social Media Networks are conversational markets, run by market makers with different kinds of incentives. I do not recall seeing descriptions like this but it would bring real empathy to the space to merely describe things like this.
Broadcast all is NOT the norm in price discovery. Why is it the starting point in social networks? We are brainwashed.
No rational being left then.
If you’re not thinking about that it just means you have no idea what stuff you are loading into your designs. It’s still there you’re just not aware of it. It’s like being the unwitting carrier of a disease or a parasite.
Makes me think of brain slugs in Futurama for some reason.
If you engage outside the scientific community, you're most likely going to encounter cranks.
Ideas that were not taken seriously even by the people who put them forward:
> Ronald Reagan launched his 1980 campaign for the presidency on a platform advocating for supply-side economics. During the 1980 Republican Party presidential primaries, George H. W. Bush had derided Reagan's economic approach as "voodoo economics".[23][24] Following Reagan's election, the "trickle-down" reached wide circulation with the publication of "The Education of David Stockman" a December 1981 interview of Reagan's incoming Office of Management and Budget director David Stockman, in the magazine Atlantic Monthly. In the interview, Stockman expressed doubts about supply side economics, telling journalist William Greider that the Kemp–Roth Tax Cut was a way to rebrand a tax cut for the top income bracket to make it easier to pass into law.[25] Stockman said that "It's kind of hard to sell 'trickle down,' so the supply-side formula was the only way to get a tax policy that was really 'trickle down.' Supply-side is 'trickle-down' theory."[25][26][27]
> Political opponents of the Reagan administration soon seized on this language in an effort to brand the administration as caring only about the wealthy.[28] In 1982, John Kenneth Galbraith wrote the "trickle-down economics" that David Stockman was referring to was previously known under the name "horse-and-sparrow theory", the idea that feeding a horse a huge amount of oats results in some of the feed passing through for lucky sparrows to eat.[29] Reagan administration officials including Michael Deaver wanted Stockman to be fired in response to his comments, but he was ultimately kept on in exchange for a private apology.[30]
* https://en.wikipedia.org/wiki/Trickle-down_economics#Reagan_...
See also:
* https://en.wikipedia.org/wiki/Kansas_experiment
It'd be like saying traffic engineering and urban design are bullshit because politicians keeps building highways—when traffic engineers know this won't solve the issue of traffic:
* https://en.wikipedia.org/wiki/Downs–Thomson_paradox
Also:
* https://en.wikipedia.org/wiki/15-minute_city#Conspiracy_theo...
Just because quack ideas attributed to a particular field are used to justify certain ideological ideas or policies does not mean the field itself is devoid of valid models.
For example, history is frustrating because you are going to end up in unique circumstances and you're not going to be able to see into the future, only the present. Yet, we must at least have an understanding of the world around us and critical thinking skills that can only come from with doing history.
Because Skinner boxes exist. There are entire multi-billion dollar industries built around them (casinos, gacha games/loot boxes), just to pick a super low-hanging-fruit example.
And very little of psychology covers what is used in ads.
This is 100% a sociological take though. Ie: it is in part doing sociology. Perhaps without knowing it, you've arrived at an Althusserian perspective on structuralism through the lens of conflict theory.
Do you suppose that these institutions be systematically studied and can we build models of how they work to reproduce the status quo?
If you suspect that you're being lied to by the consensus, feel free to read alternatives by heterodox scholars. I suspected that they're crank.
It's not like I particularly agree with the way things work now, only that we shouldn't make things worse by making obviously stupid moves in the wrong direction.
If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
(I don't think the GP comment was very good either, but taking the thread flameward is definitely not what we want.)
Engineers due tend to be pretty critical of things that can’t be accurately and consistently measured (not meant as a slight. It’s true, but it doesn’t make those soft sciences worthless), but they aren’t the only ones making these systems.
Most of the time, engineers are given requirements and build systems to satisfy them.
You wouldn’t blame a construction worker for gentrification, right?
I have yet to see any actual backing for such claims. Only standard academia's toxic social knife-fighting and Kafka-traps. The specific form of bullshit being mounting social pressure as a weapon. Where where not agreeing with them and their theories means clearly you are more wrong and a worse person for not agreeing with them and thus you must agree with them!
It always rung of a scam to justify mandatory electives to justify the existence of their professorships. There is no intellectual honesty to be found here, only snobbery and blind adherence to cliches.
There are plenty of engineers for hire, and sociologists can build their own vision.
It’s an interesting contrast to actual dark forest theory—-AI doesn’t want to destroy us, so we don’t need to hide for existential sake. But imagine walking outside and as soon as you do, innumerable copies of you spring up, and each action and sound you make is replicated and amplified. Like a weird Phantom Tollbooth meets Alice in Wonderland on DMT
That's honestly not a problem, unless someone believes they're entitled to 100% of value generated by their interactions with the world - but that's a level of greed way beyond Scrooge McDuck, and the kind of thinking that defines the dreaded "late stage capitalism".
It really is an option to publish something and not care about knowing exactly who read it, and what they're doing with it.
There's 8 billion people on the planet. Unless you're a celebrity, this is not a real problem for you at this point (and if you are, it's a business problem).
There's no way for a large model learning on the entirety of the Internet to somehow convert "copying likeness, mannerisms, writing style, etc." into damaging your reputation; doing that is something hyper-targeted, and at this point (and in conceivable future), there's no middle ground between "plausible deniability" and "someone targeting you specifically, which they could do just as well in pre-AI times".
You're also not as unique as you think. There are many people with same mannerism, many people with same writing style, etc. Nor are those things constant over time.
The flip side of not being a unique snowflake is also that anyone's contributions to the public Internet are, for purposes of AI training, worth approximately $0 on the margin, and impact the model just as much. It's in the volume of data that the patterns emerge that LLMs learn, volume too big for any person to be entitled to a meaningful part of it.
"You're not special" is a bad argument for why someone impersonating you isn't a problem. I mean even Black Mirror explained why impersonation is a problem in the very first episode many years ago. You don't even have to be a celebrity, you only have to be in someone else's way.
You couldn't, because unlike all the things we discussed about AI, those are actually harming the victim in real, direct terms.
Having some model who doesn't know or care about you talking just like you, without claiming it's actually you, to a bunch of people who don't know or care you exist? That's zero actual damage to you.
> "You're not special" is a bad argument for why someone impersonating you isn't a problem. I mean even Black Mirror explained why impersonation is a problem in the very first episode many years ago.
I remember that episode. However, my argument isn't "you're not special, therefore impersonating you isn't a problem" - it's "you're not special, so what you think is impersonating you probably isn't, and even if, it doesn't hurt you in real terms", combined with "anything that's actual impersonation and/or hurts you directly was already possible, and AI currently doesn't impact this at all". Someone wants to screw with you? You're being targeted. AI might make the attacker's job a bit easier, but it's still someone going through the effort, vs. a background process on the Internet everyone seems to think LLMs are.
Also: there's an inverse relationship between weight of accusations and social proximity. A specific person you know (and other people know you know) accusing you of something? It's a problem. Some random comments from random accounts, accusing you and 100 different people of something? Most people won't believe it.
(Except when it's about child abuse. People are extremely sensitive to this - just bringing up the term and a name in the same sentence can ruin the victim's life.)
In a world with UBI (or in a world where creators are aristocracy who don’t have to worry about money) I would agree. In a world of precarity, obviously many people are going to desire to limit reuse of their creations unless they get rewarded for it somehow.
I can understand why all kinds of artists, who make money from their creative work, have an issue with generative models. But I feel this is unavoidable - graphics and writing and video creation will follow the same path music already did decades above. For a long time now, bands and singers don't make their living through their songs and recordings. They make it through live performances and branded merch, and related things. Music is too cheap to meter, but human connection isn't, which is why musicians make money by being entertainers. This is the fate that awaits other creative arts too.
Plus pragmatically speaking this sounds suspiciously like something that's "not a problem" until it is a problem. If a poorly written LLM regurgitates my name and lines from a blog post (or god forbid hallucinates a blog post by me as a citation) in some defense of Pol Pot or something that's going to be come a problem for me very quickly.
EDIT: Fridge thought: quite frankly if someone's making money off my interactions in this economy you can bet your last dollar I want my fair share of it. It's not greedy to gain value from your interactions with the world: you already do by interacting since that's a bi-lateral relationship ie it affects each party equally. It IS greedy for a third party who did not participate in the interaction to expect value from it.
It is about the right to have my own value not diluted.
It is not about caring to know who read each one of my words.
It is about the fact that people can later say „xyz wrote that” instead of „oh well another AI generated crap comment” when in reality it was someone’s original thought.
It is not capitalism, but basic respect for other human being.
"We attribute a thought to a person through a chain of custody. Without it, the thought could be attributed to anyone, or to a generative language model. Not that it matters anyway." -- Abraham Lincoln, ~1862.
The “tech bros” controlling AI don’t want to destroy us either but they’re not against if it means $$, they’re indifferent.
And the ad money only keeps flowing harder than it did last quarter if they do destroy us, so it’s a pretty easy choice to destroy society for an easy buck.
of course this is a simplified version of real life that doesn’t take foreign influence and stuff like that into account.
The Dark Forest experience that's causing "mass migrations" on the Internet is that, if you venture out into social media carelessly, you're entering an aggressive and frequently unpleasant space which is close to the opposite of "fun" for the average person. Here follows a laundry list here of things which you might expect to encounter in an unpleasant online space, most of which have a kind of tabloid quality: scams, nonstop sales pitches, porn, extreme violence, celebrity worship, ideological extremism, racism, sexism, etc. And more (that list is not exhaustive).
So people find that signing up for the average generic platform that welcomes everybody is a bad experience, and migrate to more niche ones. That's been the path I've followed too, all the way here.
Berserker event is effectively an implementation detail of one of possible events in the Dark Forest.
Like Wikipedia says, Dark Forest is for people who are limited in tech and resources. Or have a weird sense of ethics, but ask Robin Hanson how long he thinks that would last.
It's based on another article that claims that people mostly interacting in their own social bubble, fueled by algorithms which mostly isolate people further, is akin to making the Internet a dark forest.
I think I must be confused.
So yes I think you're confused.
> I don't think any centrally governed platform at global scale is capable of doing the work, even if they hired the best and sharpest people I know. Even if they put real effort into humanist upstream product design, rather than tossing loose change to trust and safety teams sent in to clean up after the fact.
>
> Local norms matter too much for global governance of the social internet to make sense; the flattening of global diversity to fit the norms and interests of any given American techno-culture—corporate or otherwise—is both a baldly colonial aspiration and one we should scorn for the same reason that we leave the idea of effective, monolithic, planetary-scale government—benevolent or otherwise—to underbaked science fiction. Home rule and genuine resilience both require the existence of many places, many of them at least partially interconnected.
That’s what confused me, given your cited context. It doesn’t sound centralized. Things that are not centralized are decentralized, no?
The article is arguing for spaces that are more loosely connected but still moderated, probably more moderated.
At some point between, say, 2000 and 2012, everyone forgot that. The Web became an extension of the real world. Now, in 2024, the Web seems to have displaced the real world as the locus of public discourse.
> It names an experience of paranoia and anxiety that by the end of the 2010s was widespread among people with meaningful connections between their online personas and their ability to maintain their standard of living.
This kind of thing always struck me as a poor choice. The great thing about maintaining one's anonymity on the Web, or, to be more precise, adopting a collection of alternate identities, is that they can be discarded easily. A faux pas needn't follow you around, and can't follow you to where you make your living.
> The last and most most dangerous weakness of the Dark Internet Forest as a frame is that it positions the broad landscape of connection as something that “we” can simply do without—and without which we will indeed feel better and be more productive.
> On the level of the individual, this is true for certain values of “we”: for people who are, in any sense, established; people who already have the social status they required to succeed in their field; people whose work doesn’t depend on them needing to find (and re-find and re-find) readers or customers; people whose professional and personal networks are already strong enough to catch them if they slip; people with money.
> So what about everyone else? Should people without those forms of access and capital simply forgo all the benefits afforded by access to broad networks?
Through the twentieth century, as aristocracy gave way to meritocracy, people learnt how to climb the greasy pole without access to these broad networks. Social mobility is largely a consequence of the structure of the economy, and not necessarily a consequence of networking. We are often told that, 'It's not what you know, it's who you know,' but maybe the people who found other ways to the top simply aren't as effective at sharing their secrets as the shmoozers.
> This all stops being an individual problem and becomes a collective one when bad products of the social internet get worse, as when platform turmoil and manipulation helps remodel the offline world in the image of the most grotesque parts of the online one.
Conflict is coming. We've reached a point, I think, we're it's inevitable. I don't think it's clear exactly what the ideas underlying the conflict will be, or what factions will contest it, or in what spaces it will be contested, but there is a range to choose from. The winning move in this situation is to keep informed about events, but otherwise to keep one's head down. Don't challenge ideas you find objectionable, but agree in a mostly-disinterested way. Take it from someone with family members who still have the machete scars.
I feel the "american dream" got that part right. Social mobility works best when you can start working locally to move up. That is, very low barrier to entrepreneurship. "Who you know" stops being a problem then, everyone can work their way into larger and larger networks.
For better or worse, I don't think we can get rid of "who you know" issue, because it's 100% natural. When you start a venture - whether it's a local shop or an Internet company or just remodeling your dog's kennel - you don't go to Global Employee Search Directory and look for potential co-founders; you just do it with a friend. Now, even the largest corporation, when you go all the way up to the top, is still a glorified version of neighbors building a common tool shack. So is almost every subgraph in the corporate ladder, if you zoom in on it.
So the problem, as I see it, is with an economy where most people have to effectively beg strangers for a job - i.e. employees (myself, I am such a person too). In this scenario, "who you know" at the lower levels becomes unfair to the larger whole.
I'm not sure what should be done about it, though.
(This is just some random thought I have, I haven't managed to sort them into a coherent whole just yet, sorry.)
September.
You could take a screenshot, but not a photo, you couldn't have a real photo inside the computer without a scanner.
You could plug into the phone line if you had dial-up, but you didn't make calls over the Internet. If you did call someone, you had to both install Skype or something.
You could "watch TV" on the computer if you used a cable tuner, but there wasn't TV online.
Reality stayed in reality, and although the Internet has always supervened on reality, the Internet stayed inside the Internet.
Somewhere around "computer phone times" this stopped being the case.
Now politics happens on Twitter, you can order a pizza online and it's not a credit card scam, it actually arrives. You can place a call and with the right aeons it may even connect to a POTS telephone.
At some point it shifted from "The Internet is where a bunch of nerds who like The Internet talk about The Internet" to "The Internet is plugged into real life and is part of real life" to even "Real life is part of the Internet"
The TV reality. It's pretty peaceful when everyone agrees, even when what they agree upon is false.
Regular people almost never used those words. Only the media did.