Hacker News Reader

Comment by pyinstallwoes | Hacker News Reader

It seems that the medium of writing will inevitably reach a point of too much information to process, malicious or not; while simultaneously increasing the risk of silos.

What does a post-writing civilization look like? Is it reasonable to imagine scientific progress without the burden of having to battle the computational capability of sufficiently motivated actors who can easily manipulate authored information spaces?

Writing is essentially a massive attack vector to the collective human consciousness prone to Sybil attacks at an alarming saturation point in a post gpt3 society.

It feels like we need something post-writing.

View on HN

You still have the social graph of the authors. Professors are the gatekeepers of the next generation. Like venture capital, you will have to hustle your way to the top for an introduction if you are an outsider.

The social graph of the authors is precisely what computational superiority is able to manipulate due to the computational resources to create false reputation graphs that are indistinguishable from someone’s bias for base truth.

How is that possible? You start with a group of people whom you trust. Then you check papers where those people are co-authors. You add everybody to your group who is on an acceptable paper. On the other hand, you degrade or even remove those who are on a bad paper.

Computational resources are useless because you cannot fake papers with trusted authors. Trusted authors won't publish a bad paper because they want to keep their reputation.

All an adversary can have is something like a network of fake articles. Unlike web pages, scientific papers are not opinions. The scientific core group can check if papers are relevant and true. The scientists can limit those checks to authors whom they have met in person to minimize their time investment.

How do you know if your network is the truthful one and isn't participating in deception? And given that networks are increasingly non-physical, how do you guarantee it is the person in question?

The only way reputation networks maintain durability is if they stay physically constrained and limited by physical contact. Any system built externally and allowed to represent facts and truth will be compromised due to computational superiority. It must not be a network hosted by a computer. It must run on culture and physical reality itself.

Additionally agents of deception are regularly embedded in networks to encourage malicious behavior. Spies are pretty common.

See: https://m-cacm.acm.org/magazines/2021/6/252840-collusion-rin...

I think the key bias we're contesting is that I'm arguing computational superiority allows creating networks that are indistinguishable from truthful ones because they have the resources to have the "longest version of history" to act as the authority. If the history is not there, it can be fabricated and embed enough "covert actors" to maintain a narrative.

Then you also generate useful idiots who don't know they are being used to propagate misdirection or what have you. They become unable to detect any "obviously fake content" because it doesn't require fake content to conduct deception.

Very often, it is real information indeed that is most able to misdirect attention.

>How do you know if your network is the truthful one

You don't know. Evolution will decide if your seed group was good enough.

>how do you guarantee it is the person in question?

You can't. You have to role back if an identity is compromised

>It must run on culture and physical reality itself.

Yes. That's the difference between science and writing in general. Science is meant to be checked with reality.

> think the key bias ... the resources to have the "longest version of history"

History is secondary if you verify all statements. History should only grant preferred treatment.

>Then you also generate useful idiots

That's outside of the trust network. You can help them by making education easily accessible.

>Very often, it is real information indeed that is most able to misdirect attention.

That's when the game becomes interesting. You can also have truths that are not verifiable etc. It all depends on what the participatns want to know.

Largely agree with your adds. However one point you mention I've had great contemplation over:

> Yes. That's the difference between science and writing in general. Science is meant to be checked with reality.

When we live in a world that is increasingly using simulations for verification instead of reality itself, it seems as it becomes an ever increasing attack surface for controlling and manipulating what reality and truth actually is. The scary quote "Science Academics" are largely publishing statements based on mathematical models that are probalistic or theoretical with no physical extant items.

When the burden of verification is solely on the individual, yet the individual is increasingly pushed in the direction of having to believe things as facts rather than experiencing them, then it sort of compromises that integrity, no?

It seems the only way to have certainty is to be a witness and have experience over a set of events, that is the only thing that can be understood to be true (and only to you), and that is what contributes to certainty. All else is belief and unknown risk to deception.

tl;dr science is heading towards imaginary non-extant things that happen to predict physical things but they can only be verified non-physically and since certainty require physical experience then the security of the system is at risk of being corrupted.

tl;dr;dr what to do about simulations and models posing for reality?

Also interesting:

> That's when the game becomes interesting. You can also have truths that are not verifiable etc. It all depends on what the participatns want to know.

Can you expand on this? You're saying something incredibly poignant and I'd like to hear more.

>what to do about simulations and models posing for reality?

It is or will become indistinguishable from thinking. We can ask the same question about our thoughts.

The science trick is that it requires models that can be falsified in reality. If you make predictions about simulations, then you don't make science but math.

The problem about simulations is that they make verification equally or more expensive than the simulation itself. It's the end of 'scientific cooperation'. Why verify a simulation when you can make your own?

>>It all depends on what the participants want to know. >Can you expand on this?

I wouldn't call it poignant, but here you are: As you wrote "Very often, it is real information indeed that is most able to misdirect attention." - there is a process that makes choices. Truth is a tool to reach the goals of that process. We don't care about the scientific method in itself.

Most people don't need truths to reach their goals because there is kind of a market of facts. Some facts are just known, without scientific backing, and people can just choose to use them.

When there is deception there is also cooperation. The misdirecting information is only there because somebody else wants to achieve something. Maybe it's easier to cooperate with them about their goals than handling the misinformation?

To your latter point, yes, and one may not be aware of the fact they are the useful idiot. It may even be degrees useful idiots.

The fundamental issue is that there is a cost to deception that I reason is greater than the benefit to the ones gaining from the cooperation and execution of the competing trajectories of narrative control. It essentially breeds psychic cancer. It incentivized all the behavior that leads to distrust then inevitably chaos.

So the only way to avoid that is no competing factions. How do we transition to a non game theoretic civilization? Can we achieve it without assimilating to silicon?

And related to your points on truth which I agree with, is the classic Greek comparison between Techne and episteme. It translates also to understanding and applying facts demonstrating “actually getting it” vs just regurgitating the fact market. The Tao te Ching has some anecdotes on the danger of knowledge as it leads to people thinking they know something but all they really have is the ability to echo or parrot. I guess you need your messages delivered in a network one way or another.

Thanks for the thoughts. Would enjoy hearing more from you if you’re on a discord or similar.

Thank you.

I am not prepared to continue this outside of hn. How could I reach you later?

Regarding the deception: We also decept ourselves. So even if you go silicon, how do you know that that silicon is without flaws?

I can be reached at gentle.zone6594@fastmail.com

- precisely re: silicon. One can never be sure about external systems outside the self it seems, and even that is potentially at mercy to whatever can manipulate the senses. However there is an increased risk of deception / corruption by introducing a universal thing that only 0.0001% or so know how it actually works or could build one end to end.

It doesn’t necessarily mean to abandon silicon, but it does mean that it can’t ever be trusted. So designing systems around that and probably minimizing dependence. I can see this happening with “cold computation” that is very specific in purpose and low in cost. Something like the green array chips. But that does have the potential to swarm. All I know is that if silicon replicates it’s going to be in conflict with dna.

It seems prudent for a dna based life form to stick to dna for truth. All else is risk of assimilation or some form of deception at large unable to be detected.

Much more to unpack there though.

Mathematical claims may be formalised in a proof assistant.

The burden of proof shifting to a sacred script only understood by a minority of the population seems like giving the 0.0001% all the power.

Reply on news.ycombinator.com