AI: Markets for Lemons, and the Great Logging Off
fortressofdoors.com
fortressofdoors.com
I've become disenchanted with the quality of the discourse. Specifically as I've entered a more senior stage of my career, I'm looking to mingle with like-minded folks in similar situations.
Thing is, these folks don't usually lurk online, and even if they do, their voices get overwhelmed by the flood of (mostly low quality) content. I'd be happy to pay a fee to become a member of a community with vetted participants.
Ai can be used to moderate discussions however , if it is trained to remove low effort in an unbiased way
> Ai can be used to moderate discussions however , if it is trained to remove low effort [or otherwise undesirable content]?
See https://astralcodexten.substack.com/p/sorry-i-still-think-i-... for more on this topic.
If that’s the case, 100.00% open might not be optimal if 99.9% open yields better results for the resulting community.
The purpose of the internet hasn't been to be an open forum since sysadmins were blocking the talk.* hierarchy and kicking entire nodes off of usenet for spam or abuse.
Vetting, moderating ... none of them scale well when done manually. The solution I see is that everyone have their own AI filters, customised as they see fit. You can include or exclude specific people and topics, make it as diverse or narrow as you like, allow challenging opinions or not. One of the filters can be to detect AI bots. Don't make the world conform to you, be selective and just skip the bad parts.
I think people are going to trust their own AI tools that are running on their own computers more than they trust other people and especially other AI. We already know we can't face the future onslaught of information with the old methods, we need help. User controlled AI is going to be our first line of defence and our safe space, "a room of one's own" where there is no tracking and thought policing.
With the advent of large language models we already have that, indirectly - the LM is a synthesis of everything, but we let it generate only conditional to our intentions. All the view points are in there for us to reach, it depends on us how we relate to them.
AI should be like a cell membrane separating outside from the inside. It should keep the bad stuff out, take the necessary nutrients and nurture the life within.
Then maybe we should reduce the scale of the megaphones. Current networks are just unpleasant because it's mass hysteria on global level. Scale isn't everything.
I think future forum owners will be continuously finetuning their AI filters to fit their community standards.
I follow really interesting people for their thoughts about one topic (say, Movies), but whose tweets about topics like politics I really dislike (not necessarily things I disagree with, usually I just dislike the tone).
Shouldn't be too hard to fine tune a LLM to do this, definitely something worth a try.
Most notably, kids were in school with people their age, adults were at work, etc. 1st grader intellect and emotional interactions partitioned from 5th grade, partitioned from 9th, partitioned from college, partitioned from post-college.
For another example, the "grownups' table" and the "kids' table" at a holiday dinner.
Though, as a kid, I definitely benefited from listening in on the grownups on the Internet.
For example, businesses that involve $$$ reusable delivery containers don't work well because they want to give first-time customers a discount not a surcharge. Whatsapp was ridiculously cheap - something like $1/year - but they still decided that worked against their network effects.
There are some markets where you pay upfront and on an ongoing basis, like buying a printer and ink or a games console and games - but they often subsidise the upfront cost with profit from the ongoing costs.
I'm not sure how well a $5-to-join social network would work for that reason. Sites like metafilter survive - but with five moderators and one coder, they're not going to be replacing twitter, facebook or HN any time soon.
if those site could convince important/popular people for exclusivity, they could pull it off; i imagine it can work like spotify exclusive podcasts, or some such, since most people don't actually produce content, but consume.
Their mistake is having growth as a goal rather than a happy community.
I'd say HN is one of the few forums competitive on quality of discourse vs metafilter, but its business model is essentially loss-leader for VC/incubator activities, so it's a bit of an odd duck itself.
My boss of a multimillion dollar company took offense at being charged $1/year to use Whatsapp back when it came out and started getting popular, so it did have an effect
My eyes practically rolled out of my head when I heard him complain but that really stuck with me
I believe even a nominal amount would keep most people out, and that's by design. Also you don't need huge numbers to build a personally interesting community. Even a few dozen dedicated people can be plenty.
I have the same issue. People place so much trust in their preferred sources of information and don't question anything they read. Because of this I have a hard time discussing anything beyond the most basic things with many friends, since they just repeat talking points and opinions of others.
It's so enlightening when I do find someone who's well read on a subject and can have an actual conversation about it. Even better if they have a different opinion and give you some new bit of information that makes you think about your opinions or perception of knowledge. Or, when you can agree to disagree because you have the similar understanding of a subject, but have different values or preferences. Moments like that can make me really happy sometimes.
People have always been this way, but the Internet and social media are really not helping.
I think the optimism of the early internet was due to a dominance of users with that kind of personality. It wasn’t apparent to that type of person that others are not excited by that perspective hopping opportunity and not only would not take advantage of it once being online was more accessible, but would actively use the same technology to further silo and entrench themselves.
You better start believing in vetted online communities Miss Turner, you're in one!
It may not be vetting by moderators, but HN's dated frontend definitely acts as a forcefield against many kinds of people.
I find the idea of the site intriguing yet I'm not sure what I could contribute.
(although it would be an amusing way to shut the site down if he gets tired of maintaining it...)
I mean I do absolutely hate the new reddit redesign and almost exclusively use old.reddit, but the functionality there is still on another level when compared to HN.
Actually, HN is the only social media site where my browser can consistently keep the scroll position. Animations and endless scrolling are what breaks it.
FreeMasons and premium golf clubs!
If you have an interest in, say, the math behind various AI techniques. Or maybe an interest in digging into the differences between different, (but seemingly similar), cancer therapies. (Think TIL v CAR-T). You won't get that at the golf club. You will need some kind of vetted discussion group.
Like startups, communities are growth and sucessful ones grow at an incremental replacement+ rate, where ones that grow explosively fall over on their own weight, and ones that sustain a base level just die. A plan for a vetted community is a system of harmonious growth that requires a funnel and conversion scheme that yields vetted members. I see orgs spinning their wheels and arguing over the merits of individual candidates when their real problem is they have no funnel to simply select the best n% from.
Thank you for bringing me up to date.
Any other sites you'd recommend for thoughtful discussion?
I'll launch the next iteration after the first of the year, this time there might be a fee attached, at least a minimal amount to pay for the SaaS product we'll use to organize and collaborate.
The next season I told everyone to pay up front. $120 dollars, cash. Same price as every week in total, but up front.
Turns out when people paid up front they came more often and we had no payment issues from there out.
We can summarise this comment as "Eternal September"-ism.
Ref: https://en.wikipedia.org/wiki/Eternal_September
Be wary of falling trap to the feeling of: <<It has never been worse.>>
In late high school, watching the daily local television news around 6PM, I began to realise that when local news programmes interviewed local residents, they were always seeking the same response: <<It has never been worse.>>
Examples: Traffic; politics; manners; dress code; respect for elders; price of "milk and bread" (ask anyone in their 90s about it!); Internet discourse; buying/selling used cars; kids driving cars; fashion; fast food; the latest technology (radios, TVs, Internet, video games, mobile phones) ..., etcetera!
My point: Do not assume that things that are different from when you were young(er) are automatically worse. Really, really: Try harder. In 1920, "old people" were complaining about "young people" listening to too much radio -- made them "soft". Repeat for television in 1950. Repeat for Internet in 1990s. Repeat for mobile phones in 2010s. Repeat for video games in 2020s. I see this pattern repeat over and over again, and I chuckle to myself when I see old men complain about it!
As a counterpoint: This website is an amazing counterpoint/weight that "all things are getting worse all the time". I have been here for a few years now, and this community always manages to adapt to changing standards and conditions... and produce inspiring and thoughtful discourse about recent news.
And now for my over-the-top inspiring comment to end this post: Simply put: We, HN community, are the antithesis for Sturgeon's law "ninety percent of everything is crap"! (Wiki: https://en.wikipedia.org/wiki/Sturgeon%27s_law)
What I have noticed in your comment:
>I see this pattern repeat over and over again
That's not a proof that there hasn't been a decline. It could as well be that each new generation is less competent.
Discussion can't scale beyond a certain number of active user accounts without a large drop in quality. It's kind of hard to tell what that limit is because every forum has a large number of mostly quiet lurkers who simply want to read and follow the discourse. SA's $10 fee created a barrier to entry, and the culture of bans being regularly handed out to bad posters encouraged people to lurk until they were ready to make good posts. Upvotes and downvotes as popularized by reddit and reddit-like forums served as a kind of substitute for that effect but they don't work well with as many users as those websites have.
But I get invited into discord servers for developers I work with, interests, I share, and I think it is getting to be more and more common. Some people will spend time in fake networks and probably not get a lot of value out of them other than entertainment. And others will gravitate to real networks for communication.
IMO this is long overdue. And I think people should/will eventually even pay for this feature.
> Powerful nation states will be more than eager to assist them in this regard.
I agree - world governments could consider even subsidizing this in some way in order to foster fewer anonymous botnets sowing discord. The web of trust could be useful to help bolster this - if only in providing the process by which new social networks could eliminate anonymity.
I hear a lot about SSI [1] from a family member who works on this stuff. I'll be honest - I don't know whether it's promising or dystopian. Maybe a little bit of each. But I optimistically think it could be a way to help mitigate some of the problems created by anonymity.
The loss of anonymity won't be a panacea -- for example: people with legitimate accounts bound to their identity today accept bribes to create positive reviews of goods. And the loss of anonymity comes with significant drawbacks: criticizing government and powerful people becomes more difficult.
Having a digital reputation that you can taint could put people on their best behavior. But then again it's a bit dystopian [2, 3].
[1] https://en.wikipedia.org/wiki/Self-sovereign_identity
I am having a hard time to imagine a scenario that is more dystopian
How don't see how making our lives more dystopian can be a net benefit.
How many of us expose different aspects of ourselves through different identities? Maybe most people are happy to show everyone all aspects of their lives, but I'm not, in part because who I am could get me or the people I meet in trouble, harassed, imprisoned or worse in some places.
Anonymity should remain a fundamental right. We have the right to live and expose different parts of ourselves to different audiences without overlap.
Anonymity is also a counter-power to governments. A government that knows all about its citizen is bound to abuse this power.
We see it in many democracies; elected governments enact laws that restrict freedoms to keep itself in power (Russia of course, Turkey, Algeria, but also EU countries like Bulgaria). Tools like government-issued ID that become mandatory to express yourself online are doubly dangerous: they make it easy to target those who cause trouble, and they can be used to remove means of expression (who can listen to you if you are denied an ID and there is no freedom of press to expose the abuse?).
The danger is not that you'll lose anonymity. The danger is that if you use anonymity, no one will listen to you. I'd wager a lot of algorithms will just shadow ban you. This is because you become viewed, effectively, the same way as the AI agent generating brand placement tweets for Bob's Bargain Basement Bar-B-Que and Bitcoin.
No one will know the difference between you and the corporate AI's. Only the verifiable humans will end up with opinions that anyone pays any attention to. It's like everything else will be a banner ad. Whether it's a banner ad or not.
This depends on people valuing who is saying something rather than what is being said. I'm not saying you're wrong, but I'm saying that if we're just trading in celebrity rather than ideas that I'm pretty sure we can also create celebrity AIs people would listen to.
Jokes aside, now that i think about it, "article summaries" are kind common on reddit, but i've never seen such an algorithm used for comments, yet. Guess Up&Down-votes are still the best we have.
Society makes things like blackmailing, etc possible too. In a truly free society we wouldn't judge each other for all the stupid stuff we do now; the majority of people are still extremely prude/immature sexually & will judge others for anything outside the norm in a heartbeat - hell even mentioning inside the norm sex stuff is still somewhat taboo.
As a gay dude involved in a few fun subcultures both in & out of bed it's so refreshing to be able to be so open about sex, if only everyone else was the same, about all topics. Y'all are missing out.
I've seen some pretty reprehensible behavior from people who put their full name and a photo of their face on Facebook.
I think the solution to all of this is smaller tight-knit discussion groups where all the participants know and trust each other. Echo chambers will inevitability form. I don't think there is any way around this.
edit: I find it very bizarre and very fashionable to defend the integrity of a verification service that does not exist, and to already dismiss the non-existent people who could suspect that this hypothetical system of verification was corrupt as conspiracy theorists.
My point was not that such verification systems are incorruptible. My point is that such verification systems are pointless because they will inevitably be perceived as corrupt. Whether or not the verification system is actually corrupt makes no difference. Such systems are worthless because people won't trust them. Any system will be accused of being corrupt by "both sides" of any heated debate, and they're not both going to be right. But it doesn't matter of one, the other or neither are correct. What matters is that the system won't work because people won't trust it.
I don't necessarily think that's the case. Remember the old "nobody on the internet knows you're a dog" cartoon? Well, I think that for a lot of things, nobody cares if they're talking to a dog.
People only care about people on the internet being fake in certain contexts. They'll very much care if all of their dating app matches are robots, because the end goal of a dating app is actually meeting someone in the flesh.
But I don't think that people on Nextdoor and Facebook groups particularly care - or would even notice - if they're talking to robots. As far as I can tell from my limited exposure, most people like getting caremad in comments. Even in local groups, it doesn't feel like there's much connection being made between neighbors; an AI complaining about traffic or agreeing/disagreeing with someone's political opinion would scratch the same itch as someone two towns over doing it.
I haven't, but my impression and anecdotal data is exactly opposite.
Suggested remediation: just, like, disagree :) this is a self serving recommendation though because I really enjoy seeing people’s differing perspectives
Heck, Reddit threads are often so predictable that they might as well be written by bots.
So yeah, most people just want to talk and feel that someone, something is listening to them. People rejecting your idea need to recalibrate their understanding of the depth and breadth of human loneliness.
The original New Yorker cartoonist forgot that people talk to their dogs too.
I've seen similar reaction in communities around enjoying art, even though I find those have less reason to care where the art comes from.
The entire thesis rests on this erroneous sentence in the second point: “This can be done in two ways – just move to invite-only silos where you already know everybody, or big platforms where the owners do the vetting for you.”
There are more options. We employ them when we are forced to; when it becomes cheaper than not.
We can sign posts or sign that somebody is real or to sign that somebody has earned rep, or that somebody has burned their rep… and subjectively score every piece of content that crosses our phone against our social trust graph.
We can rhizomatically scale social networks, can deputize our people in our extended social network to mark content as appropriate for kids or not, or otherwise filter. There’s no specific reason why we cannot grow a kind of social nervous system that has a kind of myelin sheath against noise and spam. It doesn’t have to be specifically only people we’ve shaken hands with or our like 3 closest buddies.
I more want to at least have that individual emitter be accountable for what they post; to establish continuity. I want to know that that emitter is say 3 hops from me, and is trusted by 12 friends in between, and has a general trust score of say 7/10 overall as an overall rating by my extended trust graph.
It is less that I want people to say good things, or be truthful or whatnot - I just want to know that they are real, that there is a human behind it, that that human has an opinion of some kind. The thing is that there are a ton of sock puppets that are not real - it's more about reducing the noise / spam rather than a perfect solution.
However, the real problem is getting to know new people or vetting messages from people you don't know. In the future the OP sketched, you can never be sure that an interesting new person isn't actually a bot. Knowing the public key of that person won't solve that problem.
Using cryptography to solve the fake persona problem only works if the key-identity matching problem is solved. It would be great if the fake persona problem was the impetus that managed a solution to the key-identity matching problem. But I have my doubts.
Notably the key-identity matching problem isn't technical. Its societal. "just do government provided keys" is technically easy, but society (rightfully so) is suspicious of this. Other solutions exist, with other trade-offs. SSL certificates have centralization, revocation, and weakest-link problems. PGP-keys have spoofing, verification, and usability problems (though I liked key-base's approach here). European E-id is an interesting step, one to watch, though I fear the EU bureaucratic system might make a crucial mis-step. I really like SSI based approaches, but SSI is mostly about using crypto when the key-identity matching problem has been solved, and less about solving the actual problem.
Some technical aspects that need solutions that tend to be un-acceptable are handling key-revocation, key-theft, key-loss (as in forgetting), and key-duplication.
I also really like the idea of using the keys to hold some amount of value, such that if the keys ever get leaked, there is basically a built in bug bounty to alert the key holder (since the key thief has the option to take all the money). This also gives users the incentive to manage their keys in a sane way.
Social key management schemes are also super interesting, and will likely be a part of future key management schemes. That is, basically, allowing some set of friends and family to re-roll or revoke identities that have been lost or stolen.
Slowly but surely, I think this future is coming. Lots of good people are coming at it from different angles, but basically all converging on the same general concepts.
You hit the nail on the head. Matching keys to real people can be done in-person for direct friends, then through a web of trust for indirect friends. For accounts/keys/personas you find on the internet where you don't have a chain of friends, you can either rely on "trusted" third-party attestation ("holder of key 0xdeadbeef earned a degree from this university") - you may never know with complete certainty if that's a real human, a bot, or an alt account for someone you already know, and that's totally fine.
The "problem" of matching keys 1-to-1 with identities for everyone globally (brought up by grandparent post) is a massive red herring that doesn't need to be "solved".
Web of trust sounds nice, but it hasn't caught on. I would say because it has trouble going beyond one hop of trust if you actually consider adversaries. In person physical confirmation works, re-keying is a major hassle though.
I do like the idea of a 'web of trust is good enough for declaring you are a person'. If you get a vouching system with a recursive revocation system, that might work well enough for establishing you are a person (though not well enough for establishing which person you are).
Trusted third parties have problems. They centralize power, either in the state, or in some non-accountable organization.
I think the way I'd think about this is to imagine say a small community, such as a town of say 5000 people or so. While you cannot know each person individually, you can know of people by reputation. People do earn rep over time, and they can burn rep. It is true that some people will be unfairly downscored, or unfairly upscored - but I'm not really trying to argue for those fine grained situations. What I'm trying to argue for is simply distinguishing the very bad actors acting out of pure malice from injecting fake news, media and 'yellow journalism' into human conversations.
True some real people will be downscored (I prefer to think of this as downscoring bad actors rather than 'blocking'). And true an AI can 'sound very human' - but an AI or a bad actor will struggle to build up a reputation over time. An AI can't shake hands with you, it is harder for it to prove it is human... Other bad actors will presumably burn their reputations if they spit out a series of offensive, misleading, false, inflammatory or toxic posts...
Note I am not necessarily advocating for crypto per se as a way to establish social trust graphs (a'la PGP or say Keybase) but I am arguing that there are other options that the OP did not raise. I more want to see a wider discussion around ways to filter malicious media that either "centralized systems" OR "small social clubs". I'm not necessarily saying it has to be a cryptographic solution... but I do think there are more ways to have what we want.
[1] https://trog.qgl.org/20081217/the-why-your-anti-spam-idea-wo...
You're badly misunderstanding the parent post - it is not proposing a technological solution to a human problem, but a technological enforcement of a fundamentally human solution:
> subjectively score every piece of content that crosses our phone against our social trust graph...can deputize our people in our extended social network to mark content as appropriate for kids or not, or otherwise filter
This is a social web of trust, where real people do the ranking and trust assignments - the cryptography and other technology just keeps track of bookkeeping.
The belief that anything will “solve” this seems naive when there’s 20+ years of proof of this being an “unsolvable” problem despite repeated technological, social, and legislative attempts. There might be a new normal established with new battlegrounds drawn and we learn to “live” with it, but I’m willing to bet non-trivial domes of money against there being any true “solution” here.
It's obvious to anyone with a passing familiarity of WoTs that you seed your web with people you know in real life. GPT is not "difficult to distinguish from a person who’s confidently wrong" in real life.
If you are only accepting direct confirmations, this means you are only going to talk to people who you meet in person. This is totally fine and will work, but then you don't need any new tech -- just ask for their email / phone / nickname on your favorite social site. Or make a private forum (or a Signal/Telegram/Whatsapp group) and invite them there.
If you are accepting indirect confirmation, then once the network grows big enough, there will be bots. Maybe some of your friends meet Greg, director of marketing for Widgets Inc., and correctly confirm him as real human, and then Greg will confirm an army of GPT telemarketer bots as "real humans" so they can do the sales and earn Greg a bonus. Or maybe your good friend gets malware on their computer and their key is used to confirm a bunch of keys without their knowledge.
You seem to be intentionally or otherwise completely missing what I’m saying. PGP just establishes ownership of a private key. It doesn’t say anything about that person then choosing to sign the output of GPT or giving GPT that private key to do whatever with. And GPT can mimic whatever writing style you give it. Not hard to imagine giving it various writing samples of yours personally to learn from and imitate. So please explain how web of trust solves anything for that. Aside from trying to keep track which person in your personal network is a spam vector, because there are super connectors with thousands or tens of thousands of real-world contacts and logistically that’s not realistic to manage since most people aren’t cryptography nerds.
There’s also a bigger fundamental problem with web of trust in applying it here. Trust is not binary in that way. If I trust person A and they trust person B, trust isn’t a transitive property so in reality there’s nothing we can say about my trust about B. Also, trust isn’t binary. If I trust person A about specific science topics, that doesn’t mean that trust extends to other topics. Trust also isn’t static and sticky whereas it is generally treated as such in computer systems where trust needs to be scores and revoked automatically somehow (and then we’re back to an AI war to detect abuse and better GPT to avoid the abuse). This is also ignoring that trying to model human trust webs with a CS model that doesn’t work anything like it, isn’t a good recipe for success. Also, human trust webs have massive trust and scaling problems) (cough Theranos, WeWork, FTX, Madoff, etc etc). Notice how web of trust always sticks to basic cryptographic primitives which are easy to write papers about and solve academically but not a solved problem by any means in terms of defining what trust actually means or how a web of trust works in terms of AI content. Obviously PGP has been around forever and AI is a bit newer so maybe there will be interesting work coming out of this space at some point. AFAIK today web of trust buys you bupkis in terms of fighting GPT spam. I would recommend reading up any number of articles that discuss why pgp and signing parties failed. It’s not purely a UX issue. The bigger problem is that even in a “trusted” system, fraud arises spontaneously because it’s the prisoner’s dilemma problem - there’s a material advantage to perpetrating the fraud and there’s a better advantage to helping perpetrate fraud (much hard and longer political process to revoke that trust).
As a more concrete practical demonstration of this failure. Consider that certificate authorities, which are assumed to be “trusted core signatories” in a PKI system. A PKI systems is actually the same as a “web of trust”, it’s just that I delegate verification to a third party. As cryptocurrencies should have shown, people still prefer having a virtual account in a bank to manage those funds/ lower fees. Similarly, people will outsource the complexity of validating identities (CAs signing carts for websites). CAs have repeatedly abused this to the point where afaik the security community generally acknowledges that CAs are generally worthless - even the “good ones” struggle to do verification at scale and there’s so many CAs in typical lists that it’s basically guaranteed that there are malicious actors. And we know decentralization doesn’t actually work for end users because it’s too complicated a mental model. People want a named intermediary that they delegate responsibility to. That’s why most people defer their CA validation to browsers and operating systems. PGP would work similarly so now you’ve got people delegating key trust to Apple, Microsoft, Google, Signal, etc etc. and nerds who use open source verified key managers and maintain their own infra to manage these lists. But that’s not a representative sample of what end users will accept at scale. So you’re back to centralized control, which will be better than status quo as OSes and browsers realistically are more resitant to handing out broken signatures. And of course maybe better algorithms and methods will be developed to solve these shortcomings.
But a lot of these issues have existed and been documented for a long long period of time independent of the vague idea of using it as a GPT detector / blocked. I’ve been around the Bay Area for 10+ years and I remember having friends who thought this would take off any day now and worked really hard to make it happen, hosting signing parties and whatnot. It didn’t and I was pretty confident it was a pure nerd activity that wouldn’t have any impact in its current form (regardless of the UX challenges - the problems are much more fundamental and worse). Web of trust is seriously hard even in the simplest possible state which is PGP and that’s failing miserably despite being around for a very long time.
Would you agree that it’s on you to provide some supporting evidence on the claim that
A) I don’t know what I’m talking about
B) something more than hand waving “sprinkle some decentralized cryptography here” and actually explain how you solve the human problems that are so important here + why PGP has largely failed but suddenly it’s going to find a second life in GPT prevention.
It's not a very compelling argument for me that a (very short) 20 year history of failing to solve this problem means that it cannot be solved.
I'm willing to bet you $1000.00 that this will be completely solved in 10 years. In 10 years we will know if content we consume is "fake" or "real". It may still be offensive, and harmful to gullible people - but it will all be scored as to how real it is. Real will be defined as the likelihood that the content comes from a real human being type person, as agreed upon by other persons in-between you and that person. Content one hop away from you, from a friend, will have a score of 100% real, and content many hops away will have lower score. You will probably start your day by sorting the content you consume by the likelihood that it is real.
I did actually work on PGP - and I'm willing to concede it didn't succeed. But DNS works and bitcoin works (technically). So we do use various flavors of cryptographic trust to make sure that actions in a network are "real" versus "forged".
And yes - it's true that bots can creep into a social trust graph... so yeah, it make take effort to keep pruning the weeds in the garden in a sense.
Note in some ways I'm not really trying to argue FOR crypto per se - I'm just saying that the OP should at least critique crypto if they want to make the thesis they are making. And I'm arguing that it is a big omission to gloss over the utility of crypto and the argument will be hard to make that crypto is not a significantly powerful modifier to the original thesis of the OP.
One of the most interesting technologies the crypto space is working on are the SSI (self-sovereign identities).
https://en.wikipedia.org/wiki/Self-sovereign_identity
Even if the whole idea of cryptocurrencies is found to be a dead-end, the fact that it invigorated the research and development of SSI will change some important things about how we operate online.
There exist prototypes of tech that allows you to prove you are indeed a unique human being online [1], and reveal nothing else about your identity. Most importantly, this tech is not owned or controlled by any FAANG or government, it's an open protocol just like email.
I have listened to a podcast with an expert researcher in AI, and I remember him saying that he predicts some form of cryptographic identity will arise in order to help deal with the bot problem
[1] https://worldcoin.org/ (note, I don't work for them, and have no idea if this will be the tech that finally breaks out, I just think they're the furthest along of any of their competitors)
> Fourth, we'll see a resurgence and even fetishization of explicitly "offline" culture, where the "Great Logging Off" becomes literal
I actually think the author undersold this a bit, because even once you've siloed into social buckets with people you already know- as long as it's digital communication, you can't quite be sure it's really them. Even if you're voice-chatting while playing a game together, even if you're video-chatting, we'll reach a point where that could all be faked.
You could rely on cryptography to make sure someone is who they say they are, but that requires extra hoops and a basic understanding of how to use it and what is and isn't trustworthy. Most people probably won't bother
So at some point only physical contact will be fully reliable
(Until, I guess, Musk's brain-computer interface takes off. Then nothing is real)
Actually nothing would stop people from signing their own comments today and sometimes it is done.
With e2ee messengers I feel pretty confident that a message comes from the right / real person after I verified their public key.
a few generations ago social networking would have seemed infeasible because it would require wide spread literacy (along with many other reasons of course). widespread private key management doesn't seem that infeasible to me.
Which is good enough for many applications, I think. With friends and family, I am pretty confident, that none of them deploy a personal assistant to answer my encrypted messages. As opposed to a messenger where the service provider can inject ads into the messages.
How do I know that a person that I speak to IRL is really saying what they think or even really is who they claim to be if I don't know them well? A rest of uncertainty always remains.
But then there are twins, so... The "problem" pre-existed.
That means that a clever script would be able to figure out who a persons real friends are (that info is easily obtained already today), and create personalised videos from said friends. In such a scenario it might not be the Great Logging off, but it would certainly be a situation where you think "I had breakfast with my friend and he said X to my face, so I can believe it.", but anything online can be considered noise.
Text conversations are trivial to fake, and WhatsApp could trivially MITM a handful of conversations and tell me my friend said something different and I'd never know. But they have an incentive not to do this at a large scale, because there are benefits of running a worldwide trusted messaging service, and they would fear people would move to alternatives when millions of messages started being adulterated.
What I'm afraid of is, this will dull people because the computer models would not have the fidelity of a real person(simply because only small portion of humans thoughts are expressed and even less recorder). Many people will prefer to interact with this shallow model of humans instead of real ones in similar manner of porn replacing real sex.
It will create armies of people who have no real skills with working or even interacting with humans. Representing people as avatars and a nickname already created huge issue with perfectly nice people forgetting that they are interacting with real people online, turning into huge jerks. I can imagine how horrible in could become when people are completely disenfranchised from reality.
To this day, when people search for me, they find an old photo and my name associated with a fraudulent company selling shit products and services.
On a related note, a few of my friends have taken down their long standing personal sites and blogs for this very reason. A few bad actors took their photos, their About pages, a bunch of their blog posts, and used them as if they were their own—quoting them, tweeting them, referencing them in "their work", etc. One of them got wrapped up in an elaborate hiring scam where someone used his photo, email address, and employment history from LinkedIn and got all the way through an interview process.
The whole situation was so jarring that he wiped everything—almost a decade of writing, his LinkedIn profile, his Twitter account, all of it.
Side note, I’ve long thought that identity verification is the _one_ area that a public blockchain could actually add legitimate value to. But I’ve seen no efforts in that direction to date.
What does Web 2.0 have to do with any of this?
I think we’ve already been seeing it, just done badly. Most content marketing is somewhat automated. The tools (ChatGPT) have made a leap and the internet will be riddled with it and it’s brethren.
What’s interesting though is that future AIs will train on AI generated content which trained on AI generated… etc. Possibly missing legitimate feedback?
AI is inevitably unleashed on AI in a game of cat and mouse driven by that unholy god “marketing“ thus eventually rendering the internet useless in a race to the bottom of legitimate human interaction.
In the near future, It's likely that this is no longer going to be the case and in the moment that most of your email is spam you are going to stop trusting everything you receive.
Even right now inside corps people already (should?) distrust and double checks everything that comes from another domain.
I think this sense of disengagement is what the author is trying to metaphorically capture with the idea of “logging off”. It might just be my family and peer groups, but I do sense this in general. The internet and its various “social” constructs feel more and more like they’ve crossed that line in dating relationships where the other person becomes “clingy/needy” and it starts to grow wearisome and cringy. I think that causes people to quietly disengage.
The enabling technology behind micropayments is ironically looking like a CBDC a lot more than Bitcoin (+other cryptos), for scalability reasons.
Spam is a moving target.
Which realistically means everyone will log off and touch grass instead of pay to interact with other verified humans.
The logging off is a pipe dream for some people, even though they know that humans are informational creatures and that the "offline world" has never lived up to the desirable one since the time books were invented.
Doominess and gloominess is the real disease of our times
Seems more like extremely wealthy sociopaths & psychopaths that thrive in a capitalistic system are. Don’t think people would be so doomy & gloomy if it weren’t for those with the means to do so trying to shape society into their dystopian dreams.
Two examples if needed:
Thiel & Davos family
Doom and gloom have been around for longer than a money based economy, too.
If you take any popular medical fact (for example, about diets), you will find countless published academic articles supporting and against it. This extends into other fields like physics, sociology, and politics too. But the contradictions there (while present) are not as overtly visible.
Many popular news outlets are leaning towards reporting in a "flavor" their audience expects and mixing opinions into their reporting.
By some studies, 80%+ of internet users admit to being duped at least once by fake news or misinformation online in 2022. That's the misinformation they spotted. 50%+ Americans say they read fake news online regularly.
As for logging off, according to Reuters Institute, 41% of Americans actively avoided news in 2022. And anecdotally, my friends and I have withdrawn from social media like Reddit, YouTube, Facebook, Instagram, Twitter, and so on significantly this year. It definitely feels like a lot of information online is misinformation, either synthesized intentionally or "broken-telephoned" into fantasy by mainstream media.
I do not know how to deal with this personally but disengage from most of the online world. The signal-to-noise ratio of reliable information in many regions of the internet is meager. I mainly read Hacker News (which also sometimes features contradictory research or news), some academic journals, Wikipedia, some forums in my professional field, and some RSS news sources. Everything else on the internet has become too awash with contradictions and misinformation.
I never thought I'd have to retreat so far from mainstream media and social media. Mass logging off is going to impact these areas of the internet seriously.
I am scared of what AI language models will do to professional blogs, news, and academia. Editors and peer-reviewers there are already overwhelmed with unreliable information.
This is speculative but we may see a very strong pivot from the "social" aspect to the addictive content aspect in publicly listed social media companies in order to continue to appease shareholders, which could, possibly, accelerate this potential reemphasis on real world communication and connection.
I see SIGNIFICANTLY more homeless people in the two cities I frequent than at any other time in my life, there is a snow covered tent city half a kilometer from the glass monstrosity Google has built here and mostly doesn't occupy. Facism is on the rise globally. A third world war is possibly percolating in Europe. Google is soaking up all the bulk of revenue that used to go to publishers. Amazon has destroyed at least half of bookstores.
But sure, I guess gloominess is the issue. Does all of the above make you happy somehow?
Seems to ring true more and more :)
AI should be able to pick the most relevant information for you and present it in the most compressed form.
And connect with other people, feel catharsis, to feel less alone, loved, engaged, interested, and the other things that make us human. AI will not give us that and in fact undermine that. Perhaps then AI will push back to the "real world" for these interactions, as the article opines.
Is that true?
I rely heavily on search engines and recommendation systems, but these are gamed by SEO spam, and flame-war political content.
I want the quality of the content I see to be as high as possible, irrespective of whether it has an AI or human author, and if an LLM can write an excellent Stack Overflow answer/movie review, I'd be interested to read it.
> In which the internet gets clogged with piles of semi-intelligent spam, breaking the default assumption that the "person" you're talking to is human.
Neal Stephenson's novel Anathem briefly mentions something like this as part of the novel's worldbuilding.
I'm not sure this assumption holds true enough: incredible numbers of humans have already substituted vast swaths of direct human-to-human contact with the gigantic text-only condoms provided by the internet. We've already traded in superior quality human contact for inferior quality contact--for the bonus of more specialized content which we probably wouldn't get if we knocked on our neighbors' doors.
This doesn't necessarily invalidate many of the proposals in this article. But what about the dimensions in which AI provides or will eventually provide a superior substitute to human-to-human contact?
I think the internet will soon be clogged with bots competing with other bots just to get a few seconds of human attention. At that point the operational cost of bot farms will outstrip the benefit and so we’ll settle something like 60-70% bots maybe?
Speaking of myself I’ve gone back to reading and enjoying physical books after being on kindle for close to a decade. I never thought I’d go back to physical books but now a days reading on kindle feels a bit too lifeless so to speak, so here we are.
Additionally, when the internet is mostly created by bots, why would I go online to look up a recipe on a bot cooking blog that was generated yesterday when I could have my pocket bot generate a brand new, tailor made cooking blog for me whenever I want? Ditto for instagram/tiktok. Generate the next image based on how long I looked at the last one.
At this time we’re still safe by doing “click the joke that’s actually funny”. Once an AI figures that one out, I imagine we’ll have to prove we are made of meat.
Eliminating the "noise" is pretty easy, just take the "mute everyone" approach. No default subreddits, use ad blockers, actively disable "suggested" content where possible, and periodically review the things I'm getting my "signals" from.
The problem I'm struggling with now is verifying authenticity of references and information. Every solution I can think of (say a chrome plugin that does analysis and can watermark text/video/image content with a "FAKE DETECTED" banner) just becomes a cat and mouse game.
Whoever has a collection of current pages has the data set to be a gateway to authentic information.
If this is about ChatGPT, we should remember that it is not available in a way that you can build your business applications on top of it in any reliable or meaningful way. It is a very powerful demo of what AI can do, but nothing more yet. Also, OpenAI is spending a lot just to let you and me give it a try and this will not last long. IIRC, they are burning something in the range of 3 million USD per day.
The writing is on the wall.
[1]: https://twitter.com/michael_vandi/status/1607422866416619520
I don't think there's a 100% perfect solution to prove you're speaking to a human other than having the conversation offline. Even then nothing stops someone from wearing a smart earpiece and replying to you with whatever an AI is whispering into their ear.
If this becomes a huge issue you'd better bet that big tech is going to invest a lot of resources to solve it. The simplest solution would be to get better at verifying real human users, for example with better adversarial AI systems to detect non-humans or more document-based authentication tying your real identity to your user account (and thus any bans for misuse going to your real identity).
So, yes, there might be some temporary lemons failures on some platforms, but I would not expect a big logging off to happen on the long-run. After all I think this is a cat-and-mouse game of superior technology in producing / identifying spam.
Written language has patterns, and your response and mine also follow these patterns. Language models like GPT-3 and ChatGPT can learn these patterns because there are finite ways in which humans write text. ChatGPT responses are easy to recognize because it was fine-tuned with specific data to meet chatbot requirements and provide Q&A style responses. In the near future, these language models will be fine-tuned for specific purposes, such as spam, and will mimic human text patterns in a way that will not be distinguishable from real humans.
Who this effects is jobless people.
People here have experimented with using GPT-3 to generate comments and content shared here, well before chatGPT was released; while it can sometimes be a bit suspicious, it can also get up-voted: https://www.theverge.com/2020/8/16/21371049/gpt3-hacker-news...
Assuming the question was
> How much time in day, do you think you spend reading what random people you dont know are saying?
Rather than "so what?", which I will come back to later: For me, probably several hours per day. It's not just here, or twitter/fb, though that's probably an hour by itself, it's also:
• Blog authors (parasocial relationships!) • Newspaper writers • Wikipedia editors • Review writers on Amazon, Google maps • Question and Answer writers on Stack Overflow • The first few comments under some Youtube videos • Comments sections on news stories
(Technically also book authors given the way you phrased that, but that's a different issue and current generation text generators make no difference there anyway).
Now assuming the question was "so what?", because that dovetails into the rest of your spiel:
The problem is not usefulness to you, it's "can this be used to automate convincing you?".
Make a half-decent sales pitch personalised at whoever for whatever: "Boiling eggs? Yeah that's easy, but I find organic is the best and well worth the cost", or "Oh, you're visiting Berlin? Yeah, I had a really great time at $insert_restaurant_here" or "Don't fall for the Lib Dems, they keep saying they can beat the Tories but look what happened when they got into power. That's why I vote $insert_uk_political_party_here". All personalised to just you, in the form of a friendly looking and natural response that probably seems more natural than any of my examples, because chatGPT is only reliably worse than domain experts and I'm not a domain expert in sales.
Also: You replied to me, and I'm a stranger, so we both know you engage with strangers on the internet, QED there's a place where you might get a bot trying to sell you a product or idea while pretending to be human.
Don't make the mistake of thinking the status quo is a good reference point here: Current advertising categorisation on social media sucks (easy way to find out: make GDPR request to FB or Twitter for all the data they have about you. They think I'm interested in languages I can't speak and politics of countries I don't live in).
Well that constant will be transformed to a variable. While GPT can recognize and produce outputs of a language, this tool, or similar tools can be used to produce another language, totally from scratch or a variant of an existing language, instantly.
In the language context, i would argue it is a lot more difficult to produce effective spam than in the image context or video. People will change their language, the model will have to be trained again in the new language, and that is until people up their game and change language again. This is happening already, young people use different language in the internet than older people. I would be surprised if we don't witness in the near future, the internet of a million languages.
Maybe I'm wrong, but is there any western culture that values these things?
Maybe east US? Texans etc.
I think countries that are accepting eastern migrants will fare better, since eastern culture typically value these things more, thats in addition to solving the aging solution problem, something westerners are not willing to do(having kids, enough kids).
Consider this classic XKCD: https://xkcd.com/810/
There are 2 possibilities: The AI can comment as well as a human, or it can't.
If it can comment as well as a human, our experience of using social networks will not be degraded. We still get the same experience as before.
If it can't, then its contributions won't get as many likes/upvotes, and they won't be particularly prominent. Most content users see will be human-created.
So I think the key question is whether AI will be able to manipulate the liking and voting systems. But there are methods for preventing this: ignore votes from accounts created after ChatGPT's release, or for a site like Twitter with paid accounts, ignore votes from unpaid accounts. It's not even clear what ChatGPT adds beyond a human voting ring, and social networks presumably have lots of experience dealing with voting rings already.
I think you're massively underestimating how important motivations are. Bot accounts that can perfectly mask as human but whose prime directive is to recommend their sponsors' products would ruin trust in a heartbeat.
Also, upvotes are an imperfect measure of how good the post is and AIs would be able to game the measure.
Not too late to create training data by which the social media platform can detect and remove subsequent scams. Especially if they 'report' rather than downvote.
>A million AIs that are indistinguishable from humans but occasionally throw in a comment about how they got sick from Pepsi could make things very profitable for the Coca-Cola company.
How is this different from Coca-Cola paying influencers to post negative stuff about Pepsi?
>Also, upvotes are an imperfect measure of how good the post is and AIs would be able to game the measure.
How is this different from humans gaming the measure?
How is that going to happen? We're assuming the AI posts exactly like a human until the comment about Pepsi comes out. It'll be impossible to distinguish from a human, by assumption. Or are you going to just have the site reject everything which mentions Pepsi negatively?
>How is this different from Coca-Cola paying influencers to post negative stuff about Pepsi?
AIs don't need to be paid.
You could have a reporting system.
>AIs don't need to be paid.
At scale, this sort of service is not going to be free. But I suppose AI could make it a lot cheaper.
Coca cola can't have millions of influencers on their payroll (or at least, it wouldn't be economically profitable for them to do that). An AI could easily create and post on accounts on a scale that dwarfs real human-generated content.
https://www.ftc.gov/business-guidance/resources/disclosures-...
^ Wouldn't this apply to ChatGPT bots as well?
Out of fun I posted a few "posts" generated by ChatGPT to my local town's facebook group (complaining about the quality of snow this year, nonsense like that).
The posts got a lot of engagement (100+ comments each) and the only one who figured out it was AI generated was a girl from the local university who studies computer science.
Diversity: because language models are in some sense capturing an average (even if that average can be from a subset of training data), its comments will not be very diverse. I suspect if you try sampling away from the center of the distribution, you’ll find the quality degrades to nonsense.
Volume: the biggest problem is that you’ll end up with complete overwhelm of plausible comments. Language models cannot reason, which means they are incapable of producing the best insights (they can probably provide insight through “monkeys at typewriters” effects).
So the smart human who has analysed the subject really well, who currently has to rise above a certain amount of background noise, will now get completely swamped by it.
That is just an odd take and maybe hints that this is a generated article itself.
The old timey neighborhoods where people wave at their neighbors and have things like block parties is what they’re talking about. Communities built around human interaction as opposed to a hundred story apartment building where maybe you can recognize a couple of your neighbors if you saw them on the street.
Second, I think people will start to put a premium on accounts being "verified" as genuinely human. This can be done in two ways – just move to invite-only silos where you already know everybody, or big platforms where the owners do the vetting for you. Lots of platforms will simply take a knee-jerk reaction where they just up the amount of surveillance. "Now that there's so much bot activity, everybody need to upload their passport and driver's license, for your own safety of course!" Powerful nation states will be more than eager to assist them in this regard.
I think people would put a premium on the abundant BOTS, actually. If we look at wall street trading bots and so on, it’s actualy far more likely what will happen is that organizations will start to PREFER bots. Eventually, we’ll have self driving cars dominating all traffic, bots generating nearly all online content, and humans PREFERRING them. Because why not have a seeminglu flawless outcome every time, a bot teacher with limitless memory and eventually learning the habits and knowldge of 917 people in the classroom for a personalized approach etc.
Why have a human lawyer when a bot can go through 300 arguments and pick the one that fits that particular jury and circumstances?
Take VCs for example. They want to see a community with hockey stick growth and recurring revenues. Why have a community of humans? You can now replicate this growth entirely with bots. Twitter’s growth will look pedestrian by comparison.
You may say, where’s the money in this? The bots don’t bring any capital to the site. That’s true today, but on wall street they account for the vast majority of trading since they can out-trade everyone and they also can swarm to collude with each other.
It’s not hard to imagine that bots will start to actually receive payment for the content they produce, just as people pay each other on Patreon etc. or pay for copywriters, designers etc.
At this point the bots would be programmed to amass credits on a site, which they can turn into capital. They’ll have realistic 3D models conversing with you, modeled after really anyone. Why have onlyfans? Why have instagram models? Why pay for newscasters? Why have actors in movies? Why have salespeople make calls? Why have humans do anything online or remotely AT ALL?
My prediction is that bots will control nearly all the capital, produce nearly all the content, and humans will prefer to interact with bots for romantic conversations and everything else.
The problem will be that these bots can easily swarm at any time and dominate any group of humans, destroy reputations, move public sentiment etc.
People’s public opinion would be dominated by bot swarms.
Those who try to somehow prevail in their local communities or control the bots will be essentially disrupted and reputationally destroyed by the botswarms. It would become so cheap to destroy any number of human reputations that people would set swarms up to do it for the Lulz.
Governments that try to ban these bots can themselves be easily overcome by botswarms. They’d just send endless numbers of fooled and outraged people to their doorsteps and wherever they may be eating. They will know all their whereabouts and all their proclivities. And besides this, thousands of people will be convinced by deepfakes and fake articles online that this politician is doing something ridiculous or embarrassing. Every time the politician would try to do something, they’d get a call or visit from N constituents demanding their attention. The botswarms can just DDOS all humans who try to stop them basically. And at pretty trivial cost.
You here commenting on HN … imagine if in 12 months from now, half of all accounts around you are bots, and they all collude against you and people with your viewpoint. What would you do?
The problem is “logging off” is about as effective as Ted Kazinski trying to escape civilization by moving to the woods. Are wild animals able to escape encroachment by humans into their habitat?
Concepts like ownership, privacy, authority etc. are not laws of nature but deeply rooted behavioral patterns that proved to be useful in some global sense.
What happens currently with the explosion of "digital society" is that, largely due to greed, certain first mover groups are dismantling all that - basically because they can given the widespread digital illiteracy and the lack of any regulatory mechanisms.
The response will be indeed people trying to log off. But if you consider the callous insistence with which ruling elites impose e.g., cash-less, fully digital transactions, its arguable that the escape hatches have been locked already.
Unless some new benevolent dynamic shows up that can harness the digital domain more congruently with welfare prepare for a period of great social unrest and the destruction of major social contracts.
Btw I am not defending cash technology as such, I am offering an example of why the "logging off" option might be less available than what people think.
Social media apps have already manifested themselves (in the East) as "super apps" and purportetdly this might also be the target business model for some western outfits
Anyway... "SPAM" may be over. That is, I suspect that whatever low end, semi-legal marketing drek produced by GPT-like will not read as "SPAM" to us. Rather, we'll get whole new species of junk marketing, telescams and whatnot.
My prediction is that the first piece of weird will be AI autocomplete. A GPT that takes more context (eg all company emails) built into email & DM clients... that will escalate quickly. The line between human and machine will blurr.
The social graph of reputation systems, has a necessary constraint, it has to be retained in the blockchain so as to be incorruptible by every human power. As soon as a node can be planted by a malicious actor, as part of the social graph, the whole of it becomes invalidated.
Mr Craig Wright calls this "The Metanet", George Gilder calls this "The cryptocosm", other people call it "Web3", Balaji Srinivasan calls it "The network State". I call it, "The internet of one million governments".
[1] https://lichess.org/forum/lichess-feedback/long-playing-play...
Edit: added Balaji Srinivasan
It's a directed graph that indicate who trusts whom. If you identify a malicious actor, you only have to re-validate the nodes that receive their trust from that actor.
You can also use weights, e.g. to reduce the influence of new nodes.
I'm also suspicious of claims that an algorithm is uncorrupted by human power. Every computer algorithm is made by people who have human motivations.