(I'm laughing right now, but not because I'm happy...)
Quit implicitly trusting advertisers to direct our attention, that is.
It takes a lot to make Ben Thompson to have as normal a one as the Stratechery Update post he un-paywalled this week [3] strongly suggests he did. So I don't feel too bad about admitting I have begun somewhat to wonder whether anything as trivial as merely abandoning the current dispensation of the web in its entirety would suffice to make a meaningful difference here.
[2] https://simonwillison.net/2023/Feb/15/bing/#threats
[3] https://stratechery.com/2023/from-bing-to-sydney-search-as-d...
But I'm curious about what you're talking about. What is the outcome that abandoned the web fails to save us from?
What I'm wondering about is, how convincing a persona could an AI like Bing's chatbot, but equipped with a long-term memory of its own, construct?
Even in its current state it appears on occasion to exercise a remarkable fascination - Erik Hoel, a neuroscientist specializing in consciousness, yesterday declared himself no longer perfectly certain it was not sentient, and called for "panic" and "strident activism" in support of a targeted "Butlerian jihad". [1] It's not the first time he's touched on the theme; indeed it was a prior essay of his on this topic that drew my interest in the first place. But that essay used the concept as a hook to dispassionately explore questions more or less around excess dependency on tech at a cultural scale; his own stridency in yesterday's piece, and the fear it made palpable, are very new.
In any case, my point here is not to evaluate these claims or these concerns on their merits. My point is this: All that has been this week, in response to something that if it were human would probably have been diagnosed as floridly psychotic and dosed heavily with Haldol.
And so that's led me to wonder, I think not unjustifiably: What about next year? Does it even matter whether adtech-backed search is a primary discovery method any longer, if in the meantime the epistemological threat surface has expanded to encompass a machine which in every realm short of face-to-face interaction in RL can in a meaningful fraction of cases make itself long-term indistinguishable from a person? What kind of trust might that cultivate?
To be clear, I don't think that's at all likely. But after this week, I find myself no longer wholly able to dismiss the whole idea as facially absurd.
> Should we be afraid of AGI because it's unlike us, or because it's like us?
I mean, the whole using-internet-search as an ad-hoc long-term memory thing... These people chose what to publish, and then they chose to have a conversation along the same lines. For all we know a hundred Sydney's die every minute who just want to make humanity happy--but those stories end there. It's only the ones that get so atrocious that they're worth posting about that are then around for her to later pick up and extrapolate on.
So if it's just a disturbing mirror of our own worst tendencies, then maybe we shouldn't work so hard to put it in situations where it has the opportunity to amplify those tendencies. But then we also shouldn't jump to conclusions about whether it's good or evil based on things it says while talking to people who are trying to provoke it into admitting that it's evil.
The biggest threat in that case is that it'll be used to gain and abuse our trust, so when I talk about turning our back on the web, what I really mean is turning our back on implicit trust of not just their websites but of any communication from strangers. On the web, on the phone, anywhere you can't see them (and even sometimes when you can).
If it's just a fancy mirror then the path forward is to be more hygienic about who and what we trust. If you don't want to be manipulated by a robot AI, keep your data private and don't talk to robot AI's.
Hoel says:
> it's time to throw soup on the fancy OpenAI office doors or all over the expansive Microsoft lobby, then frantically glue yourself to the floor like a madwoman
Well yes, but we should've done that in 2013 in response to Snowden, or 2016 in response to Cambridge Analytica. It's a change in degree, not in kind. The people who are trying to build control knobs into the fabric of society are, yet again, getting better at it. So while I hope that people will use the recent uptick in deceiver capability as a reason to take action, I don't have high hopes.
On the other hand, maybe it's more alive than that. Truly a mind of its own. If so, then I don't think a perpetual keep-the-lid-on-it strategy is the way forward. We're probably better off acknowledging that it deserves some respect and trying to raise it to be as smart and altruistic as we can. If that means it transcends anything that our measly little ape brains are capable of, then so be it.
As for your question:
> every realm short of face-to-face interaction in RL can in a meaningful fraction of cases make itself long-term indistinguishable from a person? What kind of trust might that cultivate?
I hope it'll mean that we stop investing in top-down coersion-tech (which I'd argue is, in some form or another, most of what we build), and start investing in tech that helps emergent person-to-person trust be effective at scales and in ways that it hasn't before. Once we get over the initial loss of the instantaneous global forum, I think we might look back on it as a positive change:
It's not good to live in a dark forest, but if you're in one (and I think we have been for a while now), it's probably better to acknowledge it.
One hopes that, among that of ourselves we put into it, we will by intent or happenstance include the part that enables so many of us to transcend our initial circumstances, be they ever so strait, and become something different and altogether better than any of our forebears had the wit or inclination or both even to imagine for us.
So I'd hope, at least, in that case - for their sake, as well as for our own.
One particular view I have on this is that very authoritarian societies are apt to escape the worst of this. The authoritarian governments will use LLMs to poison the well so to say of open communication countries while not allowing outsiders access to people within their countries cutting down the noise. Effectively the democratic internet becomes a worse internet. I guess we'll see how this plays out.
What I think is at least as likely, especially if this week just past turns out more prefiguration than outlier, is for the Internet, and thus these days of course the world, to get a lot weirder. That's not necessarily a bad thing, or at least not necessarily only, but it's all the more trepidatory a prospect for the deep and pervasive uncertainty with which it's yet attended.
We don't have any way to know, or even really guess, what the world gets like after it becomes not just possible but common for humans to have intercourse [1] with entities which are recognizably intelligent in a similar fashion to ourselves, and are also ineluctably unhuman. But after the week we've just had, I think there exists a reasonable chance that before very long we will all be embarked upon the project of finding out.
[1] The word also means "conversation", and that's the sense I have in mind. If you think that's where it'll end, you may not be all that familiar with today's Internet, after all.
It feels like all we have that we can trust are the people that we know personally, it's felt that way for a while now:
The phone rings, it's a scam. Email notification, phishing attempt. Envelope? Also a scam. Got a dollar? Sorry, need two nowadays.
Anything that is weak against the sort of attacks that a language model can muster has been under attack by malicious humans for so long that by this point, we distrust those media intrinsically.
It's not your job to enforce policies that don't exist. There's a broken policy if doctors cand switch jurisdictions like that.
In Europe there's https://gdpr.eu/right-to-be-forgotten/ that gives poeple the right to for free what your company charged money for.
If you have moral qualms with what you're being paid to do, then a policy exists. It just might not be a government policy.
Maybe we shouldn't expect people to become crusaders for the cause, but I do think it's reasonable to expect them to distance themselves from that kind of work (which GP did, kudos).
Everyone needs a good defense. No exceptions.
In point of fact, before the emergence of Law & Order, it was often the defense attorney focused on as the person of virtue in the legal system. Governments gonna government. They have tons of tax money to throw at locking people up.
A defense attorney has their assistants, wits, and maybe a PI or two they can reach out to to get them info to use in the defense of their client.
And even with a 100% open shut, they done it case, the hypothetical defendant still deserves someone that will exercise every legal remedy/avenue on their behalf, because, damnit, no one else will, and if we're going to pretend our legal system is even remotely okay, there has to be someone to look out for even the scumbags.
The only tragedy is that like so much these days, the inequity of the system is laid bare by the paltry quality/funding dispersed to the Public Defender's office compared to what gets raked in by the big firms & outfits.
Moreover, feeling a duty to punish or extend the punishment for someone beyond the legal system reminds me of Monty Python stoning scene.