Joe Rogan Issues Warning After AI-Generated Version of His Podcast Surfaces | Hacker News Reader

Joe Rogan Issues Warning After AI-Generated Version of His Podcast Surfaces

zerohedge.com

161 pointsuberdru171 comments

Personally I love this new AI voice content booms. My favorite is the AI presidents discussing/playing/rating stuff [1], Attenborough narrating warhammer [2].

While slightly silly, I think in future people might start to take internet way less serious when you can literally make things up. Which I think is a good trend. People getting depressed or so consumed by the internet culture that they loose all interest in the real world have been a trend I've disliked. We might be just going back to the time of early, anonymous internet forums time where most people used to it just didn't take it too seriously, and a lot of communities were closed /invite only.

[1] https://youtu.be/IkaAZE_UGMo

https://youtu.be/q6ra0KDgVbg

https://youtu.be/iAq-yg72GWw

[2] https://youtu.be/X6RCLJ4pDaw

>> We might be just going back to the time of early, anonymous internet forums time where most people used to it just didn't take it too seriously

Man I was just thinking about it lastnite.

Back in the day we were (at some level) respectful to each other in places like IRC and forums, even tho nobody knew each other. And when some idiot started to f- around you had IRCOps and moderators, so it was somewhat easier and straight forward to maintain. That is something that I really miss, the old IRC days.

Today people show their faces, you sometimes know who they are, but they are at times completely disrespectful and you have no moderation at all.

then they get elected and your like...whaaat?

I don't think these things follow from each other.

Smaller-scale, pre-social-media forums and such in many ways took things more seriously, not less. The poster-to-moderator ratio was WAY lower than somewhere like Facebook or Twitter, and things got handled faster. Internet content was generally not taken too seriously, but internet communities were still serious, and they had better tools for enforcement than they do if they live inside of Twitter.

In a world where someone could spin up a bunch of generative AI bots and target them at a forum to screw with people (whether commercial spam or trolling or just subtle disinformation) that seems a lot harder to maintain.

>> The poster-to-moderator ratio was WAY lower than somewhere like Facebook or Twitter, and things got handled faster.

But, honestly and genuinely asking here: isn't that exacly what happens when all you think is about scale, money and growth, at all costs? (I mean, on the big social networks side)

I would definitely say that the big social network sites are bad for the internet and bad for "real world" society as well; I think it's a perfect example of an externality that they ignore in going for money at all costs.

The tools for a community to self-moderate is the key here, along with communities being able to define their own boundaries.

Reddit communities with active moderators are often great places to have respectful conversations. Twitter, on the other extreme, has virtually zero moderation tools for any group. You get to live with whatever rules Twitter sets for the entire platform (and die by the group that's able to hack them)

The biggest question for me right now is "can Reddit [or similar] survive hordes of generative AI spam/troll/abuse bots without changing to a paid membership model?"

I don't know if the current tools are up to the task.

Good point. It doesn't necessarily have to be a paid membership, but it would need some type of reputation formation. Ability to pay money is one way to grant you a "prob not a bot" reputation.

There's many other ways to set reputation thought:

- I've seen private online communities where you need one or two members to "vouch" for you

- StackOverflow has it's own reputation model based on how helpful you've been in the past

- Bitcoin et-all use proof-of-work as a reputation model.

I'm sure there are many more just waiting to be discovered

I think generative AI is a game changer here. The more easily people can be fooled into thinking the bot is alive and sentient, the more easy it is to scale reputation-gathering.

StackOverflow and such are probably fine since they are generally "testable" answers.

Otherwise, we've already seen marketers figure out how to deal with reputation - "influencer marketing" is far higher-reputation, in terms of whether or not people want to interact with the marketing/marketers, than email spam was a decade ago. So imagine a forum about Toyotas. Have a bunch of mostly-chatbot-powered members hang out for a while, talking about how they like Toyotas, each with distinct voices and personalities and hobbies and (AI-generated) photos of their cars in different parts of the country... then over the next year, they slowly all start realizing that man, that new Honda is making them rethink their Toyota fandom!

You could do this today without generative AI - and people do, especially for things with huge marketing budgets like political campaigns! - but it's gonna be a LOT cheaper very soon. So it'll happen a lot more.

Vouching would certainly slow things down, but if someone is managing a fleet of 50 GPT bots, once they convince 2 real people to let the first ones in, they can start bringing their others in on their own.

Hilarious. Donald Trump reacting to Joe Biden putting Chainsaw Man at number 2 is easily one of the funniest things I've heard done with these voice gen softwares in a long while. Obviously, the concept itself is already highly absurd, but the fact that the execution is so relatively good just really adds a lot.

It reminds me of the 2000s a lot, yeah. For example, does anyone else remember all of the George W Bush sentence mixing? That famous "Fuzzy Math" song that spread around as a SWF. Good times. This is similar in some regards, but significantly more unhinged.

This and some of the "presidents play minecraft" videos my kids have shown me is the best internet entertainment i've had in a long while. It does remind me of the early days funny stuff that could be found on the Internet. Super absurd and ridiculous and made just for fun.

Absolutely. This trend has put me on track for a sort of internet retreat. I can’t know if anything I’m reading or hearing is real suddenly. Previously I could be critical of it and consider sources and try to be rational, but at this point it seems reckless to be immersed in this environment.

Even in terms of work, I’m suddenly put off by purely digital things. I want to interface with the real world, hardware, build systems that humans need. I’m very interested in AI, but it’s not a problem I’m solving and I suppose I don’t really want to compete with it or interface with it more than on an opt-in basis.

By "internet retreat" I initially thought, "I'm going to sit in a room and binge YouTube deepfakes." I was mildly horrified!

But yes, I agree. I really hope that this wave pushes people to get back outside and meet more people in the community. It's really hard to say what will happen though!

We just need that in order to lower the current rates of depression/anxiety, imo, or else the situation will only get worse.

Same here! Although some get old quicker than others, I had few moments of laugh.

I like Balenciaga ones: https://youtube.com/playlist?list=PL9CIfP547-JFhCFeLeRlA2MX1...

> While slightly silly, I think in future people might start to take internet way less serious when you can literally make things up

I think that’s a very naive view point. Internet is core part of today’s society, like it or not. It’s almost like saying that since we invented home printer, which make it easy to forge money, people will just take money less seriously.

I don't. I think it will be approached differently.

> Internet is core part of today’s society

Processing is _the_ core part of all society. The internet has enabled compartmentalized access to specific types of information. Eventually humanity will realize that the information in their technology is fundamentally the _same_ as the information they deal with day to day. Instead of being subjected to the emotional magnet of these technologies they will be incorporated _into_ our daily processes to enhance them.

If you don't mind my saying so, I think it's your point of view that's naive.

OP is talking about a large cultural shift, the type that skeptics said was impossible back at the advent of the web. The orthodox opinion was that the web was a fad and we'd all be reading newspapers and using coin-operated phones forever.

The scale of change that he’s describing — where the internet ceases to become a source-of-truth — would be a complete paradigm shift. Whether it's a "core part of today's society" is entirely irrelevant… newspapers were once core too.

The reason newspapers shifted away from being core is because there were other communication media to replace them. There aren't such replacements for the Internet.

I dont think we are talking about replacing "the internet", but replacing fb/twitter/insta with many many more micro communities that might be a bit more gated for entry due the cheapness of spam. These types of communities flourished in the 80s-2000.

Urban Buildings were important before WWII firebombing.

I don't think it would be progress to abandon cities because someone figured out how to destroy them.

People have always made shit up on the internet. Even prior to Photoshop, the first rule of the net is don't trust anything you read.

I thought the first rule was to never read anything and just comment based on the title of the post.

In the beginning there was only body text and it was good.

It's the same rule, just optimized.

boomer parents told this to their millennial children. then the boomers started believing everything they read on facebook despite the millennial warnings. now everybody needs a warning not to believe what they read on the AI.

The suicide cults of the '90s say otherwise and have a lot in common with the ivermectin overdoses following people old enough to know better doing stupid stuff they read about on the internet.

i mean sure if you want to split hairs over that, but "kids, don't believe what you read on the internet" is like a meme for a millennial who grew up with the internet.

On the Internet, nobody knows you're a dog.

Yep I nonsarcastically think we need to go back to what we lost with the BBS. Mastodon is a step in this direction but EVERYONE should be an instance.

I hope that we can reach a point where "trusted internet" becomes more popular . Trusted internet where you are dealing with identities (people) you know. And every hyperscale apps has a toggle to switch from noisy place to a familiar place.

I don't know if anyone remembers this detail of the book Ender's Game, but this is exactly how their "internet" was setup.

Trusted networks where you had to identify yourself, and public networks where you could be anonymous.

When I first read it, it really impacted me and I knew this is the likely future of our internet.

Wasn't that the present of the internet back in 1985? Private networks where you were an authenticated user, and public BBSes or USENET where you could be anonymous?

I doubt that's possible. How do you know for certain it's not an AI on the other end impersonating a human? Clever people will find ways to game it.

Ironically AI spam might actually make “blockchain” or at least widespread of digital signatures useful ofc it would also mean an end to a mostly anonymous internet.

Without Internet Security, signatures mean nothing.

You may also enjoy the AI conversation between Werner Herzog and Slavoj Žižek. https://infiniteconversation.com/

(not generated in realtime I believe, due to being created in the ancient days of a year ago)

When I was growing up there were 3 rules about the internet

1. Never believe anything you see, hear or read on the internet.

2. Never give out personal information of any kind.

3. Every girl that wanted to meet you on the internet was an FBI agent.

It seems like almost every problem we've had with the internet comes from breaking one of those three rules.

I wish/hope we could/can go back.

> 3. Every girl that wanted to meet you on the internet was an FBI agent.

i can't remember who said it but there was a tagline for a while that went "The Internet, where the men are men, the women are men, and the teen girls are FBI agents".

It went: "The Internet: where the women are men, the men are boys, and the boys are FBI agents."

> I think in future people might start to take internet way less serious when you can literally make things up

I've been starting to think this way as well. It feels like we're seeing something similar to the way that mass spam calls drastically changed the way people use the phones to the point where people don't recognize a phone number, they just don't pick up anymore. As the ratio of garbage goes up, people will just slowly stop using the medium.

My semi-crazy prediction is that AI like GPT are going to be the death of large social media sites. I don't think the internet will die, but I think we're going to return to a more local, trust-less internet like we had in the 90s and early 00s, but for a completely different reason.

This is the path I've been going down as well and frankly, I think that's pretty exciting.

This might be wishful thinking, but I'm coming to believe that AI like ChatGPT are capable of bringing out the absolutely worst qualities of massive social media sites and in that way might end up saving us from ourselves.

> a more local, trust-less internet like we had in the 90s and early 00s

I don't recognize this description of the Internet from any time after 1985 onward.

This might just be a way of aging ourselves. I do feel that Facebook blatantly flipped the norm that you used a pseudonym and kept personal information private when communicating (much less publishing) online - that's the norm I grew up with in the 90s and even the early 00s.

Back on usenet in the 80s and 90s, your ID was essentially your email address (but likely using bang path notation). There no pseudonyms, and everyone was a real person with identifiable features and a real world location.

I was a heavy usenet user in the 1996-2000 era and this wasn't true for me or most of the groups i hung out on (and even less so for IRC chans).

Earlier (pre-eternal september) usenet was definitely different though and I think much more like you say - sadly that was before I was old enough to get on-line, so I never saw it.

Yeah, once AOL came aboard and the green card lawyers showed the future of spam, things took a turn.

And it was only so they could target you with ads or so your friends could find and add you so they could target them with ads

"More local" may not be the perfect description. When I wrote it I was thinking of things like AIM where, for the most part, you knew everyone you were interacting with. The more I think about it, there were plenty of chatrooms and forums at that time though, so yeah, probably not a good word to use.

I think "trust-less" is more what I'm getting at though. In the 90s and early 00s the whole, "don't reveal anything personal, assume anyone you don't have a history with as if they might have malicious intent" was still very prevalent. That only changed when the early versions of social media really started to catch on. My mental mile marker for that switch happening has always been when Facebook no longer required an edu email to sign up (2005), but I'm sure you could argue it happened a few years earlier.

There are plenty of reasons we should go back to that mindset and AI bots making it incredibly hard to verify that you are talking to an actual human maybe the thing that pushes us back toward that mindset.

Having been online since about 1983, I never had this "trust-less" experience. I can understand what you're saying though, because by the time my kids started to be online in the 00s, I was generally trying to convey to them the same point that you're making.

Different strokes for different folks I guess.

I've had many conversations with many people my age (mid 30s) about how we went from "don't post anything about yourself" to "post your entire life online" around when Facebook got big, so I don't think it's something that only I experienced. There's another commenter on this post saying something similar and I remember being taught about "internet safety" in middle school, so maybe it's a generational thing.

I'm from the same cohort as you and can corroborate this. I definitely remember how strange it felt to me when social media became a thing and the taboo of attaching personally identifiable info to your web presence suddenly disappeared.

Yes I remember when a local direct connect hub had a LAN party and many people met IRL for the first time. I continued to meet these people in later years in normal life even after the hub was shutdown. That was semi closed and people were generally trustworthy.

Trust-less information was the norm for most of history.

The past century has really been an anomaly when it came to trusting news, with photos and videos seeming to be reliable proof. Before that, societies had to be structured very differently to account for the lack of proof. You'd hear people from out of town talking about what they saw in a neighboring city.

You'd need to judge how trustworthy the person was.

People would expect a chain of narration, to understand how _that_ person came to learn a bit of information (or if they claimed to witness it first hand).

As AI generated content becomes more popular, I predict we as a society are going to go back to relying more and more on the reputation of the speakers in question.

Who might you consider reputable? People you've met personally, people who your community respects, and of course, influencers who you follow who's persuasive words match your pre-existing world views. Perhaps something else will emerge

But how do you know the tiktok you're watching of your favorite opinion affirmer hasn't been AI generated to impersonate them?

Time to restart the private book club.

The presidents play COD zombies is so funny. The script author is really witty and the dialogue flows very well.

> people might start to take internet way less serious when you can literally make things up

Regrettably this doesn't seem to be the case pre-AI - just look at the scare stories about 15 minute cities being "stealth lockdown zones" where people will be allocated to an area and forbidden to leave. Media literacy is dire and always has been: in the 19th century it was normal for actors playing baddies (Brutus, for instance) to get mobbed on the street, so stage doors were invented. In the 1960s, the BBC did a parody of Panorama and convinced people spaghetti grew on trees. Ghostwatch in the 90s remains a cult programme taken as evidence of the existence of ghosts by some.

The difficulty epistemically speaking is that in order to know something, _anything_ about the world, there must be trustworthy sources and people able to discern who they are. In the darkest timeline, fake news foreshadows the flood of misinformation by AI and renders all knowledge so difficult to trust that we enter a new dark age.

The last dark age was our illiteracy and superstitution, the next will be our own creations bewildering us so utterly that we have no idea what to believe anymore.

People will just perfect the bubble, creating entire realitys disjunct from the scientific verifiable one and dig in there.

People wealthy enough to be insulated from a post truth world are more likely not to care.

It’s gonna be a wild ride with many dangerous possible outcomes.

> I think in future people might start to take internet way less serious when you can literally make things up.

Neal Stephenson goes into this topic with some depth in his 2019 novel "Fall; or, Dodge in Hell". Through his narrative he argues against the notion that any amount of false information on the internet will lead to people not trusting the internet.

Photography at large has been grappling with this same issue since at least the 90s when photography transitioned away from film and digital manipulation of images was suddenly possible. Today, photographs are still used all the time in journalism, in insurance claims, for identity verification, and in other matters were it is important to establish truth. We still largely trust photographs (but to a lesser degree), even if a skilled teenager can completely fabricate an image or video on a Saturday.

Those who forge legal documents need to remember to only use computer fonts (and printers) that were available at the time the document was supposed to have been created: Case OVER: Attorney Spots New Font on "Old" Document https://www.youtube.com/watch?v=KPSMWEMAJLA

>We might be just going back to the time of early, anonymous internet forums time where most people used to it just didn't take it too seriously, and a lot of communities were closed /invite only.

What would prevent that some of those people use AI, even if it's invitation-only forum/whatever? I just don't think the "old internet" can come back anymore. If anything, maybe in-person activities may get increasingly popular.

Yeah it's not that concerning to me. At some point people realized that any photo could be photoshopped and started defaulting to that belief if something seemed off. That is starting to happen with video and audio.

There will always be people saying how crazy that photo of bigfoot is and how that video of aliens is proof but most people will eventually start to realize just about anything you see online might be doctored.

I think it’s a boon for privacy. If fake images/videos/sound can easily be generated indistinguishable from actual footage, if there’s something embarrassing out there you can quite legitimately just claim it’s an AI hit piece by a hater.

People spread rumors today. "it could be a lie" is weak in practice. And if not, well now all crime is legal because nothing can be proven now that evidence can be fabricated.

Then what do you suggest the solution should be if fake evidence is so good it can’t be distinguished from the real thing?

> While slightly silly, I think in future people might start to take internet way less serious when you can literally make things up.

LOL. Uhuh.

Go read up on the raft of cognitive biases that humans are subject to.

People won't take the internet less seriously, they'll just keep doing what they do now: only believe the stuff that confirms what they already think. The only difference is the tsunami of fake information will make it impossible for even well-meaning people to be able to tell the difference between truth and lies.

Back in 2016 folks coined the idea that we were transitioning to a post-truth era:

https://en.wikipedia.org/wiki/Post-truth

We're now transitioning to a post-fact era.

Buckle up. It's gonna be interesting...

I kind of expect things to go the way Peter Watts prophesied in the Behemoth trilogy - the Internet becomes infested with self-replicating spambot-descendants to the point that an open, global network becomes untenable.

No-one’s coupled an LLM with a worm yet AFAIK but I see no reason to think it will take very long.

I agree that without massive safeguards, it is only going to become harder to discern between fact and fiction. The problem is that people must be able to discern between fact and fiction in order to live healthy lives. Reacting to AI by becoming laissez-faire about your information intake is just one more way for people to disconnect from reality.

I think a good analogy is how technological advances made forging physical currency trivially easy. Governments that deployed anti-forgery measures eventually weathered the storm. Governments that didn't ended up with catastrophically weak currencies. Information and data are the ultimate currency on the Internet.

>without massive safeguards

How do you even set safeguards for this?

I personally haven't thought about or looked into it enough to say, though the sibling comment seems like a good starting point.

I think eventually we should get to the point where it should be considered prima facie fraud to use an unidentified AI in commerce. That is, you can use AI generated text, voice, or video but it has to clearly identify itself as such.

At the very least, responsible businesses can agree by common convention to abide by this code of conduct. There could be something like a "good AI seal of approval" issued to businesses that voluntarily adhere to the standard, like those web certificates that were ubiquitous in the early Aughts. Not only should the communications be public-key signed so the browser can identify them, but the the synthetic authors/speakers etc. should be named in a consistent way that unequivocally identifies them as AI.

For example, appending a numeral to the name. Instead of calling itself Joe Rogan, the AI would be "Joe 7 Rogan." Or instead of an article written by Jennifer 8. Lee, it would be written by, hmm, okay there might be a little problem in her[0] case. One thing I've been mulling around is the idea of repurposing the Apple][ symbol. So Joe][ and Jennifer][, where ][ is pronounced "two." The convenience being that it's comprised of glyphs that are on every keyboard, and they also look mechanical and nothing like normal names.

[0] https://en.wikipedia.org/wiki/Jennifer_8._Lee

And on the topic of AI Joe Rogan impersonations: https://www.youtube.com/watch?v=etaA94eM2AE

Yikes. That David Attenborough narrating Warhammer 40K was pretty damn convincing. If I wasn't aware of the context - I would probably thought it was indeed him narrating.

Don't forget the beach that makes you old and getting ratatouilled:

https://youtu.be/etaA94eM2AE

https://youtu.be/we1I6nK6OBg

> While slightly silly, I think in future people might start to take internet way less serious when you can literally make things up.

The would be great. Finally a return to the old ways.

"I have read about it on the internet" was a meme for a reason...

After listening to this, I feel annoyed at the folks in the "AI clones teen girl’s voice in $1M kidnapping scam: ‘I’ve got your daughter’" thread that was on the front page yesterday saying "ppfftt bullshit you'd need to be an idiot to fall for an AI voice" - Guess I'm an idiot. Like it or not, generative media is getting better and better by the day.

Yeah, the voice-gen software has been pretty great lately. The annoyance at folks who wanna ignore that is warranted. Their responses come off very flippant and contrarian.

at this point this is now a great metric to measure originality by - if people can’t distinguish the real you from generative AI you it just means you’ve fallen into a Rut or never had much nee to say anyway.

How does that make any sense when AI can be made to say literally anything?

This is also how swatting works: https://www.vice.com/en/article/k7z8be/torswats-computer-gen...

I don't understand how the phone systems allow for anonymous phone calls. Is it because of virtual telephone services? It feels like they should be required by law to follow KYC regulations like banks do.

And there should be a 5c tax on every phone call made

Now you may want to adjust your estimations on how fast this is going, and how disruptive (and bad) it's going to be further. Exponential change is not something we're good at conceptualizing.

I really don't want to go through the singularity. I used to dream of an AI utopia, now I really wish I were born a hundred years ago instead and only had the great depression and war to look forward to.

Here's the "podcast": https://www.youtube.com/watch?v=meu0CoYv3z8, incredible.

I prefer this one, personally. https://www.youtube.com/watch?v=nhekF0ZF1Pk

Love that one, this bionicle discussion one is top tier as well: https://youtu.be/_eybyDddRk8

That's amazing. The writing reminds me of Aqua Teen Hunger Force

That is really well done. The only thing that gives it away (to my ears at least) is it's missing the trademark stutter/repeating words.

(Not hating on Rogan, just listen to any real clip and you'll see what I mean, e.g. https://www.youtube.com/watch?v=VvswhogSiY0)

EDIT: I listened longer and the AI actually is throwing in some "uhh"s and pauses. They don't sound completely naturally but it's still really impressive

Here is another one: https://www.youtube.com/watch?v=xbxE_qODnKA

I would like a print of Altman's t-shirt.

It mimics everything incredibly well, except for curiosity ("?") and soul ("!"). Very much like print, soulless speech patterns persist through the entire interview, as if they were reading a screenplay. The hesitations and pauses are still there, but not the soul. So either AI translates English punctuation poorly, or they weren't picked-up enough times to mimic themselves into this interview. Example: "Abolutely, Joe!" is very flat and non-emotional. It is uttered as a louder sentence. https://www.youtube.com/watch?v=meu0CoYv3z8&t=460s

it was actually pretty good at adding some emotion here and there

i wonder if they have some extra input/coding to mark for inflections, etc

maybe they just tried a bunch of times and picked the best fit for the context

Train by day!

There is a need for a solution to this problem -- something like public / private key encryption or "more advanced social security" number to verify authenticity of a source via digital signature.

There is a business to be made there.

You mean like China's social credit system? Sounds dystopian

Ideally not centralized, and only for signing creative digital content made for distribution.

Not for signing (and potential blocking/tracking) financial transactions.

Power has a tendency to coagulate.

There are few to no paths to a non-dystopian outcome currently.

> There are few to no paths to a non-dystopian outcome currently.

Said Caveman 1 to Caveman 2 shortly after the discovery of fire. ;-)

Not exactly sure what you're getting at, but the idea that we can some kind of privacy when the capability to develop near omniscient technology exists seems like a great path for dystopia.

This exists, and it's a joint effort across a lot of tech companies: https://en.wikipedia.org/wiki/C2PA

As long as it is not owned by a centralized authority (government/corporation/etc) and is open source, I would be game. But this does not look like that.

I'd vouch for this.

Much like you can sign PDFs with your private key certificate[0], you should be able to sign any kind of file, including audio.

Even if my voice was AI-generated, I could endorse it by signing it.

All we need are CA's jumping into this bandwagon.

[0]: https://www.digicert.com/kb/document-signing/how-to-sign-a-p...

AI researchers and companies need to starting asking themselves: just because we can does it mean we should?

The societal implications and risks of this technology are enormous. This could lead to an erosion of publc trust in audio and video evidence. Imagine someone creating something that gives dangerous medical advice using Fauci's likeness? Or giving the worst politicians an excuse to say that something they actually said/did was just AI generated.

In the life sciences there are limits to what researchers are allowed to do. Especially with genomics.

There are great opportunities for AI to do good things for humanity. We should not waste the time and effort on trivial things like this that have negative net benefits to us.

Edit: spelling

While I agree with you, this is wishful thinking. We need regulations before things get out of hand.

Yep. And that time is right now. Things are already getting out of hand, very quickly. Can we stop the ride? I'd like to get off.

> In the life sciences there are limits to what researchers are allowed to do.

And yet we're sitting here questioning whether covid was man made and there's serious speculation human genome experimentation is active in some parts of the world...

> AI researchers and companies need to starting asking themselves: just because we can does it mean we should?

This just results in the researchers/countries who take the moral high road being left behind thus giving those who continue to do research a competitive advantage.

I think the USA is far and away the most reckless here, and has the most funding. China is not having letting their AI labs release all the new models publicly, and in fact, our research is helping them develop it faster, since US companies are open sourcing everything. (Hey, let's develop tech more dangerous than nukes and share it!)

There is no arms race, we're all in a car driving straight toward a cliff, with everyone fighting to press the gas pedal harder. We need to stop, hard.

It's kinda way too late. The ramifications for society are going to be massive and probably pretty bad overall, not even counting the existential risk which is what I think is really going to do us in. I think there may be a path to not everyone dying, but it depends on the right people in government getting the message and taking drastic action a-la Yudkowsky's plan https://www.lesswrong.com/posts/oM9pEezyCb4dCsuKq/pausing-ai...

I think you’d like a blog post I wrote in November where I put forward an outline of what such a system would look like:

https://lukas.dev/posts/how-to-trust-again/

Digital signatures, media verification, authenticity and more are all covered!

Why wouldn't the solution be just check the canonical source for the content?

IE go to joe rogan's spotify page and see if the content is linked there.

I don't think the top concern is verifying the authenticity of Joe Rogan podcasts. This is just an example.

because mis/disinformation is the real problem here. We will be seeing a lot in the media and when 90% of consumers don't bother to check the source, it _will_ skew a lot of things.

bitcoin fixes this.

It sounded weird at times, almost like someone reading a script with a higher pitch than his normal voice. But the technology is amazing and I would most likely be fooled by a short clip if the content wasn't completely out of place from him.

The ironic thing about this is that it reminds me of how the podcast used to be a few years before Spotify, before it got bogged down in politics and Joe asked more interesting questions and it was just generally funner. I just listened to this for 20 odd minutes which is probably longer than I've listened to the actual podcast in the last month or two.

ChatGPT always answers questions like: Prompt: Can you tell me about <description of thing> ChatGPT: Sure, let me tell you <slightly rephrased description of thing>

It's pretty funny to hear that in Sam Altmans voice, along with umms

There's a growing number of companies working on voice - we used one recently on a game, it's not quite ready for main characters (yet!) but for background characters and rapid prototyping on main characters (that we plan on rerecording for the final assets) it's already there. It's so close, but none of them quite capture inflection, it's the stable diffusion fingers of audio AI

I think what gives it away is when answering questions, chat GPT first repeats the question. For example the question is: "If you were to be found in a small blue room with your favorite food, what would you do?".

The answer would start with: "if I were found in a small blue room with my favorite food I would..."

Normal people don't usually talk like that.

Television presenters and interview subjects are coached to answer questions like that, repeating the question before answering it- I wonder if that's where ChatGPT picked up that particular habit.

Comedians too, though for a different reason -- gives them time to think of a funny response.

It seems sensible to me, as a defense against someone taking your answer out of context and splicing it with a different question to make you look bad.

It also helps editors later be more flexible with the format... if someone wants to reuse the insight you taped in a new medium (say, they want to put it in a documentary) they might want to have you saying the whole thing rather than to provide the context of "there is also an interviewer".

The Song Exploder podcast does this very well; each episode is recorded as an interview between the host and the artist, but because the artist is answering the questions with a restatement of the question, they can edit it so that it just sounds like the artist talking through their project (you almost never hear the host ask a question).

Pageant and spelling bee contestants aside..

Rogan is always the first one to be used as input/demo for this stuff.

I have a feeling it's going to remain the trend for every new AI tool.

There's just so much data available, I don't see how it couldn't be the prevailing trend.

meme - not trend

This isn't a new issue. Image and likeness laws exist for a reason.

A more clear example on how this could be harmful, and how we are already equipped to deal with this - I saw an ad for a supplement that had a convincing AI Joe Rogan "talking about it on his podcast" and how it's the greatest thing that everyone needs to buy. This is illegal currently, and it's not any different from hiring a Joe Rogan impersonator to talk in a similar looking podcast set to trick people. It's why we have systems to enforce ownership over trademarks, copyrights, and your own image.

I bet the lawyers are so happy about generative AI they can hardly count.

I think we're going to need some sort of 'real human' proof system where when you record any audio or video you publish the media hash, n-second segment hashes, and signed participants to a ledger/blockchain. You could also build a tamper-proof device that you place in frame that uses a combination of a hard to get-at private hardware key and the local ambiance to produce a signature you encode a as a subtle signal that can be later be used to authenticate the video.

This sounds like chaos

Also other signals like gps, ambient radio waves, gyroscopic information, etc. whoever simulates reality better wins.

Maybe that is how the Matrix started...

Oh no, your hardware key was destroyed in a fire, you don't exist any more.

There will be those who exist within the system, and those who live in the shadows of the system. And occasionally, when the system fails, people will fall into the shadows. Depending on how incorruptible the system thinks it is, it will be impossible to climb into the light once more.

The old boys club which has been outsourcing programmer for 20 years to India figured out that they can just fake it since nobody has complained yet.

And before software engineers unionize they came up with some snake oil to put us in our place, maybe try to force us back into offices, or just eliminate us.

Nobody can deny that Sam Altman and Bill Gates have been trying to "reduce costs" for a long time. The startups with devs in portugal, costa rica, Mexico, Spain, the Ukraine, India, China, anyplace where they can pay 5 dollars for a day's work.

When Bill gates said he would pay programmers 7 dollars an hour, we were all offended, we didn't realize that he was already doing it and that would be a significant raise in pay.

The AI Joe Rogan might deny it. Also waffles are space aliens and the moon is flat.

> The startups with devs in portugal, costa rica, Mexico, Spain, the Ukraine, India, China, anyplace where they can pay 5 dollars for a day's work.

I can speak to Portugal and Spain at least, and there's no chance you could ever get people in either of those to work for you at $5/day. You'd be laughed out of the room/conference call. And I'm pretty sure that's true for the rest of the list too.

Not now because they leveled up

Do you think he can place those countries on a map?

This is nuts, it even gets the plosive sounds on the mic right.

Fantastic execution. The generating dialogue I understand; how did they get the voices so incredibly lifelike?! Where do I start when I want text to speech like this?!

I never would have guessed computers writing code would come before generating convincing voices.

Genetic programming has been around for decades. GPT/LLM based coding is not the only type.

Moravec's Paradox at it again! The thing that I find really interesting is that a few years ago, generating a realistic facsimile of someone's voice was "something a computer could never do" but now I find LLMs' ability to generate believable dialogue more impressive than the (very good) text-to-speech conversion.

https://beta.elevenlabs.io/voice-lab

I don't know

There are other tts projects that run faster but if you want quality:

https://github.com/neonbjb/tortoise-tts

Follow the instructions for adding a new voice. Once your voices are added, have ChatGPT generate a conversation between the voices. It isn’t difficult at that point to generate a video displaying the current speaker synced with the audio.

https://www.elevenlabs.io/

I think we now have the technology to build the talking portraits we saw in Harry Potter. Voice, facial movements, dialogs that matches the character can all be generated by AI now. I'm just not sure if this can be done realtime yet to interact with another person.

In the future, you may be able to have a conversation with a portrait of your parents even after they pass away. (So collect as much training data now?)

That would be a pretty awesome installation at Universal Studios.

But I guess making it not say offensive things might be a problem… or not, depending on the character.

the current workflow I've seen use is midjourney for the portrait creation, elevenlabs to do the voice and D-ID to animate the picture to the voice. Apparently the entire process can take under 15mins.

As weird as it sounds I had this exact thought.

I signed my parents up for Storyworth, which emails them once a week writing prompts about their life.

I'm still weighing the ethical implications of using the entire dataset 20 years from now to generate a facsimile.

Partly I just feel sad about the future, forever chasing digital ghosts of our past loved ones...

I listened and I think it’s really good. I wonder how much effort went in to polishing the sound.

Or did they just feed a script into some voice generator?

The dialogue is excellent but the intonation doesn't sound quite right. The voice is a bit high as if he's tense or something.

The one thing that I want from all this AI voice stuff is better selection of voice for the siris and alexas out there. i.e. Ability to clone some's voice onto an assistant

Some voices are just more pleasant than others & it differs by person doing the listening

We know you want the Trump voice.

Also not just voice synthesis, but the word output should also be modeled after the character. E.g., [alarm 7:30] -> "It's 7:30, wake up you stupid son of a bitch!"

Can't wait for my assistant to tell me he's a great assistant, possibly the best in the world.

You're telling me this was fake all along? https://twitter.com/TallBart/status/1643108942627864577

I think we'll see media personalities move to a public key encryption model, where authentic streams are encrypted in some manner with a private key to verify their origin.

I don’t doubt it, but I do doubt that the majority of the public will make an effort to verify signatures before consuming media. I think the public will rely on trusted channels - like the personality’s .com or official YouTube channel.

Yeah, I think it would have to be handled in the player. Similar to https, it would warn you if the stream isn't verifiable with a known key. You'll still be able to consumer unencrypted streams, or streams whose key isn't from a recognized authority. but you'll get a warning.

Personally I prefer AI Joe Rogan discussing Bionicle with AI Jordan Peterson.

https://www.youtube.com/watch?v=kVX1PB19TYE

I enjoyed the Factorio one.

This is hilarious because Peterson explaining Bionicle lore is not that far removed from how he rambles on about mythology

zerohedge exists in the same basket of deplorable conspiracy wingnuts. Big grain of salt and they're not a reliable source.

Biggest appeal of his podcast is that he's a brilliant standup comedian, and he uses that in his podcasts. That's completely missing from this copy.

Sir, you have made my day. This is the funniest thing related to Joe Rogan.

Thank you.

Based on my experience with Substack posts with ChatGPT's rewriting layoff notices in various voices, including Trump's:

The novelty of this idea has worn off. The first post has almost 20x the views of the second. I don't think Rogan has anything to worry about.

So he only likes misinformation if its actually the real him.

Reply on news.ycombinator.com