intended

Not mention that this changes how information is found and how it is created.

The sometimes convenience comes alongside ejecting society from the previous system of figuring out what is true and false.

That was a system that was already failing, but we may have gotten on top of. Now we have one that is simply alien and inimical to any human scale solutions to staying ahead of it.

Generation capacity simply outstrips verification capacity, and with search summaries taking over links to the actual sites, the funding/incentives to run sites and publish content is removed.

The curtain call of the era of the open internet, and the opening act of the LLM mediated web, isn’t a show I am excited to have front row seats to.

> If we sorted through

If.

The labour of finding better goods has become harder with more content showing up.

I’ve seen the problem crop up in several places.

At a broad level, the issue isn’t generated content, it is the ratio of verification capacity to generation capacity (V/G). Your pain is because generation capacity has increased significantly, while verification is laborious and capacity has not (and can not) catch up.

Unlike spam which is from external sources and can be ignored, messages from other employees have to be responded to. I guarantee this is creating bottlenecks all across the firm, outside of the individuals who are feeling productive.

For fixes, theres theoretical approaches that might work?

If you need leadership to help you, then this issue has to become something that is on their radar, which means that something needs to go wrong or costs need to be registered.

The shortest conversation for that is to make people aware that generation has improved individual productivity, while moving the costs of that production to the rest of the firm.

If leadership is not at the stage to listen, then you need to move the costs you are incurring to the people who are sending them to you. Maybe set time aside to sit down with whoever sent a PR and then read what they sent together, to understand it.

It also makes a difference if tokens are being subsidized or not. If the firm doesn’t care how many tokens are being used, then you are naturally going to have over production.

A poll of estimates would be quite helpful in figuring out what the heck is actually going on.

How many are you seeing / estimating?

Drive by code dumps from people who have "democratized Natural Language code" which pushes all the work of verifying whether it works on the few active maintainers is hell.

Any collective group with open contribution will end up hating it. Why is this a surprise?

I did not expect to see a Leamington Spa beer fest poster being used as a reference image today.

What results though. The people seeing measurable improvements to their core work with LLMs are coders.

Everyone else is taking over intern level work from someone else’s team. They are reducing the friction costs of talking to someone else, for about a 30% productivity gain.

Firms are trying desperately to automate their white collar workers, and that is following the same trend as all other automation projects, and ML/deep learning efforts in history.

Process vs outcomes.

If your work has little liability then you can afford to not care beyond “does it work”.

If you have to worry about quality and ensuring you don’t get sued, you make sure the process works.

If it has to maintainable, you you need the mental model to be present in someone’s head.

How will you redistribute to people in Japan, for example?

The profits and income is earned in America. The idea that America would pay manga artists whose work was copied is … beyond idealistic.

There’s too many implicit assumptions in that sentence that run afoul of the conversation.

Just spending money doesn’t mean it’s legal, for example. Criminals expect RoI too.

If the US companies had to negotiate with the 7.3 million rights holders, we wouldn’t be here, and we would have probably got a system where the models were never released in a manner that could be easily distilled.

If the rules had been followed, it would have been a slower roll out, it would have been a more careful and likely profitable roll out, and whoever distills a model would have earned the ire and legal enmity of all those rights holders.

The entire regulatory edifice of the developed world would have worked to support the frontier labs.

Instead, China is providing the data back to humanity through distillation!

Again, incentives.

The people who publish are people who have reason to publish when they can be copied. Typically either they have already been paid, or they expect to gain market share by being free.

People who need renumeration to continue working will not publish.

Intellectual property rights, as much as I dislike the RIAA and MPAA, created a way for more players to enter the market, because it created a way for their needs to be met.

Absolutely.

This is definitely a scenario in which management has bought into the hype and the people working with the tool are aware of its limitations.

Like markets, the firm can remain irrational longer than you can push against it.

Beyond a certain degree of effort on your part, it is not within your capacity to right the whole ship.

I suspect that the people who are most likely to be allies are the CFO and budgeting teams in the firm. I’d be curious what that function is thinking of the situation.

And Businesses will buy, because the AI promise is a promise to automate workers away.

Yes, and we lived in a world of guilds, Orders and secrets which kept their knowledge tightly within themselves.

Music and art was created by patronage. Entire periods o where the art that mattered was made by the powers that moved the world.

For my whole life the goal was to move away from that era, not to see it recreated.

Also, this is a debate that English speakers can enjoy on a website for an US based accelerator. Most of humanity doesn’t even have the standing to be heard in this conversation.

> Floridi has a counterintuitive solution: Release more of them. “Simply put, if you cannot avoid it, then make it pluralistic and diversified,” he writes in his 2024 paper. “It would be a messy, cacophonic, and noisy world, but it could also be less manipulative.”

Hell no. More choice is not an infinite money glitch.

Putting more and more and more options to users is how you overwhelm systems, till people simply perform the default, least challenging action as a reflex.

That is the current state of the information ecosystem, it is controlled by overwhelming consumers, not by controlling content.

Numerals and numbers are directly based on common sense intuition. Even zero, which can be considered a counterintuitive number, was invented several times.

Immense amounts of science are built on incremental advances that apply previous insights to the domain.

The greater deterrent is knowing you will be caught.

If something is risky, and the end operator cannot be considered to have orchestrated the outcomes of too use, then that tool typically has significant restrictions placed on it.

Liability will shift to the maker of the tool if they claim that it’s easy to use, safe, or that you don’t need unique skills or training to use it.

That would be considered reckless.

Cars analogy - We have licenses for cars, and different types for different vehicle classes.

Cars have to be rigorously tested to meet standards to be considered road safe.

I recall China having instituted limits on gaming for children. I believe there are limits for social media as well.

Its relevant because those same issues occur with large models. Also, the "crappy small model", was a model trained for safety tasks, and outperformed the frontier lab safety models.

You could have, at least back when Trust and Safety roles weren't getting crushed by the AI stick.

Theres tons of research which can be brought to bear, genuine problems and challenges that require deep, thoughtful work, and some of the greatest people you will work with.

It is also absolute madness. There are so many insane, broken things about how safety is provided in the world, that you will feel like you are in a Elizabethan slaughter house, just at species scale. This is all excluding the world of child safety. Adding that to the mix gets you to existential nihilism quite rapidly.

Drinking is down for young adults and teens. My pet theory is that Covid broke the chain of party and drinking culture being passed down. (This is entirely speculative, and based on a single chart I saw.)

The relevance to your point being that removal of parental restrictions don't seem to be doing that anymore.

Education is definitely a good idea, but do note any plan that depends on education gives up the next 5-10 years before results are seen. Works great for societies and countries that think that far ahead.

America in particular has an extra challenge, since Big Tech is based here, and threats to revenue streams will be met with immense opposition.

If you want to disabuse yourself of your notions annd intuitions on how LLMs work, run safety models and tests.

On a hate speech policy test for a lightweight LLM, the presence or absence of the last full stop on the last sentence would cause the model to flip its decisions.

Why do you find it hard to reconcile?

It seems fair to guess you aren’t building your own harnesses, running multiple experiments with achievable win conditions, while providing oversight over model behavior.

Scale alone suggests your experience won’t match.

In the future, When everyone and their uncle is launching swarms to solve impossible challenges, at that point we can expect proliferation of this scenario all over the place.

These situations are novel. Lax terminology is fine when it has no impact on the intuitions, clarity and conclucions of discussion.

If this was a conversation just about outcomes, then whether models think or simulate thinking is sophistry. However, the bulk of the issue here is attributing responsibility, which relies on being clear about the underlying processes at play.

We are hard wired to assume certain priors and capabilites when it comes to "human like" behavior. Anthropomorphizing LLMs implies mechanisms that aren't present, and end up distorting/complicating discussion about the process.

It isn't helped that the frontier labs, the experts in the room, generally use anthropomorphic terms to discuss model capability.

>but It no longer makes economic sense to write all the code by hand, so increasingly I don’t.

and

> There are good developers, mediocre developers, and bad developers, and the good ones are still hard to find.

move in opposite directions. Friction is the source of learning, which creates good developers.

We have a ways to go before we hit zero, since we are inventing entire new ways to be a prick.

I think we are a year or so away from the first tik toks of humanoid robots walking up to people and punching them for views.

The fact that it is autocomplete, doesn't dismiss or minimize the threat though?

I am not sure how that link was made.

Good old ML, which is significantly simpler than LLMs, was capable of ensuring people would not be hired simply because of their names.

The fact that it is misaligned is also not being contended, if anything that contention is made easier to support.

When models are anthropomorphized intuitions of how humans behave end up driving discussion and ideas off track while being too attractive to avoid. This isn't helped when the terminology from the labs and other sources is "intelligence" "intent" and so on.

This is the perfect fracture point for both anaolgies.

LLMs simulated more than simple autocomplete.

The autocomplete analogy is rebutting a different point: namely the fidelity of the simulation to reality.

This specific argument is valid. As sophisticated a simulation an LLM is, it is not “thinking” in the same sense we assume other people are thinking.

I am not making an argument about free will, or the uniqueness of human thought, just that the correspondence to how humans reach conclusions and how the simulation produces outputs do not match on a 1:1 basis; as a result attributing traits builds incorrect intuitions.