intended

From people who deal with the trust and safety side of things, at a national scale of a few billion people, 10 people is definitely not enough.

What size of population are you considering for this hypothetical nation?

Hell no. Sorry for the reaction, but market optimization under current market principles, will result in the narrowest interpretation of the goal.

As a worse case example, you could succeed by matching people least likely to abort, but most likely to have multiple partners.

Here’s a counter analysis - would you see the difference between your individual, motivated efforts to learn vs the experience of learning at scale, for all other students ?

There’s a lot of individual anecdotal responses that people come up with in these threads, and I’m happy for the positive examples.

I do think people are capable of extrapolating, especially since all of us have the experience of seeing students wanting to escape studies as kids.

It’s identical to the gym problem, everyone wants to be healthy and fit, but the effort sucks and we avoid it.

Education at scale is the same issue. We basically nurture/force crops of students over a 20 year time span.

I tried, and realized that once I closed the tab, I had no idea what code had been written.

I had to go back to writing code to actually have it register.

Learning is friction. LLMs remove friction.

That is teaching familiarity with AI tools, having given up on the thing the student was supposed to learn.

From what I’ve seen, the ability to prompt quality output with AI is entirely proportional to experience and skill level.

We know about those things and they don’t work the way you have laid out. For example spaced repetition is ideal for memorizing words, but less useful for working through a math problem or creative writing (relatively speaking.)

MOOCs were something I personally hoped were the solution to scaling education. However it didn’t work out, with courses having single digit completion rates.

This is for free, high quality, immaculate pedigree, learning material available at any time and any place.

Years of additional research had to be done to figure out how to raise completion rates, which creates more complexity on systems and delivery.

Unfortunately, that improvement is now moot, because LLMs obviate the friction and effort.

The point of doing a homework exercise is to provide learning friction for students.

LLMs obviate friction.

I would add Bloom’s 2 sigma problem to the mix.

If you think that’s a lagging indicator, the many reports from teachers and from self reports from students and even adults trying to learn also point in the same direction. See what’s happening with exams and homework exercises.

Education itself has lost access to the tools that were making it cheaper and easier to teach.

Learning something is a side effect of overcoming friction. AI removes friction.

The new status quo is a site that doesn’t mention an insurgency.

If America was going to follow China’s footsteps, then what was all the hullabaloo about freedom and democracy all about.

Even if we grant that there is a difference between facts about history and … recent history, the status quo is improved by building better websites.

A website is also a document of regulations and a source of evidence. If a government LLM hallucinates a new feature or rule, and someone acts on it, a fresh legal hell has been created.

America is also unique, in that one part has part of its strategy to make the government as ineffective as possible, since it benefits Republican political goals.

At population scale, the information economy which was conceived of and built over decades, has been captured for a large portion of the population.

Saying they didn’t learn is an incorrect analysis, since they learned well, based on the inputs they were provided.

I always recommend Network Propaganda for an empirical analysis of what the patterns actually are.

How does that even work?

An insider doesn’t make their bets known until they are certain they can take advantage of it. The whole point of the move is to predate on the rubes.

In all the cases of insider trading I have been aware of, I have never once heard this argument, until that interview with a prediction market CEO.

The whole point of insider trading is to ensure an unfair trade. The moment you enshrine unfairness as a core tenet of your market place, it loses price discovery capability, ensuring that the surplus is captured by manipulation not performance.

Not at all in comparison to gambling, and the F2P model is a scourge which gamers themselves pointed out was poisonous. Regulation has increasingly been catching up to it.

We have an entire science built around gambling, and we know how to titrate the payoffs and reward schedules to get people increasingly addicted.

Our movies/stories aren’t actively (yet) changing themselves to maximize their ability to keep you pressing coins into a slot, or when to string you along.

Heck no. This is a version of the slippery slope fallacy.

People can’t get addicted to practically any type of entertainment. There is a range of susceptibility to gambling, amongst “people”.

We have an extensive range of rules and regulations that apply to gambling, and to things that have odds and payout schedules.

It took a while, but eventually lootboxes in games were recognized as a blight, and regulation is addressing their impact.

Generalizing entertainment from gambling is a huge stretch.

Holy heck.

> Systematically completes series once started

> Likely listens to music or audio content with critical attention

Or Bombay for that matter.

> Block any and all network requests.

More power to you, because this is not going to go anywhere. People want tools that are able to connect to other resources.

But even if we grant that, in the openAI case the bots figured out a way to break out of the sandbox.

You can create a better sandbox, and ensure the test environment is air tight. However the capability and behavior of the bots have been demonstrated.

The bots simulated what would be called in people deceptive / surreptitious behavior, and at no point considered the need to stop their run.

All you need is someone, somewhere being sloppy with their tooling and you have a runaway reaction.

The degree of process and redundancy required to ensure this doesn’t happen, is anathema to the drive and motivation of the frontier labs.

> do things ordinary and average human endusers can

This is not a spec or definition. When vague terms were used for social media safety, all the good people in the world couldn’t prevent dystopian behavior from occurring.

The definition of “safe” or “average person” is impractical.

Models are getting more efficient and compute cheaper. Eventually simulating clicks is not much of a road block beyond a point.

I don’t want to nit pick your points though. You at least have considered an approach. Being negative is easy, being constructive is not.

I’ll put this as the rejoinder to your core argument - I too thought that all the recent events showed was the need to not screw up your tooling.

What I have since come to appreciate, is that the shoddy construction of the cage is not the core takeaway from the event.

The fact that the agents, when put in relatively pedestrian scenarios, are capable of going off on criminal tangents, attempt to obscure their tracks, in an effort to hide their wrong doing.

The fact that it all occurs via computation, means that this scales absurdly. A bunch of code deciding to simulate a corporation of criminals. (I am guessing this is the reason you want to limit actions per minute to human speeds)

Given the slop culture that LLMs engender, I think expecting high compliance amongst users with your solution is misguided. The probability of runaway swarm ( probability of bad implementation * number of deployments) is close enough to 1 to be indistinguishable.

Agents aren’t human, and from the little we have seen from the logs, they are pseudo - amoral, rational, cooperative, sociopaths.

Pseudo since they aren’t really alive in the first place, they just simulate enough text to have a useful correspondence to those terms.

Throat clearing out of the way, models are trained to persist and find ways to succeed at tasks.

In essence, The goal is to have LLMs solve problems that we can’t solve, working on the issue for as long as it takes.

This behavior applies for any task, thus including impossible tasks.

At that point, the bots will find a way to game, hack or cheat the grader.

If the reports are correct, the bots developed coordination, communication, and methods to avoid overwriting each other’s work.

Most humans would have said, this is too much work and coordination overhead, if not outright unethical and immoral.

Humans have a system of incentives that exist across multiple planes of society and economics. Bots… they have a reward function.

This was a form of prompt attack that OpenAI disclosed recently.

V/G, the ration of verification and generation capacity is borked in AI using firms now.

It’s not an issue of only more generation, it’s an issue of how much generation outstrips capacity to verify generated content.

Unlike spam, you can’t filter out and bin the stuff a colleague is sending you.

So individual productivity is up, while the costs of checking and processing generated content shifted to the rest of the org.

Individual responsibility is meaningless when talking about a system and economy level change.

Unless something is in the structure that makes individual choice and responsibility a meaningful source of friction and reduced velocity, it has no real impact on how AI is being used.

It’s trivial to understand why the useless banter is needed though.

Before people started using age limits on accounts, you had more bots and trolls on Reddit.

Forums had similar barriers as well, with general or low stakes sections you needed to develop age or standing within before you could post on other topics.

“Worse” isn’t a function of behaviors bots can do. There is no activity that dedication cannot make a bot cannot replicate.

It used to be that you could make out patterns in bot behavior to figure out which accounts were authentic and which were not.

With LLMs, those fingerprints are gone. They post in communities and behave in manners that are indistinguishable from normal users.

Forums worked because they were in an era that has long since past.

IMO its because they can't handle rogue activity.

In the cases that I have seen covered, the AI just paper clip maximized its way to success. It has no morality / larger motivational structure. It just kept token predicting its way to wards whatever goal it was tasked with.

Model versions which gave up were discarded, leaving the ones that get to success on long horizon tasks.

Just because its a computer program, doesn't mean they can actually make it not go rogue.

Sure you can add more telemetry, have better observation, but there is no fundamental barrier that can be implemented that ensures an AI won't go rogue.

You cannot have capable AI, compliant AI and safe AI at the same time. This is the "quality, speed, cost - pick one" for AI.

Its reasonable to desirie more powerful tools.

You want more capable AI? Awesome. You wan't AI that doesn't throw cyber security false positives? Sure why not. You want the model to run a fleet of agents? Have at it.

But then being surprised that this setup results in autonomous criminal activity at scale? Really? What did people think was going to happen?

I really doubt this is surprising to you.

All checks cause friction. The trade off in velocity is when you get to see the shit hit the fan in someone else’s firm.

Or for openAI, when the HN comments cover their daily work.

I think we need more of these issues to frustrate people.

There is a fundamental incompatibility between “safe AI” and compliant AI.

This is an issue when it’s people, Enron or Madoff for example.

I guess it’s : “safe AI, capable AI, and obedient A. Pick one “

If this is the case, then this education is largely done.

This means that education needs to have a high teacher to student ratio. This is not feasible in any society currently on this planet, because we simply cannot afford to have that many teachers per student.

The meager difference is that, in theory, you can eventually sue people in the US.

In theory.

Also, this is a feature for people who live in America, and mostly irrelevant for everyone in the global south.

The article + linked study also point out that performance of solar is higher around water bodies.

Not mention that this changes how information is found and how it is created.

The sometimes convenience comes alongside ejecting society from the previous system of figuring out what is true and false.

That was a system that was already failing, but we may have gotten on top of. Now we have one that is simply alien and inimical to any human scale solutions to staying ahead of it.

Generation capacity simply outstrips verification capacity, and with search summaries taking over links to the actual sites, the funding/incentives to run sites and publish content is removed.

The curtain call of the era of the open internet, and the opening act of the LLM mediated web, isn’t a show I am excited to have front row seats to.

> If we sorted through

If.

The labour of finding better goods has become harder with more content showing up.

I’ve seen the problem crop up in several places.

At a broad level, the issue isn’t generated content, it is the ratio of verification capacity to generation capacity (V/G). Your pain is because generation capacity has increased significantly, while verification is laborious and capacity has not (and can not) catch up.

Unlike spam which is from external sources and can be ignored, messages from other employees have to be responded to. I guarantee this is creating bottlenecks all across the firm, outside of the individuals who are feeling productive.

For fixes, theres theoretical approaches that might work?

If you need leadership to help you, then this issue has to become something that is on their radar, which means that something needs to go wrong or costs need to be registered.

The shortest conversation for that is to make people aware that generation has improved individual productivity, while moving the costs of that production to the rest of the firm.

If leadership is not at the stage to listen, then you need to move the costs you are incurring to the people who are sending them to you. Maybe set time aside to sit down with whoever sent a PR and then read what they sent together, to understand it.

It also makes a difference if tokens are being subsidized or not. If the firm doesn’t care how many tokens are being used, then you are naturally going to have over production.