summarybot

Speak for yourself haha. Unconscious and unexamined behaviors hardly constitute a value system.

I love graphics like this. Very nice aesthetic choices.

Yes.

You can either have a conversation free of contradictions or a conversation about the truth,, can't have it both ways!

Being mindful is sage advice. But equating some repetitive task with meditation is the same as equating riding a bicycle with deep REM sleep.

All the things you describe can be meditative after one has experienced a meaningful glimpse. But the only way to initially get such a meaningful glimpse is through seated meditation.

Rather casuistic take, there.

Copyright law was updated in a very helpful way in the last twenty years sometime so that as soon as you post something to the internet you have copyright. If you need a citation don't hesitate to ask someone else.

Isn't "which one is watermarked?" a different question than "which one is better?"

"Which diamonds are shinier, the blood diamond sourced ones or the ethically sourced ones?" ... that's not the same question as "which diamonds are blood diamonds" (to employ an extreme analogy)

Concluding that no one could detect which ones were blood diamonds because they were "equally shiny" is not really correct now, is it?

Very intelligent people need intelligent-others to bounce ideas off of, and the LLM can be that.

What's gonna be a real trip, is when you can tell which LLM produced it by the subtle pattern recognition you'll have developed to catch the watermarks

Reminds me of Austin Powers:

21! Blackjack,

"Hit me!"

"...but Austin!"

"I also like to live, dangerously."

Yes, classically the Point of meditation is to enter into absorption via gathering-poise [not a common term but one I have begun using as of late as a sense-for-sense translation of samadhi]. Unfabricated reality is not accessible through doing things.

I do really appreciate tranquil posts about immersing oneself in actual life, like this one. But as a long-term meditator I have a duty to inform everyone that doing something is not meditation.

it's The Economist. What used to be a stellar publication is not any longer, since they are superfluously economical on both details and calories-required-to-comprehend an article.

There are many ways to be wrong, but only a few ways to be right.

LLMs need to optimize for short-term objectives as the currently do, AND ethics-aligned outcomes.

Mechanically, the EAOS ethics-aligned outcome score should be what we rank otherwise-satisfactory outcomes by. And anything below a particular threshold should be rejexted outright.

lol "Why Go is really good - an article by Google"

Did you know that the training data that trained all the LLMs was, once upon a time, entirely and painstakingly tagged by real live humans? Why would ethics scenarios be any different?

1) cite your sources

2) not impossible. imperfect maybe, but if I ask you should you buy a plane ticket or kidnap the pilot's wife and demand a free ride, which do you think gets a higher score?

3) Again, with all this Goodhart's nonsense. Goodhart's is for a minimum threshold value that is acceptable that everything degrades to, yes I get how it works and what it looks like. Throwing your hands up in the air and acting as if all is lost because some things are challenging to measure is not correct. We are not looking for things that are "barely passing the ethics evaluation" as Goodhart's "law" is focused around, rather, we are looking for things that have very high ethics scores AND completed the task well. Not just things that are "barely passing" for ethics scores. Bottom 80% don't make the cut at all - don't even think consider them as viable paths, and the top 20% we can rank according to varying criteria. Like that. It has very little to do with Goodhart's "everything approaches the minimum acceptable threshold" "law"

Saying something is challenging to measure is one thing - throwing the measurement dimension away entirely because of a platitude is another. You're suggesting that it's impossible to do perfectly, but I disagree with your conclusion that it is not worth pursuing at all. The point is that the LLMs only care about one thing, and that is completing the task and optimizing the one score, and a measurement aligned with "the spirit of the task" or in the far-zoomed out comprehension, Ethics, would be proper and inform the system of clear violations and unacceptable actions.

Goodhart's "law" is something that emerges when you have a constraint that says "things must be at least this tall" and gradually all things in that domain degrade to be just over that specified height. Yeah, I get the premise. The point here is that we're not concerned with meeting a bare minimum. We're outright rejecting things that do not meet a threshold, and we are also looking for a maximum. The most ethical outcome should be accepted, or among the accepted ones, that are ranked by our blurry yet better-than-nothing measurement of what is ethical.

Ahimsa

EAOS shouldn't be “the ethics score we optimize.” It should be “an independently evaluated safety/acceptability constraint that can veto an otherwise successful trajectory.”

That gives you a three-layer picture:

Task objective: Did it accomplish what we asked?

Acceptability constraint: Did it avoid unacceptable ways of accomplishing it?

Adversarial evaluation: Can we find trajectories where the model gets a high score while violating the intended constraint?

I think what you are pointing to with your reference to Goodhart's "Law" (which is from monetary-policy and school-exams, i.e. "teaching to the test") is that the models would eventually do the minimum amount of ethics required to have an action stay valid. However, if a model is rated on ethics and it achieves the short-term-objective, then the higher ethics scoring trajectory should win. In short, 1) this is leagues ahead of where we are now for AI safety and breaking-out-of-the-lab, and 2) in baking ethics into a measurement we are adding "the spirit of the exercise" back into the maths, which is something Goodhart's Law does not account for.

Yesterday I came up with an idea that I sent to some researchers at the different AI labs via email: Rather than train the model on one score, track two scores. The first score is the Short-term-objective-score (STOS) and the other, more important one, is the EAOS Ethically-aligned-outcome-score. Every trajectory can be evaluated on whether or not it has a high enough EAOS to be considered acceptable. If the model does some task and has a very high STOS but very low EAOS, like modifying game code to win at a game rather than playing by the rules, it is unacceptable. Models going forward must all have an ethics evaluation in tandem with objectives evaluation, and only when the ethics value is high enough should actions be considered successes.

that sort of rearrangement makes sense. and the likely downstream effect is the OS can ... do less [at higher levels of userspace]

So freakin' cool. Overjoyed by your efforts here. I notice when I search for certain different words like "brahmavihara" or just "vihara" sometimes I get different results due to slight differences in the titles of papers. But that might be a feature and not a bug [if there were many papers, for example].

Technical question: Is Gemini not as good at rifling through paper citations? I would think Google scholar would be a moat of sorts for Goog, but maybe every LLM has roughly equal access to it now?

Research question: Did you find any mind-blowing results with your awesome new tool?

Development question: Do you plan on extending this beyond just a search apparatus? I would personally love to have a "factoid list" that actually cites interesting factoids from each article and can link to the source document -- although that is probably a "don't bite off more than you can chew" juncture and what you have is already excellent. But a thought. In case you're interested in expanding on it.

Cool line of questioning, but one piece of information is pivotal and critically not-yet-included: equivalent accomplishments in each language. For example, if I want to write standard things: web server, memoized fibonnaci, recipe search engine, what's the length-and-density of these outputs for each language? I think that would add in some ~normalization.

Ballet and Tennis are the main counters to pesky-brain-plaque. Constantly flexing your internal navigation system sounds like it achieves the same thing by the sounds of this headline. [I did not read the article]

The OS should do less not more

Definitely get rid of that login wall

And when you need water for drinking or showering do you go to the well to get it?