a_wild_dandan

Well, an AI charity -- so founded on openness that they're called OpenAI -- took millions in donations, everyone's copyright data...only to become effectively for-profit, close down their AI, and inflict a lifetime gag on their employees. In that context, it feels rather amoral.

I think superalignment is absurd, and model "safety" is the modern AI company's "think of the children" pearl clutching pretext to justify digging moats. All this after sucking up everyone's copyright material as fair use, then not releasing the result, and profiting off it.

All due respect to Jan here, though. He's being (perhaps dangerously) honest, genuinely believes in AI safety, and is an actual research expert, unlike me.

Also, will Ilya likely have similar contractual bounds, despite the unique role he had at OpenAI? (Sorry for the self-reply. Felt more appropriate than an edit.)

What do I do with this information?

Is this a legally enforceable suppression of free speech? If so, are there ways to be open about OpenAI, without triggering punitive action?

People make different decisions due to different preferences. There’s no need for melodramatic character judgements here, relax lol. You’re not stupid. People who chose differently aren’t either.

What’re your thoughts on MLX? It’s been phenomenal on my MBP.

I'm here to intentionally get you started on hidden CLI settings. Learn me somethin'!

(in llama.cpp, for CPU)

That's not too bad. Keyword: decisions, not advice/topics/recommendations/etc. So don't execute transactions, or hiring decisions, tenant application rejection, etc. I prefer no restrictions, but the reality is less melodramatic than hellbent HN contrarians would make you believe. I think it'll be okay.

No you don't. Compare the model's performance before and after uploading the material.

Gemini works with brand new books too; I've seen multiple demonstrations of it. I'll try hunting one down. Side note: this experiment is still insightful even using model training material. Just compare its performance with the uploaded book(s) to without.

1. Social media's objective function is engagement. Outrage and like buttons are highly engaging.

2. Tribalism + scale + time = two "us vs them" groups. Politically, FPTP voting mathematically implies spoilage; the Ron Paul example doesn't extol some virtue.

3. People are furious due to these (utterly preventable) things: https://www.youtube.com/watch?v=qEJ4hkpQW8E

You can fit a Q5 Llama-3-8b or many other models on new iPhones.

Like humans, multi-modal frontier LLMs will ignore "removal" as an impertinent typo, or highlight it. This, like everything else in the comment, is either easily debunked (e.g. try it, read the lit. on LLM extrapolation), or so nebulous and handwavy as to be functionally meaningless. We need an FAQ to redirect "statistical parrot" people to, saving words responding to these worn out LLM misconceptions. Maybe I should make one. :/

Also, these two networks achieves vastly different results, per watt consumed. A NN creates a painting in 4s on my M2 MacBook; an artist in 4 hours. Are their used joules equivalent? How many humans would it take to simulate MacOS?

Horsepower comparisons here are nuanced and fatally tricky!

I show +23 points. Are popular comments down ranked to encourage diversity? Oh well, I'm proud of the brief success in challenging misinformation here!

Better title: Image classification models suck at identifying nouns that they've rarely seen.

Crucial context:

- They're only looking at image models -- not LLMs, etc

- Their models are tiny

- A "concept" here just means "a noun." The authors index images via these nouns.

- They didn't control for difficulty in visual representation/recognition of these exceptional infrequent, long-tail "concepts."

If I didn't know an object's label, I too would struggle to identify/draw it...

This is correct. To give more synthetic data examples:

1. Generate inverted problems which are easier to produce than solve. For instance, create an integration math exercise by differentiating a hairy function and reversing the steps.

2. Create (simulated) environmental data.

3. Use adversarial model competition, e.g. self-playing chess or training an artificial image generator/detector model pair.

There's evidently a commonplace myth that information quality starts pristine and exclusively gets degraded by systems thereafter. That's just easily, demonstrably false in myriad ways. That's why it's absurd to conclude that LLM output being in internet training data will cause model collapse.

Information-rich synthetic data can be created without humans, and it works. (Check out Phi, for instance.)

There are phenomenal web scraping tools to crudely "preprocess" the document a bit, slashing outer HTML fluff while preserving the small subset of actual data. From there, 8k tokens (or whatever) goes really far.

Llama-3 70B on my local MacBook works wonderfully for these tasks.

3rded.

There's nothing substantive in this article. The author expresses a seething hatred for OpenAI's CEO by repeatedly lampooning ("where's the evidence!?") brief quotes, without context, of Altman's opinions/predictions. I hope that the author escapes whatever rut made him so overwhelmingly bitter and resentful. I've been there, and it truly stinks. :/

And they're expensive. You pay thrice:

1. Engineering the generalized solution, then

2. Carrying the cognitive burden of this new indirection layer, and finally

3. Ripping out this premature -- and so likely wrong -- abstraction

Premature abstractions are like alcohol: fun borrowed from the future, and paid back later with interest. If unavoidable, choose the abstraction most narrowly scoped to your actual code base, not a future one. Humans suck at prediction. Ask me how I know.

new jargon != any jargon

Plenty of STEM jargon existed which Shannon chose to not use.

Google has slashed one term in my two word query. The results of this 50% query reduction, predictably, were complete trash.

That's a phenomenal idea! Time to find some parents.

You're describing symptoms as the disease. People elect officials, sustain media outlets that support their viewership's confirmation biases, and organize by tribal beliefs. People are the messy, complicated root cause. New symptoms will always emerge, regardless of how much focus/treatment they get. Symptoms can sometimes look like a cabal. But they aren't. The root is normal people.

> Regrettably, the marketplace is far from healthy

The unhealthy thing here isn't the marketplace, it's government policy. If you eschew popular open-source tools for closed/proprietary alternatives, this will happen. Linux will always have several orders of magnitude more experts banging on it than Windows ever will. Vendors pontificating about security will not change this fundamental asymmetry. Also, don't want product lock-in? Don't choose vendors with a significant risk or history of lock-in. Don't endorse face-eating leopards and then blame others when they eat your face.

Training is done by partially hiding (masking) text and making a (decoder-only) model predict what's missing. Training text which humans prefer as model responses is called RLHF. It's an expensive and small text dataset.

FYI, Trump has vowed to purge the executive branch of unloyal employees, aiming for unitary rule during his 2nd term. It’s unclear how much of this his conservative SC majority will allow, but…the norm of stable, institutional independence might be gone soon.