XTXinverseXTY

Hard to separate the memorability of the abstract from the underlying material.

IMO the most memorable papers contain some unexpected simplification: a complicated problem turns out to reduce to something much simpler [0], perhaps for a counterintuitive reason [1].

So a memorable abstract should advertise that the paper contains some cute little trick. This may not help you actually publish though

[0]: Attack the RLHF problem with a simple classification loss: https://arxiv.org/abs/2305.18290

[1]: Frustratingly Easy Meta-Embeddings: https://arxiv.org/abs/1804.05262

besides overstepping any reasonable definition of creditworthy, it's really stupid.

OP's was leaky slop from day one [0][1], as is his article [2]

It is arrogant and entitled for the author to take credit for the concept of RL over sequence embeddings, and none of the work that went into pretraining, not to mention the egregious target leakage [1]

[0]: Author fails to grasp the concept of virtual environments https://www.reddit.com/r/LocalLLaMA/comments/1kl0uvv/comment...

[1]: his `train.py` has `outcome` as a model input (conversation_metrics built from _parse_conversation which includes outcome): https://huggingface.co/DeepMostInnovations/sales-conversion-... https://huggingface.co/DeepMostInnovations/sales-conversion-...

[2]: 100% of this post is AI-generated https://www.pangram.com/history/97e0be84-391d-46b8-9c16-2d8f...

Scaling laws project that a model with more parameters trained for longer on more data yields predictably better performance, and that generally you want to scale these factors commensurately. More of the compute budget is being spent on RLVR [0] for which we also fit scaling laws

Researchers tweak data mix, reward shape, model architecture, etc etc, breakthroughs which reduce the cost to train a just-as-smart model. But this increases the returns to scale, which further incentivizes bigger models trained for longer on more data

[0] "...to run reinforcement learning training...at pretraining scale." https://x.ai/news/grok-4?_bhlid=b9339d7816a05adeb52bae7050cc...

Puts the onus on the AI companies to provide a specific replacement mechanism, no? Unless I'm unfamiliar with something else he's written that proposes something more specific and constructive

To Tao’s credit he obviously identified the problem very clearly and admits understandably "we did not have the time to have a more consultative process, as with Leiden; but we decided that the urgency of the situation was such that we needed to release a statement sooner rather than later".

If they could declare with certainty that Buckminster's and Alpoge's usage data had been totally excluded from training, would that set a worse precedent and reflect poorly on their de-identification process (and data access safeguards moreover)?

This may sound like a charitable interpretation of OpenAI's remark, but consider that the lie would be (I think) impossible to falsify from the outside. They could easily just say "no sir we didn't peek" unless:

1. The conspiracy to peek at codex sessions involved enough people that the risk of one snitching is non-negligible

2. Lawyers advised it would be a bad idea to make such a remark, whether true or false

Apparently this post was prompted by a scary-sounding headline in The Information[0], that Astra is a looped transformer, implying CoT monitorability may be less reliable. The day after the report, Jakub tweeted[1] that he "wanted to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4." This post seems to elaborate on that.

I imagine that the AI labs have an uneasy truce to prioritize alignment and monitorability. Following the HF incident, OpenAI probably feels especially sensitive to being perceived as reckless, lest other labs feel obligated to defect.

[0] https://www.lesswrong.com/posts/PLisnSFir8y5AHkmP/how-concer...

[1] https://x.com/merettm/status/2095023204993490967

unnecessary condescension

So you admit that chapter 7 does not read almost as poetry?

I noticed that Clawdbot’s initial acolytes seemed to skew towards solo founders and hustler/grifter types. The Mac minis were likely to spam leads over iMessage. The single top downloaded skill was for Twitter. The fastest way to monetize an openclaw agent is by spamming fake social proof for your product (including for openclaw itself).

I cannot tell, from the article, how to perform the Buteyko method.

From the "Medical Evidence" section, it seems I'm not missing much.

Seems like this signals Yann Lecun's direction now that he's leaving Meta

The EMA teacher model still seems like black magic to me (present in DINO and JEPA series but gone now)

This is an extremely common use case.

Reading your comment history: are you an LLM?

https://news.ycombinator.com/item?id=44531907

https://news.ycombinator.com/item?id=44531868

Forgive my indelicate question, but why would someone buy a PO franchise?

Friedman's thermostat

Analyst visits his lumberjack cousin one Christmas at his cabin. Notices the cousin puts an amount of fire in the fireplace, which is correlated with the outside temperature, while the inside temperature remains constant (uncorrelated with firewood or outdoor temperature). Analyst wonders what his cousin is wasting all his wood for.

http://bactra.org/weblog/1178.html

> I must've missed the part where the models began iterating under their own power

He's probably referring to AlphaGo/AlphaZero in that sentence

More intelligent people can afford to take on greater risks than most people. They make better inferences

OP shouldn't try to imitate his friend, thinking he'll find similar success. It's an unpleasant conclusion but it's conducive to staying alive

Guys like him get to do that, because he has the sort of brain that allows him to get a PhD in physics

The world is a whole lot more dangerous and confusing for everyone else

That's extremely pertinent information, why didn't you say so?

An MS probably doesn't have that much more of a causal effect on your ability to do good ML work, as opposed to an equivalent amount of diligent self-directed study, which is what I assumed you were considering. But *the hiring market is dumb*, hence my spiel about social signals.

So now I'm less confident about the hiatus. I still don't think it's a good idea, but certainly not as terrible, and I think the HN consensus would be slightly in favor of it.

If you haven't already, consider posting this question to teamblind. Yes it's an incredibly toxic community, but it attracts people ruthlessly interested in maximizing their compensation. HN is biased towards entrepreneurship.

This sounds like a bad idea. You are very smart to have asked before proceeding.

I work as an MLE at a growth-stage startup with ~20 MLE/MLS folks. I was unemployed for 6 months before getting this job with 4YOE as an MLE and DS. There are too many qualified candidates.

ML research is out of the question. Most of the people who get to do ML research have PhDs, if not the only people. This seems like a racket but it exists for a good reason. It's hard for employers to evaluate the quality of MLS candidates through an interview, they don't know what to ask him. And if they've hired one, it's hard to know whether to fire him, things rarely pan out in research. The whole time, they have to trust this dweeb to run experiments burning tons of $$$ in compute! Employers are wise to be risk-averse, and to defer to costly social signals.

If you're going to take yourself off the job market for a long time, you had better at least get some kind of legible social signal out of it, like a master's degree. Almost all of the MLEs I work with have at least an MS in a relevant subject, the rest have PhDs.

I've heard great things about bacopa monnieri[0], but many people complain about decreased motivation as a side effect

Horizontal eye movements have consistently been shown to improve memory retrieval, dependent on handedness [1]

[0]: https://examine.com/supplements/bacopa-monnieri/

[1]: https://www.semanticscholar.org/paper/Horizontal-but-not-ver...

"Emergence of life in an inflationary universe"?

https://news.ycombinator.com/item?id=30047650

don't rub it in

I got a disappointing score on mensa norway[1] and boy, that made a lot of things suddenly click for me. I have yet to exploit this information for my benefit, as far as I'm aware.

I think this concept is called an information hazard[2][3]

1. https://test.mensa.no/Home/Test/en

2. https://www.lesswrong.com/tag/information-hazards

3. https://en.wikipedia.org/wiki/Information_hazard

I think he just means that it can be an incredible pain to install.

> according to George Hotz

In this video from June, George Hotz says to go with "3090s over 4090s. 3090s have NVLink... 4090s are $1600 and 3090s are 750. RAM bandwidth is about the same." Has he changed his recommendations since then?

https://youtu.be/Mr0rWJhv9jU?t=1157