Hacker News Reader

Comment by bonoboTP | Hacker News Reader

Citing 2023 makes it seem like this is newer than it is. Compression, prediction and intelligence have long been known to be deeply connected.

View on HN

You expect every blog post to find the earliest relevant paper to cite, just so one could look at the year (without reading said paper - which would have made clear that the connection isn’t recent) to assess novelty? I don’t think that’s reasonable.

It’s a blog post. If it was, say, a peer reviewed paper by Hinton or LeCunn that fails to cite Schmidhuber, that would be reasonable criticism in my opinion. (Spoiler: they fail to cite him)

Why would blog posts not be subject to such criticism?

Either the author knew of prior work that argues the same thing and they ignored it, or they didn't know. And if one writes a 1000+ word article premised on this idea, wouldn't one be presumed to know at least in which century the idea originated from?

Arguably these kind of blog posts should be more subject to such criticisms, because the blog posts purport to "teach" the general public about a concept in an authoritative tone (or at least the author seems to pose as knowledgeable in the subject), while for academic papers, everyone who actually reads the paper knows where the ideas came from anyway and it's mainly an issue of attribution (and maybe about fairly distributing the citation count...)

> Why would blog posts not be subject to such criticism?

You're asking why casual comments from amateurs made for fun on the internet shouldn't be held to the same standard as those made by funded career academic experts writing for other experts over months and meant as part of the permanent record of the field?

Personally, I think that's a bit like asking why a friend having you over for dinner isn't always an elegant 7-course meal with wine pairings. I guess you can expect that if you want, but to me it sounds like a child expecting to go to Disney every day: ignoring the economic realities of the situation is a recipe for eternal disappointment.

I think you shouldn't criticize the single blog post in vacuum.

If the blog is a personal collection of "today I've learned" then different standards apply then "my blog is my research publishing platform".

Both are valid blogs. The fact that it ended on HN front page didn't mean it was targeted towards this audience

This is why I hate online arguments. I didn't say earliest, but to give a pointer to the general era, and make it clear how standard the concepts are. They are part of undergrad, it's basic things. I'm not asking for a deep lit review. But modern exposition to anything related to Ai / ML / stats have extreme recency bias and young'uns are led to believe there was nothing before the transformer paper.

I guess it depends on the actual goal of the post. Education? Point people to background. Surely if you have a deep understanding of the subject, you have favored references.

If it's not education, where people are supposed to know everything, who is the audience? If the audience is supposed to know all of the background, then the post is not saying something original to the audience.

I agree the quality of the text itself is good. But I knew the background and I left (well, actually began) already agreeing.

I am happy and want people to write and share stuff. I think it's a really good exercise if only for the author. To be clear, I think the post itself is good except for the gap I mentioned in my original post, which in the grand scheme of things may seem like a nitpick. But if the same type of content makes it to the front page often without me understanding who the audience actually is, I am also allowed to wonder why out loud.

> You expect every blog post to find the earliest relevant paper to cite

This should be expected out of everyone. If you don't respect the reader enough to do this, why should we read your posts?

I think papers should be retracted for not citing prior art, even if you weren't aware of it.

If the blog were about calculus, and stated that an elegant proof of the Fundamental Theorem of Calculus could be found in such and such undergraduate textbook, would you be upset that the citation wasn't to either Newton's or Leibniz' work?

Upset no.

But I would like it to be a violation of norms around citation. We should respect the reader and the truth.

I would want them to point to point to what they think is the best resource. (Assuming the point is educating or explaining something to the reader.) Which, If everyone followed, would automatically point back to newton/leibnez.

That compression and prediction are deeply connected is obvious from simply considering how compression algorithms work.

Nothing about the concept of "intelligence" is nearly as well defined as the other two, nor as well agreed upon.

"Understanding" is clearly linked to compression. Taking observations and coming up with a more compact representation that explains them, analogous to coming up with a compact set of axioms that generate facts, or a small Turing machine or short program that generates a list of strings.

Intelligence is a broader concept but definitely involves understanding how a system/envoronment works and making predictions about its unfolding, especially actionable ones that allow you to steer that state towards some goal states.

These are not new.

At least since the 1960s: https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_induc...

The two contributioms that come to mind are the Hutter cash prize for the best compressor of the English Wikipedia and the work on PPM compression-based text classification by the late Prof. Ian Witten's group at Waikato (NZ) [1,2]. The model that best compresses the input string was likely generated by the distribution from which the compression model was 'trained'

[1] https://news.ycombinator.com/item?id=27440093

[2] https://www.computer.org/csdl/proceedings-article/dcc/2000/0...

Reply on news.ycombinator.com