hatthew

dupe: https://news.ycombinator.com/item?id=49890732

> there are plenty of dark places just outside your city

In the eastern half of the US and the western 3/4 of mainland europe, there are very few spots that are below bortle 4. In these regions of the world, people in big cities (i.e. most people) would still need to drive 1-2 hours just to get to bortle 4, let alone below that.

GP is very clearly speaking for america?

I think it's inevitable long term (decades/centuries) and arguably too big to fail short term (months/years). I'm not an economist so I don't have any opinion on how true it is to say they're too big to fail.

Saying "AGI/ASI isn't inevitable" is like saying "war isn't inevitable". Sure would be nice if it didn't happen, but realistically it's going to happen and it's better to mitigate the fallout rather than try to fight very large intrinsic incentives. I appreciate that this article doesn't conclude "let's stop doing AI", but rather "let's try to examine and adjust incentives".

> AI is just matrix math

Following the definition set for decades, AI isn't necessarily even as advanced as matrix math

I was on firefox on a newish mac, but it was when there were some any fish that they fully covered the screen, couldn't see the bottom of the pond.

This is a very good way of thinking about it! I like the nuance. Personally I don't usually include that much nuance when I write about this, because I don't expect my short comment to generate dozens of threads of discussion.

Information models aren't really something that one has, they're more of a semi-arbitrary frame of reference that we can choose. I'm assuming an information model where all public knowledge is accessible to everyone: you, me, your LLM, and my LLM. This means that conveying public knowledge doesn't convey any information (other than the fact that that knowledge is relevant to our conversation). This is a premise of my argument, and I'm fitting everything inside that frame of reference.

The LLMs don't have access to private information other than what you give it. If you give 300 bits of information to your LLM, the information it has is now "all public knowledge + 300 bits", and any text it generates is a subset of that information. By definition, it can't add any more information. If you give those 300 bits directly to me, I (and optionally my LLM if I want) now have "all public knowledge + 300 bits + my private thoughts", which is a superset of what your LLM has. Anything your LLM can infer can be inferred by me and my LLM. All your LLM can do is repackage that information into different text.

My opinion is that I don't get value out of that repackaging. I would rather read your packaging of those 300 bits rather than the LLM's packaging of those 300 bits.

For this current discussion, we could use an information model saying that your LLM has a knowledge base private to just you and it (e.g. local files or other conversations), and say that's separate from the prompt you give it. It sounds like maybe that's the information model you're thinking within.

In that frame of reference, maybe your shared knowledge base has 200 bits, and you type 100 bits into your LLM's prompt. My argument would still be that you're "giving" 300 bits to the LLM, and you should instead give them to me by sharing your knowledge base, or giving me the relevant information in your knowledge base in your own words. The latter option is definitely more work for you, and is the weakest point in my argument, but I'll still hold that preference.

It's running at 3fps or something like that. Is that a me problem or a "lots of traffic" problem?

My guess is that in the near* future, reasoning will no longer happen in a way that can be neatly decoded as human language.

*near meaning single digit years, which is far for AI I guess

We're not talking about humans vs LLMs in general, we're talking about humans copy-pasting LLM output as if they wrote it themselves. It's less "LLMs are bad" and more "if you want to communicate something specific to me, LLMs are a bad substitute for your own writing".

For a given information model, compression isn't a useful term, because "information" refers to a concept at its maximum compression. By definition, it can't be compressed further.

I think GP's point is that if the larger math community doesn't care about option C, Fefferman shouldn't have given that option in the first place.

Keep in mind that I'm using "information" in the information-theoretic sense. One could argue that because that particular solution to NS is provable, it is therefore implied by the propositions they started with and adds no new information.

And I'd still rather read a human's interpretation of the solution to NS than read whatever the LLM wrote.

Professional speeches are usually equivalent to LLM slop in terms of information content. They're written to sound nice first and foremost, and communicating information is a secondary goal.

Using an LLM to write something persuasive is a good way to persuade me that you don't care about the topic personally, and therefore I probably shouldn't care what you or your LLM think.

Okay, sure. The difference between option 1 and option 2 is completely irrelevant to this discussion and I gave what I thought would be an uncontroversial take on it because I thought it didn't matter. The point is at least one of option 1 or 2 is almost always better than option 3. Option 3 is what I and others are arguing against.

From a purely information-theory perspective, the simplest solution is to say that yes, any content derived from existing information carries no information itself.

From a realistic perspective in the context of people copy-pasting LLM output, my thoughts are that asking an LLM to research for you is more defensible, but it's still better to read the LLM's research results and write the important parts in your own words (partly because the LLM probably used way more words than necessary for the context).

In information theory, each bit is a coin flip by definition

> I think it’s normally known to you and the LLM

I disagree. Unless you gave additional information to the LLM yourself, the LLM doesn't know more than your audience does about what the meaning of such a comment would be. An LLM could certainly come up with something plausible, but it wouldn't necessarily be what you intended.

Maybe things will change in the future, but I know where things stand right now. Would you rather learn about math by interacting with the teacher, by using the internet (including public LLMs), or by having the teacher tell an LLM "teach a math class" and then copy-paste its response to you? Option 1 is the best, but option 2 is at least better than option 3.

genius

Art is an abstract method of communication. When you choose the words of a poem, you're (hopefully) doing it to convey a feeling within you to the reader. If you write a poem about a beautiful spring day, it's probably because you experienced one, or you're remembering one, or someone was telling you about one, and that evokes feelings within you that you want to put into words, right? Surely you wouldn't write a poem about something you don't care about in any way?

When you talk about something you're wondering about, you're saying that you're missing information. Your ponderings are dancing around the void in your knowledge, defining its boundaries, and maybe imagining what answers might be able to fill that void.

When you put your thoughts into words, they're insufficient. You have so many ideas swirling around in your head, and you can never put them all on a page in the fidelity at which they exist internally. But words are the best we have. Whatever words you write are your best attempt to convey your thoughts to me (barring other media). You're distilling your inner voice that speaks a language only you can understand, into an outer voice that others can understand.

I don't think I'm exactly refuting you here. I think what you've written makes sense, and caused me to think about many things, more so than any other reply to me today. But I also don't think your comment is refuting the point I was trying to make, mainly that LLMs rarely add value in human-to-human communication.

I could probably have pasted my comment and yours into an LLM, and it would have come up with a clearer thread connecting my words to yours. But that thread probably wouldn't have been any of the ones either of us saw, would it?

Thanks for adding a new perspective to the conversation :)

I love it :)

- An LLM has more knowledge, but it doesn't have information about what you specifically want to convey. Anything you give to the LLM may be good information. Anything the LLM adds is not additional real information, because whatever it adds can be inferred based on whatever you wrote. Or the LLM adds additional information that can't be inferred based on what you wrote, which is even worse because that's basically just misinformation.

- If you're giving additional prompts to the LLM to refine its output, then you're the one adding real information, not the LLM. The LLM is just rephrasing the information and adding noise.

Yes, but specifically compress the concepts, not necessarily the text encoding. "100 commas" and ",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,," may be different amounts of ascii data, but they both represent the same concept that's worth around 20-30 bits of semantic information.

If there's a significant risk of misinterpretation when writing only 300 bits, then the idea you're trying to convey is worth more than 300 bits. In the process of prompting the LLM to add information, you're giving information to the LLM that you could instead just give to your readers directly. If you're just using an LLM to expand your thoughts in the hope that you get a better piece of writing, maybe you should put more effort into your own writing.

(This is all under an information model that assumes the LLM and your readers have equal access to knowledge, which I probably should have made more explicit in my original comment.)

Summarization—especially of private context—is definitely one of the main exceptions to LLM writing being useless. However, it's still preferable to turn your private context into shared context and then give pointers to that, rather than having an LLM attempt to summarize it.

From an information-theoretic perspective, if I can effectively transfer a latent concept to you in 300 bits of quantized communication, then by definition the latent concept itself isn't more than 300 bits. Any bit over 300 in the communication I send to you is fluff.