still_grokking

> Except they’re not storing copies of any of these.

Copyright isn't about storing copies. It's about making copies.

How do you think the AI companies extract all the statistical info about some content?

Spoiler: They copy that copyright protected work onto their machines where they do the data analysis.

At this point copyright was already violated. They usually never had the right to make copies for the purpose to extract statistics about it in the first place. "Because it's on the internet" doesn't give you such a license.

The current LLM "AI" only exists on the grounds of massive copyright volitions. It's the exact same situation as with Napster in the past.

No mater how often MS and friends hallucinate something about "fair use", it is not, and the only realistic legal outcome in the long run will be the same as for Napster: AI training is piracy. On a global "internet scale". (The only sad thing is likely nobody of the directly involved will end up in jail, like they do e.g. with people sharing torrents).

That's why for example fan-made Star Wars movies are totally OK, and Disney has no issue with them, right? In such cases not even the plot is based on anything Disney "owns".

Also, it's OK to make a movie out of a plot summary someone who read some book gave you, right?

You really think such things become legal if you have some ML algos in the loop?

https://web.archive.org/web/20220307083651/https://fairuseif...

Given the fact that you need to review, research, and usually correct every detail of AI output, how can that be faster than just doing it right yourself in the first place? Do you have some examples of such tasks?

What are examples of these (very profitable) use cases?

Producing spam has some margin on it, but is it really very profitable? And else?

So the semantics of this program would change if I added a line containing an exclamation mark somewhere after this expression? That smells like the worst kind of action at a distance…

Also, how is the expression `doSomething!` typed then? Does it have different types depending on where it's written?

And how do I actually write something `traverse`-like? How to actually write

  list.forEach (lambda f => readFile! f)

than, if I really want the effect to run inside the lambda body?

That would work better and more efficient.

But than there's no "AI" in there. So nobody would like to throw money on it currently.

It would be even more "funny" if you'd wrap that `readFile` inside a thunk that gets captured from the environment by the lambda body. Than you'd have a reference to an effect that is supposed to run only inside the lambda but could be executed by external code. Depending on the effect this can have unexpected up to catastrophic outcomes.

That's why Scala introduces so called "capture checking" to make direct style effect systems safe.

https://docs.scala-lang.org/scala3/reference/experimental/cc...

To understand the problem better see:

https://www.inner-product.com/posts/direct-style-effects/#ca...

Where is the list of files in that example?

I also don't get how running

  list.map (lambda f => readFile! f)

would result in a list of Tasks. `readFile` needs already to return a Task as it's an I/O operation, right? Appending the exclamation mark will CPS transform that Task value by wrapping it into (possibly nested) calls to `Task.await`, as I understand the article. But CPS inside lambdas is problematic as you need to also CPS transform the lambda, more or less rewriting your whole program "inside out" (all up the whole call stack!). Also every higher order function would need to be there in two variants. HOFs need to get "shifted" manually for that. That doesn't scale.

I mean, it's not impossible to implement something like that. But it's not trivial. The problem with the HOFs remains still. See for example CPS-async in Scala:

https://github.com/rssh/dotty-cps-async

Sorry to be so direct, but your rules just don't make any sense to me. Let's go through them one by one:

> Recommendation based on previous views. Recommendation based on what is currently being viewed is permitted. Recommendation based on current view that is customized to user is not permitted.

This would ban all kinds of news aggregators, or even just simple help-desk / support ticket systems.

> Ratings up/down.

This is one of the most important features of an internet forum! Just imagine HN without votes. It would be flooded with nonsense!

> Sorting based on rating or based on users' interest. Chronological sorting is permitted.

I don't want a chronological information thread most of the time. I want to see the things that are relevant to me. A news ticker full of stuff you don't care about is just a big waste of life or work time.

If I'm working on project X with technology Y I want to get relevant information. Without the need to search for it explicitly. The computer knows anyway what I'm working on, so it should show me the relevant information. That's the whole point of a computer: It processes information for you so you don't have to go through it manually.

> Suggesting content is forbidden. Specifically subscribed content is permitted to be suggested - but only chronologically, or in response to a search.

How do you discover interesting things you don't know about already? Should we ban the "see also" section on Wikipedia, too?

What again about work organization tools? Should they be kept dump instead of helpful?

> No public comments. Private comments permitted. Comments in room/forum/group permitted.

I don't even know what this is supposed to mean.

Is a blog post a public comment?

Spam email is a private comment, right?

Reddit or YouTube comments happen in a room/forum/group I guess?

> User must specifically subscribe/join to see the comments.

Yeah, sure, you need to be logged in to read Stackoverflow comments. But you can only see them when you joined the discussion of some question, otherwise no comments for you. Do I get this right?

> Comments sorting/notification rules same as above.

Sure. Your inbox full of trash notifications about "chronological events"—that don't matter to you.

> "Reactions" to messages show up as additional/new replies, and are not attached to the original message.

OMG. Back to "+1" comment threads on GitHub & Co…

---

The whole point is: Something that works like "social media" is just a communication tool. A tool by itself is not good or bad. The difference is in who's interest the tool is applied. When I install something that works like Reddit on my own servers and use it to discus internal topic related to my workplace this tool is likely great. The same Reddit-like software operated by an company that seeks out to sell ads to people is likely dangerous to the public.

The problem now is, the whole internet is run on ads. So more or less any communication platform on the internet is potentially malicious.

Of course there are exceptions, like for example HN. But these are rare cases. Also note that something like HN ticks a lot of the "not permitted" boxes outlined above. This just makes no sense as I don't think HN is harmful. More the contrary. So the stated "rules" just don't work.

> extraordinarily cynical and depressing

You mean, like the reality humans created for themself to suffer in?

So you want to ban all kinds of software forges, like for example GitHub? Or other regular coordination platforms for business?

I guess the definition of "social media" is not as simple as outlined above.

Thirty years earlier people were also "always at home". Watching TV, or playing console games…

The big difference was: At least the youth was at home together with others doing so. You went over to your neighbor to watch some TV show for example.

What changed quite lately are indeed the always online smartphones. You don't need to go to your neighbor, even if you want to "do something together" (of course online, like everything nowadays).

Also people don't even call each other. The default is chat, or very often voice messages. But no real talking.

So no, media and electronics aren't the main driver here. It's the always online devices everybody uses everywhere the whole time. Human minds likely aren't constructed to be always connected (or one could argue, actually disconnected) in such a way.

It's not primary the quantity, it's the quality of media consumption that significantly changed. Also the media is the first time completely interactive and personalized. Compare to for example TV (which was back than "the devil who destroys the morale of our youth" according to some elder people). You didn't get targeted personalized ads the whole day on your TV…

The psychological manipulation that the pre-smartphone media could possibly exercise on individuals was much more constrained compared to what's possible (and actually gets executed) today. Smartphone apps are a great example as they're now mostly constructed to induced addiction (the companies selling ads call it "engagement").

That's my point. Imho it's impossible to even come close to a realistic chance to beat this game without knowing more or less every detail of its internal mechanics. And you can't infer this knowledge just from playing. You need to look it up from some out-of-band source.

So only letting an AI play this game will not reveal anything about the "reasoning" capabilities of said AI.

Letting the AI read the source (or some other "spoilers") and than look at how it performs playing would be maybe a usable test. But that's still a big "Maybe" imho as Nethack is just "too random" I think.

> the mechanics are certainly random in the sense that they involve dice rolls.

Sorry, but that's still not what I've meant.

Dice rolls are randomness in the usual meaning of this word. But "random" can also mean "arbitrary" and/or "illogical" things. Imho Nethack mechanics are also random in this sense. They don't make sense at all… :-)

> Even moderately experienced players will fail close to 100% of the time. So getting to an almost 50% success rate, to my mind, shows a great deal of skill!

OK, you have definitely a point here.

Still not my cup of tea, such games. (And this is strictly personal, and unrelated to the rest of the discussion). I just don't like games where it's very likely that I will loose despite "doing everything right". I have no problem with games punishing merciless even small mistakes. That's OK. But having an outcome that depends mostly on the whim of the RNG is just nothing I enjoy. When I "do everything right" I like to get rewarded appropriately for it. (Of course some level of randomness is still OK. But if the RNG kills you most of the time no mater what you do this is just too frustrating for me. So I'm clearly not the target audience for Nethack… :-D).

> I don't think this is true any more for modern AIs such as AlphaGo and its predecessors, which learn distributions of possible outcomes rather than deterministic predictions.

Sure. But how do you learn from a distribution where no matter what you do you will fail in, say, 99,9% of the cases?

> IIRC the latest versions can even self-play games like Poker to a superhuman level.

Do you have some links regarding this? I thought Poker is still one of the games where AIs don't play better than humans. OK, maybe it depends on the Poker variant. There are simpler and more difficult ones.

> If there is any systematic deviation from pure randomness, enough data will reveal it!

I would agree in general.

But now we're back to the initial question: Is there enough systematic deviation from pure randomness in Nethack? Given that even people who know all the mechanics, and know some good end-to-end strategies will fail in most cases (actually, like you said, in almost all cases). And given that it's (imho) impossible to come up with this knowledge about mechanics and strategy just by playing this game. I have my doubts.

You've just restated the halting problem.

Nobody claimed that there is an algorithm to translate arbitrary programs into an equivalent lookup table. (Because that's the exact same proposition as stating that there is a program that can compute whether an arbitrary program halts when executed).

The point is: Any specific program can be translated into a lookup table. Computer programs and lookup tables are equivalent!

You claimed that computer programs are somehow "more powerful" than lookup tables. That's just plain wrong. They're exactly equivalent in "power".

> Wrt inefficiency, training will be inefficient

That's the "heating the planet part" I was referring to. :-)

> but the programs can be extracted to CPU instructions / CUDA kernels during inference

This just makes my original question more pressing: What are the NNs good for if the result will be normal computer programs? (Just created with astronomical overhead!)

> Cyc looks cool, but from my cursory glance, is it capable of learning, or is its knowledge graph largely hand coded?

The whole point is that it can infer new knowledge from known facts through a logical reasoning process.

This inference process was run since 40 years. The result is the most comprehensible "world knowledge" archive ever created. Of course this wouldn't be possible to create "by hand". And in contrast to NN hallucinations there is real logical reasoning behind, and everything is explainable.

I still don't get how some "dreamed up" programs from your project are supposed to work. Formal reasoning and NNs don't go well with each other. (One could even say they're opposites). Imho it's "real reasoning" OR "dreamed up stuff". How "dreamed up stuff" could improve "real reasoning"? Especially as the "dreamed up stuff" won't be included in the end results anyway, where only the formal things remain. To what effect are the NNs included in your project? (I mean besides the effect that the HW and energy demands will go through the roof, ending up billion times higher than just doing some lambda calculus directly…)

And yes, these are genuine questions. I just don't get it. It looks for me like "let's do things maximally inefficiently, but at least we can put a 'works with AI' rubber stamp on it"; which is maybe good to collect VC money, but else?

What do I overlook here?

I think my wording was bad. There are two kinds of "randomness" here at play and I didn't differentiate properly.

For me the game mechanics as such are "random". Because you can't discover them by just playing (imho).

At the same time the game is ruled by a dice. (So even the best players will fail almost 50% of the time which is almost as random as tossing a coin, and strictly not skill based).

> Given that AIs can play hundreds of thousands of runs in the time it would take me to play one, I'm a bit more optimistic that they could learn the mechanics eventually.

My gut feeling says the opposite. How do you infer any kind of rules from almost random events? Especially if the "logic" behind the "non random" parts is actually also quite made up and arbitrary (so in a sense also "random", even to people with reason).

An "exploration-based reward function" wouldn't be enough. Because this would assume that exploration has (more or less) deterministic outcomes. But given the dice in Nethack it actually does not! You can do "everything right" and still die in almost 50% of the cases. How to infer any meaningful "world model" from such events? Imho you can't.

(I can confirm that looking up spoilers will let you make progress in Nethack. That's why I think it's boring. I've tried hundred of times prior to looking up spoilers and didn't make any progress. But after biting the bullet and starting reading some walkthough it was actually quite easy to reach some deeper levels. Until I've hit the next invisible wall. Which would require again some out-of-band knowledge… I know that reading the next spoilers would also make this wall go away. But I've lost any interest in this game after finding out exactly this: It's impossible to play without a walkthough; and with a full walkthough it's actually considerably easy, and comes down to "just having luck". At this point I could just toss a coin to determine whether "I won". That's maximally boring. I don't like dice games; and the exploration part in Nethack leads nowhere because the world is arbitrarily made up. You can't discover the mechanics without already knowing them…)

May I ask why there are NNs in your project at all? Just to heat up the planet and make Nvidia share holders even more happy? :-)

I mean, what I've seen in the Readme makes sense. But doing basic computer stuff with NNs just makes the resource usage go brr by astronomical factors, for imho no reason, while making the results brittle, random, and often just completely made up.

Also: Do you know about the (already 40 year old!) project Cyc?

https://en.wikipedia.org/wiki/Cyc

This software can indeed "reason". And it does not hallucinate; because it's not based on NNs.

Of course one can beat Nethack. Many people did. That's not the point. The question was more: Without "spoilers" (a.k.a. "hardcoded heuristics and goals")?

(I never made progress because I didn't try hard enough. It became very boring after finding out that this game is random and quite nonsensical, and one can't come up with some strategy only by playing it often enough. I'm in general not enjoying dice games. I prefer games where you can come up with some winning strategy by curious observation and logical thinking.)

Because of the nature of Nethack I don't think it's a good AI test as such.

Maybe it would be if one let the AI read spoilers / walkthoughs and than let it try playing. Such a test could than maybe probe for the AI's text comprehension, and the ability to map the gained understanding to concrete actions. But just letting it play Nethack unprepared does not give any insides into the AI's capabilities, imho. It will just fail over and over again. Because it's (imho) impossible to beat Nethack without spoilers. You just can't extract the needed knowledge from playing. Even from playing it millions of times.

Every computable function can be represented by a (possibly infinite)¹ lookup table.

Computer programs can only compute computable functions. Therefore any computer program is (in theory) equivalent to a table lookup.

¹ For finite inputs, the lookup table can be finite, and for infinite inputs, the lookup table can be infinite but still countable, as the set of computable functions is countable.

Is it actually possible to beat Nethack without reading up some "spoilers" upfront? I've never heard of anybody who managed to do that.

Even when you read up all kinds of info about the game before you attempt a run it's extremely hard to reach higher levels, yet beat the game. (I myself never reached any later levels despite I know some tricks by now. Tricks impossible to infer from just playing the game; you need to read them up…)

Imho there is no winning strategy for Nethack. It's some random stuff "you need to know" to progress even a little bit paired with complete rule of the dice while encountering maximally nonsensical "puzzles".

But OK, maybe I'm just too dumb for this game and don't see the "logic" behind the things the game presents.

Depends on the FS and tools used. With for example XFS and `xfsdump` / `xfsrestore`, or ZFS and `send` / `receive` you would only copy the data, and something like a "vacuum" would happen also.

It's much simpler on Linux. You just do `# apt install nvidia-driver`, and reboot. That's all.

This also won't install all the horrible security nightmares that come with the Windows drivers, as that stuff doesn't exist on Linux.

So overall a much better experience than on Windows.

You don't run your dev tools on the production boxes.

And you don't run your apps directly on the production boxes either these days.

There is just no reason to endure the constant pain of outdated dev environments.

Of course that doesn't mean that you can't develop against a stable environment, in case you plan to deploy directly onto such thing. That's usually independent of your tooling. You just need to look for the right versions of your deps (which is trivial within some "container"; which runs of course on your up to date dev box).

I'm not talking about dead-code elimination (that's something that compiles do since decades, without AI), and not about something like Scalafix's automatic refactorings (which are actually deterministic and correct, because it doesn't use AI), but about some true AI that could simplify code—and remove in that process at least 80% of it, because that's usually just crap that piled up over time.

Like said: Wake me up when when you can show me metrics where the use of AI shrunk code-bases by significant large amounts. For example going from 500 kLOC to 100 kLOC, or something like that. (Of course without loosing functionality; and at the same time making the code-base easier to understand). That would be success.

Everything that goes in the opposite direction is imho just leading inevitably to doom.

How does semantic information work than (showing inferred types and such stuff)?

How does code navigation work in such a setup?

With powerful high-level languages you can reduce boilerplate to almost zero. (And for the rest you can use code-gen.)

Copy-paste on steroids (AI code assistants) OTOH will lead to doomed code-bases even faster than manual copy-paste did in the past. People are going to learn this really soon. Maybe two or three years left until the realization will kick in.

> I'm told that Google has internal AI dev tools now that generate more code than humans.

Given that code is not an asset but a heavy liability the elephant in the room is the question: Who is going to maintain all that huge piles of generated code?

Wake me up when there's an AI that can safely delete code… That would be a really disrupting achievement!

But GP is actually arguing to call all your things "potato".

Imho that's the worst you could possibly do! Indeed.

Wasn't Eternal Blue a reference to the Windows "blue screen" bug it was based on?

I would guess there are also technical references in the other names.