galaxyLogic

Great idea

My favorite metaphor for programming is playing chess. Your opponent in programming is the complexity, you don't see its moves before the coding and design progress, before you make your choices/moves. You solve a problem by writing some code but that causes new problems down the line you didn't know existed before you made your choice of writing some specific code. or choosing a specific design.

Chess-players too are in a very "meditative" state when they play, and they enjoy it, I assume because it let's them focus on the game and forget about everything else.

Maybe something like: When you do derivation you analyze how the function behaves at a given point. Whenever you analyze something you analyze a given thing, not all things. When you do integration you look at a function and then produce something that describes the function on every point. You come up with a closed integral-formula for the shole function. You need to synthesize something that describes the function everywhere. But when you derivate the dericate, you also produce a closed function that describes the function everywhere. Oops, I think I got lost here too.

It's also about poor children getting worse primary education . According to Google "roughly 21% of U.S. adults are functionally illiterate".

If you never learned to read, good luck getting higher education.

I'm not defending communist societies like Soviet Union or China but I think "social democratic" countries like those in Scandinavia have shown generally good education outcomes.

Frankenstein!

What can you gain by looking at the weights, whether open source or not? Are they not what determines the model's output, but in an oblique way? We can't really fix the weights ourselves, weight by weight, or can we?

I'm just thinking in JavaScript I can do this:

  let s =  "abc" + "def";

Why can't I do:

  let regExp = /abc/ + /def/;   

If JavaScript (or some other) interpreter can turn /abc/ into a RegExp, why can't it do the same for

/abc/ + /def/

?

2 RegExp problems:

1. You can not compose a bigger regexp out of smaller ones

2. A regexp can not "call" other regexps

> Mathlib, the dominant Lean library, is a human-curated formalization of an ever-growing fraction of existing human mathematics.

So why don;t they use AI to write Lean programs? That should make the AI-proofs more readily human undersrndable.

I think what is going to happen is revival of "Methodology".

"Methodology" was a big thing in the past just before we got into "Agile Extreme Coding", instead of trying to model the big picture of SW development projects just jump into coding agilly. Implement it feature-by-feature

Granted the methdologies proposed ( See: https://www.ibm.com/docs/en/rational-soft-arch/9.7.0?topic=m... ) may have been too heavy and not flexible and not improved enough. But now with the rise of Agents I think we need to revise and perhaps re-invent them for AI agentic development.

The example was public speaking. Many people get very anxious if they need to do that. But if they think about it rationally they realize there is no real danger there. That's how they know it is an irrational fear, and they can use the breathing technique to make it managable.

So maybe it's more about if you know you should not or can not run away from a fearful sitution you should take a deep breath. Whereas if you are on a dark street alone and a group of hooligans look like they might attack you then feel fear and instead of breathing deep, you should run.

I'm a programmer so to me this brings to mind the idea of classes and subclasses. A program is implemented by having a set of classes. The classes can be organized into a class-hierarchy where they inherit methods from their ancestor-classes.

Now assume originally you did not have the feature of inheritance in your programming language so you would just create all the classes you need without orgnizing them into an inheritance-tree. Then you upgraded to a language that doe shave inheritance and you wanted to refactor your program to omit duplicate definitions of methods.

What kind of class-hierarchy would you come up with? There is no single way to do it. Some ways are better than others. There migh be more than one optimal way.

Same goes with generalization general, it is part of the language we create to describe things and there are many different languages we may come up with, some simpler, some more difficult to understand.

The LLM has its context-window. When it gets over that I assume it starts more or less repeating itself. Whereas human context-window has memories and inputs from all of one's life. Therefore great authors don't repeat themselves.

Now even if an LLM has a large context-window it is probabably not the case it rememebers all of your previous prompts and all of its previous replies. If you ask it to write a book you should probabaly give it all the previous 50 books (or blog-posts) it has written for you so far and you should tell it not to repeat itself. But in practice the context-window and the cost of token would become too expensive for it to write 50 unique books.

Maybe the problem is "all-or-nothing" -nature of LLM context window. Humans don't remmeber everything from past but they remember something from ALL OF their past.

That is definitely the essence of AI: It is the average of all the inputs it has been trained on.

Frank Zappa was once asked about guitar virtuosos like John McLaughlin and his answer was somemthing like "You can maybe plays solo faster than anybody, but can your playing surprise me?".

I find myself fixing the spec of my software often, and that makes lots of existing code obsolete. Creating working code is getting cheaper with AI, but creating great specs which make the software easy and intuitive to use seems to be more difficult for AI.

Why? Because you have to actually use the product to discover what is wrong, or sub-optimal, with it.

But fear is often good. Breathing slow to counter your fear should only be done when you know it is an irrational fear.

Good point, it's about "Intellectual Capital". If you can convince people that you know something better than someone else then you can start charging for it.

In Software Development we always had "Gurus".

The irony is that there's so much doctrine about Buddhism, Zen, Tao. Yet reading all those books and "becoming an expert" is probably not what zen is about.

Zen I believe is about meditation where thoughts quiet down and you experience the moment. When you do, it means your brain gets rest because it is not "chasing thoughts". And that means when you stop the mediation you brain is well-rested and can work better because it can focus much better, the runaway thoughts are not leading it astray continually. That means your brain, and you, can accomplish more.

It is probably true that many Western zenists have figured this out and use meditation for that purpose. Think about star-wars, the "force" is about being able ro lift a rocket-sdhip with the power of your concentration. That's the myth.

Whereas I believe when you achieve a zen-like state of mind it becomes less important to you whether you can lift the rocket-ship or not. In the Eastern tradition Zen is it own goal.

Is this something like what you meant?

Wouldn't writing out loud do it too?

Ironically it used to be the case that best developers who actually were able to accomplish something were gradually promoted to management. HArdcore developers who really loved coding were resisting that but there was a pressure definitely. And it makes sense managers are better managers if they know more about the tasks they are managing.

Now every developer is getting promoted to management because they are expected to manage the AI-agents. But their status in the organization nor pay does not really increase does it when every coder is doing that.

Sounds about right. Those people seem to want to create the impression that they are "AI Power Users". That gives them more power inside the organization. People come to them to ask for advice. Also if their output is not good they can claim that is because the AI budget didn't allow them to do more.

It's more power to power-users. And more dumbness for dumbos

Consider also Chiness EV prices. Unfortunately you can't buy them in the US it seems. https://www.reuters.com/business/autos-transportation/averag...

Note it is easy to confuse "free" with "paid by advertisements".

Motive probabaly doesn't matter in the end, outcome does. But understanding the motive is a good thing.

Maybe the financiers of a project just need it, they need it working, not to generate revenue for them?

I'm thinking using a local http-server instead of web-workers . The local http server would do all the server-logic except also passing data that needs to be shared to a non-local server.

What if we had a local server running on the same PC, which then relays the request to some shared server on the internet?

True, but that is a great fact to start from, and understand.

Then the next question becomes "HOW do they predict the next token?" There are many ways that can be done, why is this particular algorithm so GOOD?"

When people say "We don't understand how LLM works" isn't it really saying we don't understand how this specific algorithm used to predict the next token works? No, it is not, because "we" do understand how all those algorithms work there are many descriptions of them available.

So the question then really is "Why is the prediction this algorithm makes, so good, as compared to some other statistical algorithms?"

It's not about "Why does AI work so well?". It should be "Why does this particular XYZ algorithm work so well?"