swid

Some old TV remotes used to work this way. They were made by Zenith and are called Space Command remotes. Apparently they are the reason TV remotes are sometimes called clickers.

https://www.theverge.com/23810061/zenith-space-command-remot...

N+1 only makes sense for E=2. For each parent in the list, fetch the child’s data. If there were multiple children, or if the children had their own children fetched 1 at a time, it would no longer be N+1.

So I think 1+1 (or 1+N/batch size) is still correct.

I have used Dataloaders in the past in both Go and Typescript and it just about solves the issue automatically. Dataloader is a specific library that is copied for different languages that allows you to write code that looks like N+1, but it will group the N part into batches, so it becomes just 1+1. It’s kind of weird to explain, but it should be easy to understand once you start working with them.

It makes this story even more interesting to me that the line would even appear on a map that is drawn like that - it looks like a toddler drew it. Crazy.

This American Life just did a segment where the experts were very skeptical of both the theories presented in this article - that the first encounters near Spain were caused by a traumatic event of a matriarch, and also that orca pods would communicate with each other to spread the behavior.

The theory presented in that segment was the first orcas are sort of like rowdy teenagers participating in a fad, and the newer event near Scotland was something of a coincidence.

It’s a good segment regardless of what you think, and the first half of the episode was one of the better reports of ChatGPT I have heard, so also of interest to people here. The name of the episode is “Greetings, People of Earth.”

I’ve know mountain climbers, hang glider/paraglider pilots, scuba divers, sky divers, and more and we pretty much say that exact thing. I’ve been casual acquaintances with people who have died doing extreme sports.

Usually, the people in these sports say if the worst were to happen to them, they accept that risk and wouldn’t want their fate to stop other people from following their passions. That the feeling of being alive chasing these dreams fulfills a part of them, and they would be lost without that fulfillment.

We all do take risks for our own happiness, different people just want different things from life and will drawn that line in different places.

My guess is the five people on the sub would not want this incident to stop people from attempting to visit the Titanic.

The post details how yearly subscriptions is one of the major issues, not monthly ones. There is already an update to monthly prices from 1.99 to 5.99, but the people on the yearly subscription locked in 0.99 or something like that.

The 30 day window was not enough time to rectify that and would cost him 50k in the first month to cover the diff. The author suggested he needed at least 3 months to implement changes and switch at least some portion of yearly subscribers to a higher price.

It's used for fine tuning a pre-trained model. This takes an LLM that is already capable of emulating lots of different kinds of personalities, and narrows it down to act more like the examples. Since the heavy lifting has already been done, 15k examples of a chatbot following instructions they way you want has a significant effect.

Or, it’s a simulator capable of producing output that is plausible for any level of intelligence up to the capabilities of the simulator. It could be capable of producing text similar to anyone from 0-140 IQ depending on the prompt.

It will act “smarter” if the prompt indicates a smart person wrote the text.

The exponential divergence formula is true but says less than you think - the same math would be true for human output; we are likely to find a few falsehoods in any non fiction book for the same reason, right?

> Fundamentally, if humanity is given data set A, it will eventually generate dataset A' of 'higher quality' than A.

This same phenomenon happens with evolution, which is obviously not ‘intelligent’.

Each human can basically be randomly exploring a problem space near things they have seen and understand, but they have a way of knowing when they hit on something new and interesting.

Even a dumb AI could randomly try a bunch of things… as long as it doesn’t try the same thing twice and had a lot of time, who knows what it might come up with. Validating the solution is usually easier than finding it.

I noticed they bought four horses without enough money to do so. (It correctly said two characters would have to borrow 20 gold each, but only 10 gold remained in the party, leaving them 30 short).

It’s an interesting debate to bring up, but I think not really the same kind of prediction without explanation I am talking about.

The second post I linked turned out to be really interesting, it both aligns with my thoughts while also adding new ideas and concepts. It makes a distinction between GPT and the agents it simulates, the simulator and the simulacra.

A good enough simulator can simulate an entity capable of explaining lots of things, depending on the limits of the simulator.

> .. we can abstract, and formalise, and prove, and demonstrate. You can't do that just by predicting. That's a power over and beyond modelling data.

I think we will find this not to be the case. And of course, many people like me do think that the ability to predict well enough implies the ability to do everything else.

Since I wrote my reply, I found two other posts on HN that seem relevant to that if you want to read more:

AI vs. AGI vs. Consciousness vs. Super-intelligence vs. Agency https://secondbreakfast.co/ai-vs-agi-vs-consciousness-vs-sup...

Simulators (Self-supervised learning may create AGI or its foundation) https://generative.ink/posts/simulators/

Here is a quote from that last one - I am still reading it since it is long, but

> ... I call this the prediction orthogonality thesis: A model whose objective is prediction can simulate agents who optimize toward any objectives, with any degree of optimality (bounded above but not below by the model’s power).*

> This is a corollary of the classical orthogonality thesis, which states that agents can have any combination of intelligence level and goal, combined with the assumption that agents can in principle be predicted. A single predictive model may also predict multiple agents, either independently (e.g. in different conditions), or interacting in a multi-agent simulation. A more optimal predictor is not restricted to predicting more optimal agents: being smarter does not make you unable to predict stupid systems, nor things that aren’t agentic like the weather.

Thanks for the offer, but it feels to me like you are missing the point I would like to be making. Maybe we are all just stochastic parrots in a sense? (FWIW, I do basically understand the notation).

I think the universe might be computable like Stephen Wolfram says - if this is the case - in theory someone could eventually walk you through the math and series of computations to make and then say, "see - humans are not intelligent, they are just biological machines without free will. With unlimited time and computation resources, we can run a human on a Turing machine in a virtual universe."

Even if the universe is not fully computable to the point we can perfectly simulate it on a Turing machine, we already do a pretty good job simulating lots of things, and it's not clear the loss of fidelity would make much difference. As I said, "intelligence" could be an emergent property of certain highly complex networks. I don't think the substrate of the network matters (digital vs biological) - it's just certain states may come with a feeling for the network.

Being able explain the output of the system by math is very powerful - but it fails to capture emergent properties of the universe we would generally prefer to believe in like consciousness.

Anyway - we see slime molds act as path optimizers; lots of algorithms we find useful have physical analogies. I don't know how the brain learns, but biologically, there must be some kind of gradient to follow and reinforce some neural connections while reducing others. Math is the language of the universe, but we believe in free will. Explaining the math of LLMs will not bring us into an agreement - this is unfortunately philosophical I think.

It’s just overly reductive. LLMs do not work like Bayesian or Markov models that fail to scale. In order to predict what word a human will say next past a certain accuracy, you actually have to model more than text. You have to model that person’s behavior in a sense, or otherwise you can’t reach that accuracy (the tree example would not require that much - I used it since I had seen it before).

I like gwern’s[0] example of a dozen people jockeying for power at a dinner table. Part of the next token prediction includes predicting how these Machiavellian schemes play out and why. Next word prediction may give the model a license and reason to figure all that out.

[0]: https://gwern.net/scaling-hypothesis

> Why? Any large probabilistic model in your example would also predict "an" due to the high attention on apple.

I’m not ignoring how the tech works and this is a simple example. But that doesn’t preclude emergent behavior beyond the statistics.

Did you catch the GPT Othello paper where researchers show, from a transcript of moves, the model learned to model the board state to make its next move? [0]

I’m beginning to think it is reasonable to think of human speech (behavior will come) as a function which these machines are attempting to match. In order to make the best statistically likely response, it should have a model of how different humans speak.

I know GPT is not human, but I also don’t know what form intelligence comes in. I am mostly certain you won’t figure out why we are conscious from studying physics and biochemistry (or equivalently the algorithm of an AI, if we had one). I also believe where ever we find intelligence in the universe, we will find some kind of complex network at its core - and I’m doubtful studying that network we will tell us if that network is “intelligent” or “conscious” in a a scientific way - but perhaps we’d say something about it like - “it has a high attention on apple”.

[0] https://thegradient.pub/othello/

I’m sure you don’t think it follows a human is a glorified rock? Both objects are processed by the universe under the same rules, and understanding how consciousness arises directly from physics is likely intractable.

One reason I hate the “glorified word predictor” phrase, is that predicting the next word involves considering what will come well after that.

I saw a research paper where they tested a LLM to predict the word “a” vs “an”. In order to do that, it seems like you need to consider at least 1 word past the next token.

The best test for this was: I climbed the pear tree and picked a pear. I climbed the apple tree and picked …

That’s a simple example, but the other day, I used ChatGPT to refactor a 2000 word talk to 1000 words and a more engaging voice. I asked for it to make both 500 and 1000 word versions, and it felt to me like it was adhering to the length to determine pacing and delivery of material that signaled it was planning ahead about how much content each fact required.

I cannot rectify this with people saying it only looks one word ahead. One word must come next, but to do a good job modeling what that word will be, wouldn’t you need to consider further ahead than that?

Thanks for the correction, guess I did not read that post close enough.

Even if it is free, I have no interest in playing chess against a superhuman chess bot. You don’t even have to know how to play chess to use the moves the bots recommend and win against a grandmaster.

The line is blurry today, but we are moving into territory where humans will not be able to solve programming challenges that require under 200 lines of code faster than AI - we are slower to read and type. The AIs will likely get better at understanding the problems, requiring less help from humans and fewer attempts to find a solution.

At some point using a language model to compete in these kinds of programming contests will absolutely be like using a poker or chess bot to compete in those games.

Some posts yesterday showed that network is usually disabled, but with the right prompt, you can enable network and someone got it to like a Twitter post.

The foundation of cryptography is that there exists algorithms in which the only way the beat them is to iterate over a computationally infeasible number of states to be able to decrypt something.

To not believe my assumption would require you to believe that such algorithms do not exist.

There is no proof of that for our crypto systems. There is a proof you can’t prove ZF theory, but you also probably can’t prove it false. If you hide that question in the thing you iterate over, you can take it on faith you won’t find a counterexample and say the program won’t terminate; but you haven’t, and in fact cannot, prove it.

There are also lesser problems that have been around a while which we haven’t been able to prove, but might be provable… you can iterate over numbers looking for an counter example to the Collatz conjecture… I think that’s the best example of a simple program that is outside our current knowledge of math (better than the crypto one), but maybe someday we will have an answer.

Because some problems we lack the ability to solve any other way. There are at least three problems which are famous, as described in the article I linked.

see if you follow this one one, for a real computer:

Suppose I pick a n bit key, and encrypt a phrase with it. I will provide you the plain text phrase and either a real or fake cipher text. You have to determine if a program which loops through all the keys will decrypt the ciphertext into a the phrase.

Either you know weaknesses in the crypto system, or you have no choice but to iterate though each key to see if the output matches the phrase.

There is a relationship with proof of work crypto as well. Even all the Bitcoin mining has trouble finding hashes with a certain number of leading zeroes, add a few more zeros and no one will ever find a working seed again. There is no way to speed up the process beyond iterating over the seeds.

You can tie this to the halting problem by just saying to go back the key 0 once the key space is exhausted.

Both people are me. I think there is an answer in that article, but I can’t unpack it for you. It explains why what you are asking about is so impractical, and can point you to topics to gain a better understanding.

In the real world, a program will either halt, loop, or crash, but this doesn’t help us if a 6 instruction program can take longer than the age of the universe to run while still terminating. No matter how fast you go through its states. 64mb leaves a lot of room for unique states without a cycle.

You asked if we could solve if it overflows a computer efficiently (without overflowing it), so I am showing that is not possible generally due to limitations of math; both our current understanding but also because math also has limits. (Gödel’s incompleteness theorem.)

Attempting to overflow it is impractical using cpu cache memory sizes, no hard drive needed. No matter how fast you go through states, there are too many possible states in the cpu, and too many states even a small program can iterate through while still eventually terminating.

Here is a good article you might be able to understand that pretty much provides an answer to your question.

https://www.quantamagazine.org/the-busy-beaver-game-illumina...

The busy beaver problem show how many states a program can have given a certain number of Turing instructions. We don’t actually know the value of that number past five I think, but it already shows us brute force is out of the question.

Could we solve it without brute force? Well, there are plenty of math problems we can’t prove; we can write a program that only halts if counter example is found, and we would be unable to prove if the program halts. We can also write program that will halt if it can prove ZF set theory is false, but we already know it’s impossible to prove that (it only possible to prove it is false). So symbolically deciding if a program halts is also impossible in the general case.

If there is a fence, I think it is because of quantum decoherence. More and more qbits leads to a possibly exponentially greater chance for decoherence, and the result of the quantum computation is invalid. The program will need to be run many times to get the valid result.

What is the lower limit to battle decoherence? Can we push the number to zero, or will there be a limit?