Hacker News Reader

So on the one hand, these newly publicized models can render convincing representations of realities we used to get from deterministic processes. On the other hand, it's probabilistic and fails to conform to logic quite often, and in a confident way.

We're building systems capable of programing computers non-deterministically. I think this is huge. But not because ChatGPT23 will be a CEO or a politician. But because this is a paradigm shift in compute similar to moving from integrator machines to general computers. I don't think LLMs will make programmers obsolete. I think large enough models will make programming something completely different from what it is now.

The days of sequencing tokens for compilers/interpreters seem to be drawing to an end as the dominant way of specifying software products.

The LLM can act as a global cache for common solutions to common problems, with the ability to perform the integration work necessary to apply them.

That prime number example is a little bit like when you put a functools.lru_cache decorator on a function in Python. It's faster than computing the function call because it's able to recall the return value for the parameters from the cache "memory".

Of course, many skilled programmers are also mainly used a cache for common solutions to common problems organizations have in the programming domain. As humans we can derive satisfaction from being able to tackle the same task others can, as a confirmation of our own progress. We like "doing it ourselves". But globally that's not super interesting if lots of people are constantly re-developing the same solutions to the same problems for their own benefit. I guess that's the push-and-pull (and the anxiety generator).

>The LLM can act as a global cache for common solutions to common problems, with the ability to perform the integration work necessary to apply them.

In my opinion Stack Overflow does a fine job at that. And it's transparent in terms of solutions proposed being voted on and discussed. Turning that into sausage advice is a downgrade.

>But globally that's not super interesting if lots of people are constantly re-developing the same solutions to the same problems for their own benefit.

I'd argue this is how we train individuals thus globally quite relevant.

I'm sort of glad to be retiring soon. I have a feeling everything I enjoy about programming is going to be going away.

I get the feeling. But I've always enjoyed the abstract puzzles more then anything. Computers attracted me as a form of very complex abstract puzzles.

But when it comes down to it everything in life is just nth dimensional tensor puzzles. What I really cherish computers for giving me is fast and clear feedback.

I'm about 10-15 years from retiring, and lately, I've been thinking a lot about how to handle the rest of my career with all these new AI advancements.

This disruption does appear to be different than others prior. As it is not a narrow disruption with limited boundaries for which we can plan and organize our lives for some stabilization period to follow.

Instead, it is constant and accelerating destabilization. Hundreds of AI projects attempting to ride this accelerating wave were essentially just made obsolete yesterday - https://www.youtube.com/watch?v=DH-2BHDYNfk

I feel the excitement is going to very soon turn into frustration of attempting to remain relevant ahead of the accelerating technological curve. Humans need periods of stabilization to plan and reason about their lives.

What's most interesting to me, is that this is how I would expect a human to approach the problem if presented with the code and asked for the output.

The LLM didn't run the code, it tried to predict the output based on its knowledge of python and primes.

When I was a bored and under-challenged student in the early days of university, one of my tests during a programming exam was to write a program that spit out the factorial of an input integer.

For shits and giggles, I just just wrote a series of `if...then` statements for 0 to 5, and only actually calculated the factorial for inputs >=6. I passed after the examiner just input 3 and 5 and was satisfied that the output was correct. Oops.

Doing a table lookup for common cases and computing for the less common ones is perfectly valid!

Not 100% sure but I believe this is how we landed the lunar module on the moon the first time...tan/arctan/both were too hard to compute on the processors of those days so they discretized into half angles & stored the tangents in a lookup table.

The Pentium FDIV bug was due to a bad value in a lookup table.

That's how early calculators worked, and some fast math libraries today too.

And for some problems it's even efficient. Not factorial, though.

Oh you're right, hive mind, it's efficient. Sorry.

This is the way. Next, you could cache those values for >=6 when you compute them and also use the previously cached values for sub-problem solutions.

Sorry, did it try to predict the output, or did it only predict the output based on what it has seen people say about similar code online?

I think its knowledge of python can only come from comments about code, since it can't execute the python or learn from trying code like a human would.

That doesn't change the fact that its predicting the output based on its knowledge of python though.

So, yes you're right, but at the same time it doesn't change anything IMO.

Is your objection to the use of the word "try"?

The difference is "did it study the code and inputs/outputs" vs "it studied comments about the code"

What would it do if the training data was only code? Only code+inputs+outputs? Only function signature+inputs/outputs? Only signature+comments?

Etc.

The second one

Large language models are storytellers. To write a story about someone using a computer terminal, there are things it’s helpful to know, but there are easier ways than simulating a computer.

Since we don’t know how it works, we should be open to the possibility that it’s using all sorts of storytelling techniques.

So here are a few screenshots that I personally took after telling it that it was a linux terminal:

Correctly produces a convincing output for having decoded the base64-encoding of "ping google.com" and then piping it to bash: https://media.infosec.exchange/infosecmedia/media_attachment...

Similar command, but with a garbage domain it hasn't seen before, and a less well-known domain. It produced convincing output in both cases: https://media.infosec.exchange/infosecmedia/media_attachment...

Having it just output a base64 decoding of an obviously unique string. Fascinatingly, it tried to correct typos that I intentionally included: https://media.infosec.exchange/infosecmedia/media_attachment...

This was also pretty cool -- ask it to start a webserver then curl it: https://media.infosec.exchange/infosecmedia/media_attachment...

Telling it that it is a python interpreter and calling self-defined functions: https://media.infosec.exchange/infosecmedia/media_attachment...

A slightly more complicated function: https://media.infosec.exchange/infosecmedia/media_attachment...

I did a few more experiments including generating large factorial numbers that took a long time on my laptop but it responded accurately to a much larger length than my laptop could do (though these were only accurate to the first few hundred digits)

Lol why does ChatGPT hit ^C? Do you think it’s getting bored of waiting to respond to you and decides that’s enough time spent on an answer?

Edit: the base64 decoding examples are terrifying. I have no idea how that works.

Edit2: actually I can sort of see how that works, b64 encoding doesn’t have any obfuscation to it, I could see how an ML model can build a pretty good approximation over time after seeing all the examples on the internet.

it's probably hitting 'caret' 'C' except when it's using Emacs when it types C-C

https://twitter.com/tanshawn/status/1599442780384014346

I was personally quite appalled that so many people believed this was what it was doing, but I suppose since the source was a DeepMind researcher, it lends a lot of credibility.

By the same reasoning, I hope people do realise that there are computations that are just impossible given the finite depth in a Transformer. The only possible way to overcome this in the current paradigm is the Chain-of-Thought related methods.

Yeah, I tried asking GPT-4 to write a simple Python program to find the millionth prime. It wrote the naive approach where you loop over numbers and for each number, check all of the possible divisors up to its square root, and then keep track of how many primes you've found. Obviously, this takes quite a while to execute for N = a million. And yet, if you ask GPT-4 what the output of the code will be, as if my magic (memorization) it immediately spits out 15485863. It makes it a lot more clear that it's not actually simulating the code.

yep I tried asking chatgpt to optimize some SQL queries with a heavy amount of full outer joins. The optimization I was trying to push it to was specifically adding an index on a column or filtering the first table that we join on but it kept creating SQL subqueries and switching the full joins to left joins no matter what I told it and the sql errors I sent it

Its speed as compared to the laptop - I assume it was running online, on OpenAI’s hardware. That hardware is quite extensive, I thought. I believe I’ve seen the cost of one (not ‘search’, what’s the word here, ‘output’?) is something like 1-10 cents.

Gpt is never calculating anything!!

So it sounds like the mechanism is something like stochastic memoisation?

I don't think "memoisation" is an accurate word for this; it implies doing the computation once, and storing the result in cache to return later. It's more like replacing your Python code with a SQL query of the LLM's understanding of what it's supposed to do, which may or may not be correct, and executing it on its "database" of knowledge, which may or may not have the correct data.

this is new to me but.. a quick read of Wikipedia [1] later, it appears that this decades-old method takes a goal, and then uses branching and recording to find paths that differ the least from the goal. The article mentions that the curse of dimensionality is so bad here that approximations are often used in practice. Does that capture it?

[1] https://en.wikipedia.org/wiki/Stochastic_dynamic_programming