I built a plane spotter for my son in minutes

At least this person does not fetch all planes and then iterate over all of them but sets boundaries in which area to get all the planes.

It's saved me quite a bit of time in terms of understanding how to use APIs as a client. I find the API docs that Apple provides are so bare bones and don't illustrate how the APIs are meant to be used, and you really have to search around for example projects or That One WWDC Video Where All Is Revealed. Asking ChatGPT first saves a bunch of time having to do all that searching.

It knew the data model and methods that existed in code, and used these.

Just builds a local vectordb with your code base

Currently working on an Elixir project with Ash Framework, and I wouldn't trust a single Chat GPT output for either of them.

Greenfield? Unless we disagree on the definition of greenfield (no prior work), then a plane spotter app is anything but greenfield. GitHub returns 11 repos for JavaScript alone.

The wider context window and better tooling are all being worked on.

similarly, what about the vast number of times a prompt like this this _will_ have worked for other people?

we don't know either of these two numbers, especially not which one is larger than the other

I checked the city I live in and three more cities I know quite well, and I have to disagree. This reads/feels like a content farm, which (if GPT wrote the content, as you seem to imply) is exactly what it is (as I have excessive Adblock I don't know if there are ads, but I don't think they are needed for the definition of a content farm).

When you pay, the results come much quicker and are far less likely to timeout.

https://www.ted.com/talks/sal_khan_how_ai_could_save_not_des...

- https://smaller.fish/posts/l_plus

- https://www.kdnuggets.com/2023/04/datalang-new-programming-l...

- https://github.com/forrestchang/gptlang

- https://6502.is-a.dev/posts/aoc-2022/

However, there's many reasons it would be good at trivial tasks, and arguably most programmers spend most of their time on said trivial tasks, its just the count of them that's overwhelming; switching between multiple different languages, looking up syntax (at least for me!), and deciding on what's a sensical solution for the use case. Nobody cares about the performance characteristics and just wants it to work.

So anyway, I don't think a blog post about how you build a new algorithm to come out of any of the LLMs of today, but that doesn't mean I find them underwhelming.

At least that's how I use it and I'm super happy with what it gives me.

‘Meh. Let me know when your cat can recite T.S. Eliot’

While its knowledge is impressive, it continues to spout nonsense fairly easily.

You can write programs in markdown, it's a lot like literate programming that Knuth envisioned. I've been using this technique a lot for side projects lately --> https://swizec.com/blog/programming-in-markdown/

ChatGPT is able to analyze JSON dumps of dependency graphs and identify communities of files that should be modules. This sounds promising and I'm figuring out how to make it more practical for real use --> https://swizec.com/blog/finding-modules-in-a-big-ball-of-mud...

And while I haven't written about it yet, I've been able to get GPT-4 to analyze a blogpost and suggest phrases you should link to your past articles. Needs a little more tweaking with the prompt before prime-time because right now the result, while good, lacks some zhuzh. Whole thing was cobbled together using the markdown programming technique I mentioned in the beginning.

I'm at the point now where I'm so tempted to turn markdown programming into an executable script that can save me some copypasting and perhaps where I can say "Inject other markdown here" (basically functions), or "Go look up documentation at <link>" (that's been a painpoint).

So yeah there are dozens of us spending time with these things. Dozens!

Speeding up trivial tasks like this is game changing for many things. The amount of "trivial" things I've built with chatgpt that would've just stayed as an idea in my notes is crazy already.

But even if its expertise mainly opens the door to trivial stuff, that alone is a huge boon for accessibility for a lot of folks.

> I spent a month with an LLM and build a new programming language

For the past few months I've more or less been doing this. I'm self-taught, so my CS knowledge is sketchy and has lots of holes. I never took a compilers or programming languages course. I've done lots of reading but it never fully clicked for me. Having an LLM outline the process for me at a high level, provide pseudocode or even code snippets for less clear bits, and also (try to) answer whatever questions I throw at it related to my external reading has been pretty incredible -- things suddenly clicked. Another way I use it is to ask for important introductory texts or papers when exploring a new area

Yes, occasionally it spouts nonsense -- you have to evaluate its responses critically. If you ask it to do anything complicated or involving fast-moving APIs or less well known languages or frameworks, it will straight up make shit up. But we shouldn't throw the baby out with the bathwater here

A snippet of a conversation from earlier this summer:

Me

  In Pierce's types and programming languages, he describes the untyped arithmetic expression language. 
  In one of the definitions by inference rules, he defines the language as the minimal set T such that
  
    - 0 is in T, true is in T, and false is in T
    - If t is in T, then succ(t) in T
    - pred(t) is in T, and iszero(t) is in T.
  
  That makes sense if t is numeric but how are these defined if t is Boolean? 
  For example, true is in T, but I don't see how this definition can work

  if T is to also allow boolean values, like how is succ(true) well-defined?

  In Pierce's presentation in "Types and Programming Languages", the untyped arithmetic language allows you to form terms like succ(true), even though they don't have a meaningful interpretation in "everyday arithmetic". When we say the term succ(t) is in T, we're referring to the syntactic formation of terms, not their semantic meaning.
  
  The language defines the syntax of valid terms, but it doesn't give them any inherent meaning at this stage. This is why we can syntactically form terms that don't make sense semantically. When we actually evaluate these terms (if we were to interpret them), many of them would result in "stuck" states, where we can't proceed with evaluation. For example, while we can form the term succ(true), there's no evaluation rule that tells us what this term evaluates to.
  
  The definition you cited doesn't discriminate based on the nature of t. All it says is: if t is a term in T, then succ(t), pred(t), and iszero(t) are also terms in T.
  
  Remember, this untyped language serves as a stepping stone. It illustrates what can go wrong without types. The oddities you're pointing out motivate the introduction of more refined, typed systems later in the book, where such nonsensical combinations are explicitly prohibited by the type system.

Okay, I'll bite. I think quite often solving difficult tasks contain lots of trivial tasks. And speeding the trivial tasks can speed the difficult task up considerably when you can outsource the trivial tasks to an LLM.

I do not know how to make a catchy short video on my last side project scratching my itch (VR environment/metaverse to build 4 dimensional objects and move around them in 4d, see https://www.brainpaingames.com if interested). It was multi-year thing to get it where it is now (not pretty yet, I admit...), and it may of course just me being dumb, but yes, I consider it was/is difficult. Especially given I have never done anything in either C# or Unity. The speed-up ChatGPT gave to this project during last half a year was absolutely not trivial. I guess I must have saved years of my limited side-project time to get this far.

The whole AI hype reminds me of the ancient Greeks going to the Oracle at Delphi to obtain answers.

From the perspective of an outerspace alien, what is the difference between asking a god like figure for answers or an AI is some data center?

I've spent more than an hour with the tools, ChatGPT and SD being my two big buckets and it's great for productivity within some areas, mostly in fast mockup/art and creative writing, mostly fiction level.

I've use it to format and transform my own work, and as a tool it's useful.

Like any of these tools, early adopters get an edge for a bit and then the market adoption rolls out and we equalize.

I see it mostly filling in the 80% of the easy stuff, with humans stepping in for that last 20%, which has been my approach with good success.

However the hype train around it has gone past annoying and into downright scammish/spamish marketing.

2. Automating away well-defined trivial tasks frees up more time for figuring out the difficult tasks.

3. If you're running solo, being able to ask for alternative perspectives or to validate ideas - even if that's against a compressed archive of Reddit answers - is still valuable.

4. How is this better or worse than making progress through asking questions on X/Reddit/mailing lists/IRC/Usenet/your local library? I'll tell you: it doesn't irritate other people as much, and it's likely a lot more efficient. I get it, "I spent 20 minutes with an LLM and made a one-page HTML website!" doesn't sound impressive, until you compare it - about a year ago - "I spent two days going through awful ad-laden tutorials and made a one-page HTML website!".

The big cognitive gap people might be running into - and I think you might as well - is that people think these LLM tools are there to replace people and to do things that weren't possible before. They specifically aren't. They're tools to do stuff we know how to to do, but a bit faster.

Nobody is making explicit claims of magic. Nobody - other than breathless media - thinks this will replace people doing the jobs they do today. It's just going to make it a bit faster and quicker to get some stuff done.

They're better screwdrivers, not replacement geniuses.

May I ask, have you tried to do something difficult with it? Have you concluded it's impossible, or are you guessing that it isn't due to a lack of signal? I think it might actually be impossible, but I still think they're valuable tools for the reasons above.

And that's before you get into all the issues of suddenly steering the ai after a long game into providing scoring reliably, sometimes it confuses the scenario from the game proper, or hallucinates actions the user never did.

I had the prototype coded in a evening. But the rest has been a month of banging rocks to get it to work reliably start to end (well I've yet to solve scoring, but still, the rest works.)

LLMs are incredible. They really are. They can think fuzzily about human concepts, and interface those fuzzy concepts with cold hard computer things like programming languages and databases. But we need a whole bunch more ancillary technology before they'll really start shifting the social needle.

I have near-zero web dev experience before this project. After about a month, I started to rely on it less and less. But I still use it to debug when I hit the wall.

Already, I use ChatGPT almost daily to help me spot bugs. Doesn’t do a perfect job but it is at least 10x smarter than the rubber duck on my desk.

I used it to port 7 modules python code into Node/JavaScript. I’m familiar with JS. It did a reasonably good job.

And I use it all the time to prototype. Four weeks ago, I knew nothing about chrome extension development. Three weeks ago I released an extension on the chrome store. It’s not that ChatGPT wrote the extension… far from it. But it got me past the initial knowledge gap pretty quickly so that I could focus on the parts of code that I already know how to do.

And so you see, that’s why you and I don’t have youtube channels, and stock images of us making comically exaggerated gestures. Just so that we have a thumbnail template for:

“Is Elon actually right? LLaMama autogpt 5 about to go supernova?”

We knew this was a goldrush. At least it’s still pretty funny from a Carlin-esque perspective.

Fifty years later I was directing construction of systems of systems that could remove a tumour from a little girl's brain by taking 2D X-Ray images, reconstructing them as 3D density maps, identifying the organs in those maps, identifying the tumour in those maps, computing the best way to align proton beams in order to irradiate just the tumours, and then position the patient and beams, operate the accelerator to generate the protons and zap the tumour. It involved, literally, billions of logic statements - every one of which was trivial in its own context. Spacetime is also flat, everywhere, at the right scale. But we could orchestrate those billions of mathematical and logical trivialities into something very nontrivial because of huge productivity gains in producing them and stringing them together during my lifetime. A 30X productivity gain in producing even trivial (and the example give was not trivial) elements, is part of the path to radical innovation.

Anyone who tells me the latter: nope, it's as good as a university student in an industrial placement/internship.

Anyone who tells me the former: nope, the algorithm definitely had to learn a lot to be able to produce even a half-baked mess, and even 3.5 clearly knows more than I do about almost every subject.

So…

> it's fundamentally limited in what it can produce,

Indeed

> and the massive productivity benefits you see (30x faster!) are for trivial tasks, not difficult tasks.

For many people, programming at all is difficult.

Even for us, switching to a completely different language and API isn't a matter of moments.

Don't get me wrong, it absolutely screws up; but at 80% right-first-time (in my experience) going up to 90% just from giving it back the error messages from the compiler or interpreter, that's better than most students and much better than coming from a different domain.

The same in reverse means you should never ever rely on it for anything critical, even though it can still save time if you can perform fast reliable tests on the output.

I suspect something interesting is going to happen the price point of tailored individual LLMs running full time drops below the threshold where modeling an existing business unit’s org chart as a collection of interacting LLMs becomes viable.

Can you have a Director of Engineering LLM managing a pool of project manager LLMs and platform engineering LLMs and system administrator LLMs and infosec LLMs and software engineering LLMs and frontend engineer LLMs etc.

Can all of those LLMs create organizational plans, give each other guidance, self correct each other, give each other feedback, and work towards a solution?

For the low end of the market, can you create a software engineering organization with one well compensated Lead Engineer/CTO managing a huge collection of LLMs as a business unit? Or will the error rate compound making the ultimate output of the business unit just noise?

I can't see the full details of the states/all API call in the code screenshot but it appears to be iterating through the entire dataset of Open Sky Network flights and for each one determining if it is within the observable radius.

That's a lot of work and inefficiency and could hit the API rate limits.

A more efficient approach would be to call the states/all API passing a bounding box ( supported by the API ) and then just eliminating any aircraft that fall within the disjunction of the bounding box and the observable radius ( the corners ).

The more magic is involved in getting everything set up, the less well you understand the system you just built, and the harder it will be to do what you need to after the initial scaffolding. Wiring up the initial boiler plate is tedious but gives you a complete picture of the system.

It's called "on Rails" for a reason.