Experimentally Inventing a Programming Language Using Chat GPT

Pretty amazing interaction. Every time I've tried ChatGPT I get two questions in, an error happens and then all subsequent requests are rejected and I'm told to slow down, even if I wait quite a bit.

Maybe it's just under high demand right now.

This is very scary human-like. I was a teaching assistant and asked students to fix issues in their code. They acknowledged these issues but submitted again the same erroneous code.

It’s very clever in how you can tell it that it was wrong and it admits this mistake. Has it been written in such a way that the operators instructions are given a higher weight compared to what it knows?

I’ve asked it to generate python code and then format it with black. It reproduced the original code and then what was supposed to be the formatted code, but it was identical.

When told of this it admitted it had made a mistake and this time correctly outputted the formatted code.

I actually got it to work through debugging its own code by running it and giving it error messages (which reference the line), or describing the thing that wasn't working. I didn't try asking it to add line numbers because I figured it has a tenuous understanding of the relationship between tokens as it is, but it seemed to understand what needed to be fixed from the error messages, the output (which was generated by log commands it added upon my prompting), and a description of the problem. See https://github.com/HartS/gpt-hacker-news-extension

In the situations where it gave me partial output, I copied the last line and said "continue from "...""

I think it may need a more explicit description other than "immutable" for how they want the primes calculation to work.