I asked ChatGPT to write the code to print "Hello, world " as a junior developer
hachyderm.io
hachyderm.io
https://gwern.net/doc/cs/2005-09-30-smith-whyihateframeworks...
What did happen was that the big framework gave you increasingly more entrypoints into how the application was constructed.
But there always was a way to just go with defaults if you didn't want to adjust anything (at least in Java-Land, which this one specifically calls out for this behavior)
Wherever a framework should give the developers the option to adjust things to such a degree is another question that wasn't raised by this post. It just claims that you needed the increasingly silly factories to construct your application, which isn't true, as they were always optional.
This is a sequence of prompts in a single context which is “pre biasing” the LLM to respond the way you want it to.
Many such “emergent behaviours” are from the asker, not the LLM.
When you look in the mirror, you only see yourself.
> In the final exercise of this experiment, I challenged ChatGPT to step back, assess the code's objectives, and propose a better solution. The attempt was unsuccessful at first, as it continued to tweak the existing program and preserve the "existing architecture." Only when I instructed it to envision starting from scratch did it offer a new solution, as follows:
Surprise.
Naive usage.
So to round-out the joke, ask it how a 'galaxy brain' would write it, and we should end up at console.log(message) again.
I’ll give ChatGPT a pass because the prompt was so contrived. I would be curious how it would respond if prompted with a more open-ended problem.
The problem is that you only get out of GPT what you ask of it. For example, it’s likely that more senior developers would write abstractions because they can simplify unit testing. Others might have dived into writing TypeScript types, e.g. meta-programming or generics.
The output ChatGPT actually gave is interesting but not inspiring or relevant. They didn’t even uselessly add Winston for logging. It’s like when you ask GPT for images of “more, even more, work harder, etc.” - you sorta get more of the same, not something new or innovative. You have to ask for what you want, it’s not that creative and doesn’t really understand yet.
{-# LANGUAGE ImportQualifiedPost #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE QuasiQuotes #-}
{-# LANGUAGE TemplateHaskell #-}
module Main where
import Foreign.C.Types
import Language.C.Inline qualified as C
C.include "<stdio.h>"
C.include "<stdlib.h>"
main :: IO ()
main = [C.block| void {
system("python -c 'print \"Hello, World!\"'");
} |]I followed it through to the accompanying blog post[1] and I found the examples for "we close at 6pm on Friday" interesting because none of them work. They fail the stated problem (they don't test for Friday) and the unstated problem (when do we re-open? the examples re-open at midnight). And of course, without the prompts I can only guess whether the fault was in the answer or the question.
If the complex version worked and the simple version was naïve - or vice versa - it'd make for a much more interesting conclusion.
[1] https://koenvangilst.nl/blog/keeping-code-complexity-in-chec...
Tried the same with Haskell and the Fibonacci series because there's a similar joke on the internet. It's similar but not as stereotypical.
Then you start to realize that everything needs to sit behind some abstraction.
Well from a very pedantic and technical perspective stdout can be tested but overall IO cannot be unit tested.
It can be tested via e2e or integration tests meaning you need external factors to verify a side effect occurred.
Because there's no underlying training data that a stochastic parrot would need to generate this output the gap by this lack of data is only bridged by something that can be described a high level word: "understanding".
chatGPT understood the request and delivered answers to the request. I'm not saying those answers are correct. That doesn't even matter. What matters is chatGPT gave a biased answer to that request that indicates understanding of several concepts including ranking in human social structures and complexity of code.
One could say that the answers are incorrect. chatGPT hallucinated those answers because staff engineers don't write code like that. These people don't get it. The act of the hallucination indicates "understanding" regardless.