Hacker News Reader

Comment by lordnacho | Hacker News Reader

You learn a lot from ChatGPT though. If you want to know the ordinary answer to some essay question, cgpt will give you that. If you leave it at that and hand it in, maybe Chomsky is right.

But it's also a essay writing productivity tool. You sketch out what you want mentioned and cgpt will fill in the gaps, like those apprentices that helped the renaissance masters. I keep coming back to the phrase "small intentions" when I discuss cgpt: both in essays and code, it is already quite good at filling in details.

With less time spent on details, students can spend more time on the architecture. Ultimately, we'll have more people able to practice higher level creativity. This is already a thing that you do in math class; after your primary school years, you get a calculator to work out the arithmetic.

View on HN

The problem with using ChatGPT for "filling in details" is how often it is confidently, utterly wrong.

My friend tried to use it to write a legal letter and it completely invented a state law and cited it very specifically.

For him, it was annoying and amusing. For other people, mistakes like that could be very serious.

> after your primary school years, you get a calculator to work out the arithmetic

Do we? I do a lot of mental arithmetic as a software developer. When I was an exchange student my host-dad was an engineer and he was incredibly good at it. As he was thinking through a problem he'd do all sorts of ballpark math to get a feel for things. Eventually, yes, the numbers would all get crunched properly. But even then he'd also be doing the math mentally to make sure that something hadn't gotten screwed up. A person who uses a calculator for everything starting from a young age just doesn't develop that skill.

And I think there will be a similar problem with ChatGPT. Can an excellent writer get a performance boost starting with ChatGPT? The jury's out, but it's not impossible. But can one become an excellent writer by tarting up LLM-generated prose? I doubt it. Learning to write is a process of sweating the details. Especially so when the output is an essay. Real higher-level creativity requires foundational work. I think at best you'll get the sort of content that comes when an uninspiring politician hires a mediocre ghost writer.

That said, a lot of writing is bullshit. And I mean that both in the Frankfurt sense ("speech intended to persuade without regard for truth") [1] and in the Graeber sense ("completely pointless, unnecessary, or pernicious") [2]. And that's where I think ChatGPT will shine. Marketing copy to fill in the "lorem ispum" space on the product page? ChatGPT to the rescue. Long sections of TPS reports that nobody will read? ChatGPT again. And once they expand ChatGPT to being able to make PowerPoint decks, a zillion ineffective middle managers will be able to return their focus to Candy Crush, harassing the interns, and creating pointless meetings.

I think that's also bad, of course. Lowering the cost of bullshit production looks like a net societal negative to me. But it would be successful in the traditional tech industry metrics of "gains users" and "produces revenue".

[1] https://en.wikipedia.org/wiki/On_Bullshit

[2] https://en.wikipedia.org/wiki/Bullshit_Jobs

> A person who uses a calculator for everything starting from a young age just doesn't develop that skill.

But is that a reasonable fear? It's actually quite hard to use a tool without understanding it, so you won't get a lot of people reliant on calculators who don't understand calculation. I've certainly never met someone for whom it was a problem.

About using tools you don't understand, this is easy to try. Get some film editing software or 3D rendering software (assuming you don't know this area, like me) and see where you get to. I've never been able to use a tool without understanding at least something about what it does.

> It's actually quite hard to use a tool without understanding it

The whole point of many tools is to not understand what's going on. I had a friend in college who could happily drive a car, but had been discouraged from understanding how it worked, because opening the hood of a car was not ladylike. So whenever anything happened, she would be filled with anxiety and take it to a mechanic, even for perfectly mundane things like putting in more coolant.

Similarly, people who have a math aversion can lean on calculators to stay innumerate. I see this happen frequently when I pay for something in cash using change. E.g., the bill will be $9.77. I'll see that and think, "Oh, I want the quarter", so I'll put down a $10 and 2 pennies. Some people are fine with it, but a notable fraction of cashiers (whose whole job is handling money!) will look at me like I have two heads. But they'll dutifully punch it in and act like I'm a wizard when it comes out to be a single coin.

> Get some film editing software or 3D rendering software

The correct analogy here is to the word processor, not ChatGPT. If somebody who doesn't know how to write opens up Word and expects an essay to come out, they'd be similarly stymied. Or we could look at essay-writing services. A quick search turns up places that are for "learners seeking assistance". But I think we all know that nobody is really using those services for inspiration or a rough draft so that they can hone their essay-writing skills with the assistance of a professional.

I taught high school math, there's many who rely on calculators for even the most basic calculations because they can't do them and have no concept of what it means to, say, multiply two numbers or how to even go about doing it.

The model for mathematics is something like a “good book” of once-entered, unchanging posable questions and theorems. More may be added, but once added there is a sense of permanent definitiveness and relevance.

For your example and beyond-mathematics calculator analogy, maybe the dates of historical events are equally definitive, but I don’t see much beyond that being so. Science goes through transformations. Either the questions remain the same (what is there, what exists), but the answers to them change as science develops. Or, the questions change, and the answers don’t (what did plogiston, aether, or general relativity predict).

Your expanded “calculator” paradigm seems unfounded. It’s not analogous to math.

> Your expanded “calculator” paradigm seems unfounded.

Please explain. GPT makes a utility out something that previously was not. The GP didn't say it was exact.

They didn't say exact. Exact isn't the concept I'm concerned about.

There is something unique about mathematics in that its answers never change. And that static structure can be tapped into with calculators. A "calculator" outside this domain reeks of potential danger to me. Asking a question to some other field whose answers are open to revision doesn't give one transparent access to timeless knowledge. There is always going to be a caveat that the answers given by GPT are clouded by time and circumstance. You can't just copy the answers GPT gives you and call that knowledge, like calculator results do allow.

I would hope GPT knowledge comes with the understanding that GPT v.XX, exposed to Y circumstances, produced Z results. We don't do that for calculators.

Is the high level structure of writing any more important than the low level structure?

Maybe it will tend towards everyone sounding the same. Which then gets amplified as LLMs train on writing that is essentially mutations of itself.

Me no like.

> Is the high level structure of writing any more important than the low level structure?

Well you can make a point about the fall of Rome with badly cited sources or spelling mistakes. But if your essay says Rome fell due to conquest by the Han dynasty it can't be saved by any number of correctly spelled, beautifully cited details.

ChatGPT will let you write an essay making any point while filling in little things.

It sounds like you're advocating a system of education, thinking, and work that is exemplified by 'Software Architecture Astronauts', people who have not recently or ever contended with the details, the nuts and bolts, of actual software development and maintenance.

To be honest, the idea that you can produce valuable intellectual work by outsourcing the details is absurd. This would have seen me absolutely fail my challenging philosophy and engineering coursework, and end up a fundamentally incompetent dilettante in work and life.

Great points! But I want to quibble with this a bit.

> To be honest, the idea that you can produce valuable intellectual work by outsourcing the details is absurd.

It is absurd. But it's also very common! There are an awful lot of people who have done very well not by understanding anything, but by posturing and performing. E.g., take the ghostwritten book. The books of many politicians from all parties could generously be called mediocre. But they don't need to create valuable intellectual work; they just need something anodyne and book-shaped with their name on it.

Surprisingly often, people get away with this kind of absurdity, especially when rich and/or powerful. Indeed, one of the fascinating things for me about Musk's ongoing clown show at Twitter is that we're getting a real-time look at what happens when somebody who refuses to contend with the details fools themself into thinking they know what's going on.

That totally matches my experience where Architecture Astronauts have free reign. They are rewarded not for making this work, but talking impressively about big things. So what they produce is a lot of talk. When it's implemented by actual developers, does it work out well? Usually not! But that doesn't matter, because the people who empower them are also people who talk impressively about big things without understanding them.

Totally endorse the quibble. I think ChatGPT will be most often deployed in areas that are tied to a lot of economic value but are also full of bullshitters. Ghostwriting political memoirs is a good example, but its a niche, SEO spam is that big bucks at the moment.

Well I didn't say you should never concern yourself with the details.

But at some point in your life you will have gone through the details so often you won't be getting them wrong, but you'll still be spending a lot of time on them. With a tool like cgpt you can save yourself a lot of time.

Of course it's up to you where you take the trade. If your thing is a medical device that will kill someone if it isn't checked, maybe you carefully write the code yourself. If you're building some POC website, why not let the AI fill in a few API calls for you?

And don't forget, plenty of details are already outsourced. Entire reference books are full of details that we prefer not to keep memorized.

Plato:

“If men learn this, it will implant forgetfulness in their souls; they will cease to exercise memory because they rely on that which is written, calling things to remembrance no longer from within themselves, but by means of external marks. What you have discovered is a recipe not for memory, but for reminder. And it is no true wisdom that you offer your disciples, but only its semblance, for by telling them of many things without teaching them you will make them seem to know much, while for the most part they know nothing, and as men filled, not with wisdom, but with the conceit of wisdom, they will be a burden to their fellows.”

https://www.goodreads.com/quotes/259062-if-men-learn-this-it...

I'm advocating cgpt as a tool that allows you to spend more time at the level of abstraction that is appropriate for the problem. If you're writing an essay about the fall of Rome, you don't want to be spending loads of time finding the details of a reference (year published, secondary author's names, exact title...). You want to be talking about economics of the time, incentives for different groups in Roman society, and so on.

Right, it seems like you're advocating ChatGPT as an information retreival tool, which seems perfectly acceptable. Looking up exact dates and names is tedious.

By "filling in the details" I thought you were referring more generally to the essay writing process, where the structure of an essay is its skeleton and the "details" are the just as essential pieces of argumentation and evidence.

If you write a sociology, history, or philosophy essay, for example, writing the high level structure (intro, topic-sentence 1, topic-sentence 2, ..., conclusion) and then outsourcing the rest would be intellectually bankrupt. You can't understand produce a synthesis argument of Hume's The Wealth of Nations and Theory of Moral Sentiments with ChatGPT.

Is that bait? Adam Smith IIRC. Hume did Treatise on Human Nature.

Not bait, but it could have been! I switched my example halfway through because I couldn't think of a 2nd Hume book, but didn't properly edit.

Edit: Here's the first paragraph of what ChatGPT responds to my incorrect request for that synthesis argument. Thankfully we had you reply and not ChatGPT:

> David Hume's "The Wealth of Nations" and "The Theory of Moral Sentiments" are two of his most famous works that address important aspects of economics and ethics, respectively. While these two works seem to address separate topics, they are related and complement each other in many ways. ...

ChatGPT Prompt: produce a synthesis argument of Hume's The Wealth of Nations and Theory of Moral Sentiments

We can do both, ChatGPT can code. So the Astronauts should be able to produce working systems that pass functional tests. The majority of the detractors of these models don't see the forest for the trees. It only gives new capabilities, they remove nothing.

> So the Astronauts should be able to produce working systems that pass functional tests.

Nah. This is just a new version of an old mistake, one people have been repeating for decades. People keep trying to come up with ways for people to "code" without understanding what's going on. Code generation wizards. Visual programming tools. Model driven architecture. And a bunch more.

The hard part about making software isn't writing a bit of starter code. What we saw with the "code wizard" approach is that somebody clueless could click some buttons and get something working. But then they couldn't maintain it. It just kicked the "understand what's going on" problem down the road from "I don't know how to start" to "I am now trapped in a hell of generated code and people are yelling at me".

If some particular application is so standard that it can be produced with only a shallow understanding of how software works, then the right thing isn't using ChatGPT to produce source code. It's when people who have a deeper understanding produce an app that is configurable in the right ways.

My 8 year old asks you to look at from a different angle and evaluate what it can actually do right now.

I would add, just because someone made a prognostication decades ago does not invalidate the same prognostication now.

Have you tried it?

Those aren't decades-old "prognostications". They are things that happened.

Could it be different this time? Maybe! But if you want to say that it will, then you have to make the argument. Or maybe you can get ChatGPT to do it for you?

Have you tried coding with ChatGPT? It works now. You can use it now to architect a system AND produce working code.

Oh? Then it should be easy for you to point me to an example of working system of real-world complexity built this way, yes?

I'm especially interested to see how the system gets improved over time as users try it out, lessons are learned, needs change, etc, etc.

Please stop.

Stop asking you for you to back up your questionable claims? Now that you've stopped making them, sure, I'm glad to.

I have not, but others have detailed examples where it produces rubbish. Certainly not the kind of risk to take with safety critical systems like what an astronaut may rely on.

Exactly. Here's an excellent instance of it being confidently and ridiculously wrong: https://www.reddit.com/r/AnarchyChess/comments/10ydnbb/i_pla...

If it can't handle the relatively simple rules of chess, it sure can't handle coding.

> It only gives new capabilities, they remove nothing.

This is not how technology works in the world. See the arguments of for example McLuhan. Technologies are appendages for the human body or intellect, and at the same time they numb the part of that they amplify.

Also the point of the Architecture Astronauts idea is that they don't produce working systems. It's a specific pejorative, it doesn't refer generally to people that work at a high-level without harming the systems they engage with.

You clearly have a bone to pick with software architects, if anything it should reduce the amount of Space Travel for said architects.

McLuhan, a media analysis astronaut. Average is the message.

> Technologies are appendages for the human body or intellect

We previously had fleets of people calculating tables of numbers for hours per day. Simple calculators removed that need and increased the speed of results by orders of magnitude.

You seem to be arguing for handicap system, an intellectual conservatism. Is that correct?

What does ChatGPT remove from ones ability to write low level code or for a software architect to both architect a system AND produce a working model?

Reply on news.ycombinator.com