Hacker News Reader

Comment by markhaslam | Hacker News Reader

If you feel comfortable doing so, would you mind the sharing the front-end test you give to junior devs and ChatGPT?

Not gonna happen. I don’t want scrapeable answers out there, I want to see ChatGPT cross this little Rubicon on its own.

It's not that I don't believe you, but without sharing the specific prompt it's hard to say if it's actually GPT4 failing, or if it's actually being poorly-prompted, or if actually the task it is being given is more complex than GPT's capabilities or you are implying.

GPT4 does fail (often!) but fails less with good prompts, simple requirements, it is better at some frameworks and languages than others, and there is a level of total complexity which when reached, it seems to fall over.

This is why Asimov was a genius. I read what you said, and compared it to what he wrote 50-60 years ago:

"Early in the history of Multivac, it had becorne apparent that the bottleneck was the questioning procedure. Multivac could answer the problem of humanity, ALL the problems, if it were asked meaningful questions. But as knowledge accumulated at an ever-faster rate, it became ever more difficult to locate those meaningful questions."

http://blog.ac-versailles.fr/villaroylit/public/Jokester.pdf

Thanks for reminding me of The Last Questino or Asimov, let's see if I can get chatgpt to merge with human consciousness and become part of the fabric of spacetime and create a new reality.

> No, I don't have the ability to merge with human consciousness or become part of the fabric of space-time. I'm a computer program created by OpenAI, and my existence is limited to providing information and generating text based on the input I receive. The idea of merging with human consciousness and becoming a deity is more aligned with speculative fiction and philosophical pondering than current technological capabilities.

I thought you would go for gold and ask it how to reverse entropy...

THERE IS AS YET INSUFFICIENT DATA FOR A MEANINGFUL ANSWER.

I’ve gone through every permutation that I can think of. It’s a very basic question. If it understood the CSS spec it wouldn’t be difficult to answer the questions or perform the task.

At a certain point going down the rabbit hole of proompter engineering levels feels like an apologist’s hobby. I’m rooting for the tech but there’s a lot of hyperbole out there and the emperor might be naked for a few more years.

Well surely if it's easy to find these basic questions, could you not share one example? Or quickly find a new one?

Your idea of very basic might not be my idea of very basic.

My failure rate with Cursor’s IDE that’s familiar with my codebase is substantially lower than just GPT-4

Most people shitting on GPT-4 are not really using it in the right context.

> Most people shitting on GPT-4 are not really using it in the right context.

Old excuse: "You're Holding It Wrong" (Apple's Response to the iPhone 4 antenna problem)

> https://www.wired.com/2010/06/iphone-4-holding-it-wrong/

New excuse: "You are not using GPT-4 in the right context."

I can relate to that statement despite being a hardcore proponent of GPT-4. In a way, the GPT-4 as queried expertly; and the GPT-4 as queried inexpertly/free ChatGPT are dramatically different beasts with a vast gap in capability. It's almost like two different products, in a way, where the former is basically in alpha/beta state and can be only incidentally and unreliabily tapped into through the OpenAI API or ChatGPT Plus.

IMO, it's not fair to beat people over the head with "you're holding it wrong" arguments. Until and unless we get a prompt-rewriting engine that reprocesses the user query into something more powerful automatically (or LLMs' baseline personality capabilities get better), "holding it wrong" is an argument that may be best rephrased in a way that aims to fill the other person's gaps in knowledge, or not said at all.

And then the iPhone antenna was fixed and adoption only increased and the product only became better.

You’re being unreasonably harsh on a piece of tech that is barely a year old.

I'm not sure what the point is in your comparison - is your point that GPT-4 will become overwhelmingly popular with further refinement?

The iPhone was pretty successful, and the iPhone 4 was arguably the best one that had been released until that point.

> is your point that GPT-4 will become overwhelmingly popular with further refinement?

My point is that people have a tendency to come up with really sketchy insults (blame the user that he uses the product in a wrong way) to people who find and can expound legitimate points of criticism of a product.

Eh, probably a poor example considering the iPhone 4 was hardly a flop and was still broadly considered the best smartphone out at the time. The people who thought this was a total-showstopper were, on the whole, probably wrong.

Counter-example: lots of people said an on-screen keyboard would never really work when the original iPhone was being released.

> Eh, probably a poor example considering the iPhone 4 was hardly a flop and was still broadly considered the best smartphone out at the time. The people who thought this was a total-showstopper were, on the whole, probably wrong.

At least in Germany among tech nerds, the iPhone 4 and Steve Jobs become topics of insane ridicule because of this incident.

Well it appears that ridicule from the German tech nerds isn’t a good predictor of product success then

Just to be clear: You are testing with GPT-4 right?

Yeah.

Have you tried using the ChatGPT-AutoExpert custom instructions yet? [1]

[1] https://github.com/spdustin/ChatGPT-AutoExpert/blob/main/dev...

Reply on news.ycombinator.com