Hacker News Reader

Hacker News

Comment by AverageDude | Hacker News Reader

Parent Full thread

AverageDude

Yup. I always wonder why is my experience not like others? Are those PR people for Microsoft?

Example:

I gave chatgpt a list. Which looked like

st street

av avenue

Convert this to yaml format as

  st:
    name:
      street

And so on.

It failed spectacularly. Not even once but about 10 times. Even if it succeeded, it kept changing the output by doing ops which I never mentioned in the prompt (like reordering and merging duplicated values to a single key)

View on HN

> Even if it succeeded, it kept changing the output by doing ops which I never mentioned in the prompt (like reordering and merging duplicated values to a single key)

That's something I don't see mentioned enough; if you change the input to a LLM, that may potentially change the probabilities of all the output tokens. Most of us would be surprised if we told a junior developer to fix a bug in a specific module, and they submitted a PR which modified literally every file in the source tree, but that's entirely plausible with a LLM. Asking it to "fix" one thing may change/break completely unrelated things.

I have human intelligence and I can't figure out what the output you're hoping for here is.

Update: I tried that with GPT4 and got this:

    st:
      name: street
    av:
      name: avenue

GPT 3.5 didn't know what to do with it.

It’s a yaml key value format using a map. And even if it nailed the format, it kept changing the output on it’s own. I mentioned not to change order or remove duplicates, it kept doing that anyway. I gave it 100 elements, it kept giving me around 80. And yeah, it was GPT 3.5.

Reply on news.ycombinator.com