Hacker News Reader

Comment by reb | Hacker News Reader

Another skill that'll be pretty valuable in this era is one many of the best technical leaders possess: the ability to coach desirable outcomes from squishy autonomous black boxes over time.

Getting excellent, exact performance out of deterministic systems is an impressive feat, but autonomy means variability, and getting excellent performance out of variable systems (especially illegible ones) is a different game.

View on HN

I agree 100%. I first had this thought last night when reading “Eight Things to Know about LLMs” (select quote from section 8 below)[0]

> Simply prompting a model to “think step by step” can lead it to perform well on entire categories of math and reasoning problems that it would otherwise fail on (Kojima et al., 2022). Similarly, even observing that an LLM consistently fails at some task is far from sufficient evidence that no other LLM can do that task (Bowman, 2022).

The eerie thing to me is that this is coaching. I have never coached anything that isn’t alive by any definition. My feelings about this realization are ambivalent.

[0]: https://cims.nyu.edu/~sbowman/eightthings.pdf

Doesn't prompting "think step by step" simply increase the probability the model will generate its output in the style of places where it saw "step by step" before (eg: tutorial content)? It's not really "deciding" to think step by step. Maybe I am mistaken.

You're not mistaken, its basically perturbing the seed for which the sentence grows.

It's more than just that. The intermediary steps act as a kind of working memory, ensuring that the model is less likely to get lost in the weeds.

Reply on news.ycombinator.com