Hacker News Reader

Comment by charles_f | Hacker News Reader

There's a bit of snake oil in all of that.

By any means ML is a very specialized subfield, where you need solid math basis and a deep understanding of the science behind it all.

But I struggle to see the same thing for AI. If by "AI engineers" you mean someone who builds an LLM*, then it's very much just ML. If you mean someone who integrates with the LLM someone else built, then it's very much just backend work. Sure, you might need a few days to understand a few concepts, I've integrated with Paypal in the past and that doesn't make me a payment for engineer.

* wont even get into the argument of labelling LLMs with AI

View on HN

If you have React engineers, why couldn't you have AI engineers?

I think the post is right to try to delineate the roles of "builds LLMs or other ML models" and "uses those models", as (as you noted) the skills required are very different (vs. "creates a JS framework" vs "uses a JS framework" are the same skills, just on another level).

"AI engineer" is, as you say, inaccurate, but as swyx says, it's least cringy of the alternatives.

> Sure, you might need a few days to understand a few concepts, I've integrated with Paypal in the past and that doesn't make me a payment for engineer.

No, but the post explicitly calls out the breadth of knowledge that the future AI engineers will have to have. It's not something you can pick up in a few days, even now, if you want to be on top of things.

I can look up some docs and hook up static file hosting on S3 and that doesn't make an AWS engineer. But there are people who full time work on providing solutions using AWS-everything and have built their entire careers (and companies) on top of that specialization.

The difference in scale is the difference in job description.

> If you have React engineers, why couldn't you have AI engineers?

We most definitely could. That's one way to prop up a cottage industry, stay relevant, command hefty salaries, and maintain that charade of importance through complexity by creating Frankenstein features in search of problems and promotions.

like the canned messages that are increasingly finding themselves into any app where you can send messages to people, be it email, whatsapp, tinder, whatever

"Hey John! I see you like traveling, I love traveling too! What are some of the most memorable trips you've had?"

like, who would want to receive such an obviously canned message lol?

soon, it will just be the apps talking to each other, no humans required

And that's how you get black mirror - hang the DJ

Because AI as terminology is vague and can mean anything.

Graph search is AI, pathfinding is AI, decision trees are AI, fuzzy logic is AI, expert systems are AI, machine learning is AI.

Just say LLM analyst or whatever.

Yea i agree, maybe calling them LLM Engineers as against AI Engineers would go down better with folks though? ML enables LLM's that enable AI, So its definitely semantics

Maybe the term "engineer" did the concept a disservice, but prompt engineering has a lot of parallels to the field of UX.

Currently there's a lot intuition involved so it can come across as made up, but there are novel concepts which meaningfully affect the end quality of what you make, and it takes time to learn and/or discover them.

As time goes I expect our understanding of what underlies "good prompts" will start to bridge the gap from intuition to science much like how UX bridged into neuroscience and psychology. If you understand things like attention and logits that's already kind of happening: you can use that knowledge to identify gaps in the abilities of LLMs and start to bridge those gaps.

People are convinced that future LLMs will obsolete prompt engineering. To me that'd be like people from the 90s thinking computers are going to obsolete UX because more powerful computers will be better at making user interfaces, and in turn anyone will be able to do it.

In some ways they'd be right: Today you don't need a UX expert at PARC to integrate a WYSIWYG interface into your product. Computers got so powerful that in milliseconds we can download libraries that implement the interface and render it across any form factor you can imagine. So now a WYSIWYG on your contact form is nothing.

But as computers got more powerful they could do new things, so UX advanced onto improving how we interface with those new things. Things like the Vision Pro will unlock new areas of UX based on novel capabilities they posses.

I think people are making a similar mistake with LLMs: they're focused on this idea that we'll just do the current things but better with more powerful models. But the more powerful models will be something we can "prompt engineer" into usecases we haven't even considered yet. (I also built notionsmith.ai and I'd argue it fits into that bucket a bit)

> where you need solid math basis and a deep understanding of the science behind it all.

I __REALLY__ really wish this were true. But I'll be honest, I know quite a number of researchers at high level institutions (FAANG and top 10 unis) that don't understand things like probability distributions or the difference between likelihood and probability. There's a lot of "interpretability" left on the table simply through not understanding some basic mathematics, let along advanced (high dimensional statistics, differential geometry, set theory, etc). The AI engineering often "needs" less of an understanding.

But I don't think this is a good thing. I specifically have been vocal about how this is going to cause real world harm. Forget the AGI, just look at how people are using models today without any understanding. How people think you can synthesize new data without considering diversity of that data[0], can create "self healing code" that will generate high quality and good code[1,2], how people think LLMs understands causality[3,4], or just how fucking hard evaluation really is[5] (I really cannot stress this last one enough). There is a serious crisis in ML right now, and it is also the thing that made it explode in funding: hype. I don't think this is a bubble in the sense that AI will go away, but I think if we aren't careful with how we deal with this then it isn't unlikely to see heavy governmental restrictions placed on these things. Plus, a lot of us are pretty confident that just learning through data is not enough to get to AGI. It just isn't a high enough level of abstraction, besides being a pain (see the semantic deduplication comments about generation). But academia is even railroaded into SOTA chasing because that's what conferences like. NLP as an entire field right now is almost entirely composed of people just tuning big models instead of developing novel architectures (if you don't win, you struggle to get published despite differing factors). We let big labs spend massive amounts of compute to compare to little labs who can get similar performance with a hundredth, but don't publish those works. It is the curse of benchmarkism and it is maddening. Honestly, a lot of times I feel like a crazy person for bringing this up. Because when I say "ML needs a solid math basis and deep understanding of the science behind it" everyone agrees, but when the rubber hits the road and I suggest mathematical solutions to resolve these, I'm laughed at or told it is unnecessary.

[0] https://news.ycombinator.com/item?id=36509816

[1] https://news.ycombinator.com/item?id=36297867

[2] https://news.ycombinator.com/item?id=35806152

[3] https://news.ycombinator.com/item?id=36036859

[4] https://www.cs.helsinki.fi/u/ahyvarin/papers/NN99.pdf

[5] https://news.ycombinator.com/item?id=36116939

Well, and frontend work. ChatGPT wouldn't been anywhere near the product it is without the webterface.

As far as PayPal integration, no but it makes you the subject matter expert (SME) in the room above a room full of people who aren't, and maybe aren't even developers.

You could have just read the article to see what the author meant by AI Engineers. You also would have seen that the entire article was addressing a point you tried to make, and you could have responded to the arguments made in the article instead of, ya know, just sounding off based on having read nothing but the headline

I have read the article (though I will admit, quite quickly). The author's definition for AI is by any means not the single one out there, as they outlined themselves. I disagree with the premise that this will be the job of the decade, and I disagree with the premise that it is inherently different from the rest of software as a service. This is a tool, that can be used to solve certain problems, I doubt there is a useful differentiation. That's my point of view, it is antagonistic to what is presented there, and I don't see why you think it departs from usefully commenting on the article.

Reply on news.ycombinator.com