Hacker News Reader

Comment by simonw | Hacker News Reader

Do you want to train models from scratch, or do you want to build cool things on top of AI models?

If the former, I suggest digging into things like the excellent Fast AI course: https://course.fast.ai/

If the latter, the (relatively new) keyword you are looking for is likely "AI Engineer" - https://www.latent.space/p/ai-engineer

There's an argument that deep knowledge of how to train models isn't actually that useful when working with generative AI (LLMs etc) - knowing how to train or fine-tune a new model is less useful that developing knowledge of the other weird things you have to figure out about prompting, evals and using these models to build production-quality apps.

View on HN

This is the right question. There are relatively few people doing "Capital A Capital I" work building and training generative language/image models. Especially now that people have realized that calling the OpenAI API is going to work better than dedicating a team to tuning your own BERT model.

So some options are: * Building ML models for traditional applications - forecasting, ranking, recommending, all of that. Data Scientists and ML Engineers haven't gone away. * Working as an "AI Engineer" building on top of existing APIs and models. Very hip, but also in flux - I couldn't tell you whether that role will still be considered rare & valuable in a few years time, or what skills will be core to it. * ML Ops, engineering work for building & serving AI & ML models. It's always good to be selling shovels.

I would try to get into a team you're interested in as a SWE and then upskill or pivot. In my experience this is more effective than trying to completely reskill and sell yourself as an unproven prospect for MLE or AI Scientist work. AI/ML teams still need software, and in fact many of the best researchers are not great software engineers.

> Especially now that people have realized that calling the OpenAI API is going to work better than dedicating a team to tuning your own BERT model.

At 100x the operating cost.

If you have any kind of scale, using a model that's appropriately sized to your task is going to be better.

> If you have any kind of scale

I think that's the key also in terms of team size, if you don't have a dedicated team to take care of training / running a model then it makes sense to use an existing API and pay for it until you have enough scale.

Gemma2 might be good enough for a drop-in option as well. The benefit for those who built on ChatGPT API is that as more open weight models get released, they can silently migrate their API usage to self-hosted models … ChatGPT allowed them to prove product-market fit early!

That is a fair point.

Lots of times building the inefficient thing to prove market value is better than building the scalable thing that doesn't solve anyone's problems.

I want to agree with this, but after reading the whole article, I have no idea what the skills of an AI Engineer are.

Why is that not the job of a software/product engineer?

> none of the highly effective AI Engineers I named above have done the equivalent work of the Andrew Ng Coursera courses, nor do they know PyTorch, nor do they know the difference between a Data Lake or Data Warehouse

they're explicitly not trained in ML/AI. any software engineer can write a good prompt, call an API, and deploy that on an http server.

why is that not just software/product engineering?

An AI engineer wires up APIs to each other and returns the result as JSON, the same process as any other web dev.

Like any other web dev job there are differences across domains that can make it valuable to hire someone with past experience in your particular industry (in this case LLMs), but the only reason this gets a brand new title and others don't is hype.

So much less complicated than your average web developer job in 2024?

If you're only doing backend AI work, yeah, probably.

It basically IS just software/product engineering. An AI engineer is a product engineer who's ahead of the curve from everyone else in terms of building on top of LLMs and similar.

It might not even still be a speciality in a few years time. Right now though there's a lot of depth to the field that people who aren't focusing on it are missing out on.

The fact that in 2024 you can still get sub-1-hour applicable reply from the creator of one of the top web frameworks ever is everything that makes HN amazing.

One thing I've encountered more than once in the industry right now is that a lot of companies want to hire "AI Engineers", but they task their staff Very Serious Data Scientist with handling the interview. Inevitably, all the questions will be minutiae about different ANN designs and training processes. And without fail, no one involved will be actually dealing with that stuff on a day to day basis.

I agree with your overall point. I'll add that the Fast AI course really does focus on building practical things, and I suggest it for anyone who wants to learn the basics of neural networks, even if you'll never train one from scratch.

This is an extremely good insight and piece of advice. Already we're starting to see what looks very similar to a "frontend" and "backend" kind of development model similar to the previous tech generation.

While there are advantages to going "full-stack" in this analogy most people focus on one or another

I looked at the latent.space site and it didn't seem to be updated/maintained. What I would like is a reputable playlist/roadmap of things to consume to go from Senior Engineer to AI Engineer.

Foundation you need what, a primer on Linear_Algebra+Calc+Stats+Prob. At a minimum you have to know the language. Then, do you need anything of classical AI? Anything of classical ML? Do you even need to have any knowledge of Deep Learning other than it exists? Or transformers? OR do you just need a bunch of tutorials on how to implement GPT API's, structuring and managing prompts, etc.

I know all this information is out there but has anyone linked it together? If so, is it paywalled? I did a quite search on AI Engineer and before I knew it a site was asking me for 20k+ and 1.5 years of my life. I already have a master's in CS, so I assume I can make faster strides and do it for less. Can anyone advise?

Reply on news.ycombinator.com