The revenge of the data scientist
hamel.dev
hamel.dev
Data scientists were appreciated largely because of their ability to create models that unlock business value. Model creation was a dark magic that you needed strong mathematical skills to perform - or at least that's the image, even if in reality you just slap XGBoost on a problem and call it a day. Data scientists were enablers and value creators.
With GenAI, value creation is apparently done by the LLM provider and whoever in your company calls the API, which could really be any engineering team. Coaxing the right behavior out of the LLM is a bit of black magic in itself, but it's not something that requires deep mathematical knowledge. Knowing how gradients are calculated in a decoder-only transformer doesn't really help you make the LLM follow instructions. In fact, all your business stakeholders are constantly prompting chatbots themselves, so even if you provide some expertise here they will just see you as someone doing the same thing they do when they summarize an email.
So that leaves the part the OP discusses: evaluation and monitoring. These are not sexy tasks and from the point of view of business stakeholders they are not the primary value add. In fact, they are barriers that get in the way of taking the POC someone slapped together in Copilot (it works!) and putting that solution in production. It's not even strictly necessary if you just want to move fast and break things. Appreciation for this kind of work is most present in large risk-averse companies, but even there it can be tricky to convince management that this is a job that needs to be done by a highly paid statistician with a graduate degree.
What's the way forward? Convince management that people with the job title "data scientist" should be allowed to gatekeep building LLM solutions? Maybe I'm overestimating how good the average AI-aware software engineer is at this stuff, but I don't see the professional moat.
I don't really see why evals are assumed to be exclusively in the domain of data scientists. In my experience SWEs-turned-AI Engineers are much better suited to building agents. Some struggle more than others, but "evals as automated tests" is, imo, so obvious a mental model, and can be so well adapted to by good SWEs, that data scientists have no real role on many "agent" projects.
I'm not saying this is good or bad, just that it's what I'm observing in practice.
For context, I'm a SWE-turned-AI Engineer, so I may be biased :)
One of the key value is that it forces some thinking about what is the task you want to solve in the first place. In many cases, it is difficult if not impossible to do it, which implies the underlying product should not be built at all. But nobody wants to hear that.
Doing eval only makes sense if making the product better impacts something the business cares about, which is very difficult to do in practice.
But aren’t there still plenty of opportunities for building ML models beyond LLMs, albeit a bit less sexy now? It’s not like you can run a business process like (say) AirBnB’s search rankings or Uber’s driver marching algorithms on an LLM; you need to build a custom model for that. Or am I missing something here? Or is that point that those opportunities are still there, but the pond has shrunk because so much new work is now LLM-related? I buy that.
Conversely, data scientists are doing software engineering, including webdev. It’s an interesting time. I think it’s less about the job title demarcation now, and more about output.
> You recognize that you haven't really needed strong mathematical (or coding) skills to create models for some time.
For anything NLP sure, it definitely wins. However, I’ve just recently used some big fancy OpenAI model to actually just label thousands of text data for me, just so I could build a classifier with CatBoost. Guess what, inference speed is at a guaranteed sub 100ms and it costs $0 in tokens. The”AI Engineer” solution here would be just run every classification request through an LLM.
AI Engineering is going to have the same problem we had when Data Science as a term arrived and you had every Statistician saying they’re just re-inventing everything that exists in statistics, poorly.
When LLMs appeared it was just so much easier to use then as an uber model and leave behind the training and inference infrastructure (if you can even call it that).
Now that LLMs can code I expect we'll be coding up custom model pipelines more and more... but only when we stop subsidizing LLMs.
Perhaps they can use the LLM to write and deploy these models without needing a Data Scientist but that seems risky to say the least.
In my company, the most Data Scientist-adjacent people are the Data Analysts but they tend not to have programming experience beyond SQL and basic Python and they aren't used to using the terminal etc.
The LLM solution will be much more flexible because prompts can change more easily than training data and input tokens are cheap.
One of the points of the article is the importance of gathering data to support your conclusions.
> prompts can change more easily than training data
Training data is real, and prompts are not. I don’t think this is an apples to apples comparison.
1. think of the context data as training data for your requests (the LLM performs in-context learning based on your provided context data)
2. think of evals as test data to evaluate the performance of your agents. Collect them from agent traces and label them manually. If you want to "train" a LLM to act as a judge to label traces, then again, you will need lots of good quality examples (training data) as the LLM-as-a-Judge does in-context learning as well.
From my book - https://www.amazon.com/Building-Machine-Learning-Systems-Fea...
Gets pretty meta when you’re evaluating a model which needs to evaluate the output of another agent… gotta pin things down to ground truth somewhere.
You might hit a local maxima or go down a blind path. I tend to completely start my code base from scratch every week. I would make things more generic, remove unnecessary complexity, or add new features. And hope that can move me past the local maxima.
Was the data scientist role only about building NLP models? Are the LLms gonna build Churn prediction models? Tell the PM why stopping the A/B test halfway through is a bad idea? Push back on loony ideas of applying ML to predicting sales from user horoscopes?
Maybe the role is a bit tinier in scope than 10 years ago, but I see it as a good thing. If you looked at DS positions on job search sites the role descriptions would be all over the place, maybe now at least we'll see it consolidate.
So yeah, the bag-of-words model got replaced with a chatbot several years ago (when chatbots were all the rage back in like 2017) and will probably get replaced again with an LLM-enhanced chatbot soon. But the meat and potatoes are those classification and regression models and they aren't going anywhere.
Much of my work takes the form above -- in other words figuring out what to do. once i've decided, it can of course spit out the boilerplate code much faster than I could, and I appreciate that. But for the moment I think I still have some job security thanks to the first issue.
Though for greenfield development, writing the test cases (like the spec) is equally as hard, if not harder than writing the code.
I also observe that LLMs tend to find themselves trapped in local minima. Once the codebase architecture has been solidified, very rarely will it consider larger refactors. In some ways - very similar to overfitting in ML
Sitting with a running system teaches you things you would never think to measure.
In my experience, this is missing a big part of the work: confirming what the data actually is, sometimes despite what people think it is.
I teach ML for a living and was amazed with what the tokens gave back to me after many rounds of experiments. If Kaggle was still a thing, AI would generally beat it.
The challenge I've seen is that most data science/ml modeling work is quite weak. Folks don't even know the basic tools well. Not sure if giving AI to them will really open up many doors to them.
As always experts love minions of juniors doing their deeds. Non-experts get to wade through slop.
A _huge_ part of a DS is saying "No" to bad ideas posed by non-experts. The issue with LLMs is all they ever say is "Yes" and "Wow, that's such a great idea!"