It's time to become an ML engineer (2022) | Hacker News Reader

It's time to become an ML engineer (2022)

blog.gregbrockman.com

38 pointsjxmorris1248 comments

Nearly two years on from this article's publication, 'AI' tools are still as useless as ever for productive work.

Curious to hear what kind of work you do. Because there are definitely fields where productivity as 10x'd because of AI tools.

also curious to hear which fields these are

Anything with using tools which you are not an expert with. If you know how to do things and only use one specific language or framework -- there is nothing to use AI for.

Not management. AI hasn't learned to play golf for them yet.

Definitely has been true for my work. LLMs have absolutely have been useful, I even forked an IDE (Zed) to add my own custom copilot to leverage a deeper integration for my work.

But even if we consider AI beyond just NLP, there's been so much ML you can apply to other more banal day to day tasks. In my org's case, one of the big ones was anomaly detection and fault localization in aggregate network telemetry data. Worked far better than conventional statistical modeling.

I usually assume there is a caricature of "AI Tools" that all of the detractors are working backwards from that are often nothing more than a canard for the folks that are actually using AI tooling successfully in their work.

Seriously? 10x? Either those were cushion jobs or you oversold your claim

Some jobs it's more.

I had to sign a 140 page contract of foreign language legalese. Mostly boiler plate, but I had specific questions about it.

Asking questions to an AI to get the specific page answering it meant I could do the job in 2 hours. Without an AI, it would have taken me 2 days.

For programming, it's very good at creating boilerplate, tests, docs, generic API endpoints, script argument parsing, script one-liners, etc. Basically anything for which, me, as a human, don't have much added value.

It's much faster to generate imperfect things with AI and fix them that to write them myself when there is a lot of volume.

It's also pretty good at fixing typos, translating, giving word definition, and so on. Meaning if you are already in the chat, no need to switch to a dedicated tool.

I don't personally get 10x on average (although on specific well suited task I can) but I can get a good X3 on a regular basis.

But what you're doing isn't a real job. Who hands someone who doesn't speakt the language a contract to sign? Don't you have a legal department that does this job for you and has people that are specialists for that?

Also, what are you going to do if the AI answered inaccurately and you signed a contract that says something different then what you thought?

You are confusing not being able to do something well with being able to do it quickly.

But you shouldn't do some stuff you can't do properly at all, not quickly and not slowly. As a layman, you can't sign a contract in a language you don't speak, even if you have a whole year, unless you can become more-than-fluent in that language in a single year. That's just not something you should do, and the AI isn't reliable enough to help you with it. That's what a legal department is for.

Similar to "real work," a "real job" is a definitional problem that attacks people's senses of self.

I am actually pretty sure that the thing described literally isn’t a real job, at least not working for a serious employer. I can’t imagine a company telling someone to sign contracts in a language they can’t speak and somehow try to make a sense of it.

Either it’s their own company and they’re doing something unwise, they are doing it without the knowledge of their superior or their company shouldn’t be trusted with anything.

The point was that „AI helps me translate the contracts I want to sign“ isn’t a good example of „AI increases my productivity“ because that’s not something you should ever do.

I would never in my whole life sign anything in a foreign language that I don’t understand. It’s the perfect example of what AI is: let’s do anything that looks like a job well done and fuck it. That is not convincing. It’s suicidal.

Please give us a source for the x10.

We never have any proof or source for that, and when we have one (like the Devin thing) it’s always a ridiculous project in JS with a hundred lines of code that I could write in one day.

Give me some refactoring in a C++ code base with 100k lines of code, and we’ll be able to talk.

being 10x more productivie with ai doesn`t mean that someone is now a full engineer. ^^

No, but if "productivie" becomes one of the variable names eventually used in legacy work . . . full engineer achieved!

This whole area is so drenched in bullshit, it's no wonder that the generation of BS and fluff is still the most productive use. Just nothing where reliable facts matter. I do believe that machines can vomit crap 10x as fast as humans.

Yep, I think this is definitely the case for e.g. software engineering.

They don't help you with the non-productive parts, freeing up more of your time for productive work?

Still remarkable though that they didn't mean a prompt guesser when they said, “ML engineer”.

They can be incredibly useful for many kinds of knowledge work. I've been much more productive using LLMs than before. I'm at the point where I dont know how I ever managed without.

Can you even call it "knowledge work" then?

the difference is that before LLMs you had to use search to find answers to unknowns. The productivity boost of knowing everything immediately doesnt fundamentally change the work itself. I understand over reliance might become problematic at some point. My solution to that would be a bunch of GPUs so I have an R1 locally, anytime. the future is here, some people just havent noticed.

After crypto-bros, we now have AI-deniers

I don't know that I'd say _useless_ but there's _a long way_ to go. I also suspect that many developers are going to wind up being dependent upon them at the expense of getting familiar with reading code/documentation, exploring with debuggers, writing forum/blog posts, etc.

Here is one example from the last time I asked Claude a question about message filtering on AWS SQS, which is a very common exchange in my (relatively limited) experience with these tools.tool

# me responding to a suggestion to use a feature which I was pretty sure did not exist

> ... and you're sure this strategy works with SQS FIFO queues?

# Claude apologizing for hallucinating

> I apologize - I need to correct my previous responses. I made a mistake - message filtering with message attributes is NOT supported with FIFO queues. This is an important limitation of FIFO queues.

If I didn't already have familiarity with this service and its feature set, I would have wasted 15/30/60 mins trying to validate the suggestion.

I can't imagine trying to debug code that was automatically generated and inserted into my toolchain using Copilot or whatever else. Again, I'm sure this will all get better but I'm not convinced that any of these tools should be the first ones we reach for.

Chatgpt is really good at small throwaway scripts to accomplish some task that could have taken a good portion of the day to write up otherwise (almost always gets the code right on first try)

It can add usability features pretty well too. "Look at the variables in the script and add a feature to accept them as command line arguments and environment variables. Make the CONTENT variable mandatory."

They are also pretty good at software translations nowadays. You can give them a yaml file and they know what's a variable or placeholder and what's text to be translated.

In the time since this publication, the Nobel Prize in Chemistry was awarded to an ML Engineer & AI researcher.

While it's accurate that there's a lot of AI slop out there, it's also a reasonable assertion that working at the frontier of AI & ML research can be a worthwhile use of your life.

The article isn't about AI tools. He's saying that people should consider becoming engineers working on building AI training systems.

> Nearly two years on from this article's publication, 'AI' tools are still as useless as ever for productive work.

Fully disagree. When developing I use llm's all the time. its far quicker for me to ask an llm to build a react component than for me to type it and have to remember all the little nuances for a technology I use a couple of times a year.

When proto typing things or doing quick one offs, using an LLM makes me 10x more productive.

OK, but React development is one of the simplest development tasks out there. It’s probably the most popular UI framework with the most data for the LLM model to generate off existing solutions. Low hanging fruit that also provides the least value to companies since react developers are a dime a dozen.

Also, one could argue that the LLM provides less value since a simple Google search for most react questions can produce a very viable answer, since 90% of what most React developers do has already been done and shared on the Internet.

I find this take interesting in the present. I work at $BIGCO that has done lots of ML in house for many years. I have noticed more and more ML projects being killed because their work is superseded by external model vendors. Why spend millions on a team to build a new model, when you can pay thousands to rent someone elses?

It's a good question, and there are sometimes good answers. Does someone else's model understand how to standardize and interpret your company's data? Will they notice and tell you if something isn't looking right? What if the model needs to change to account for opportunities or risks from evolving data sources or business environment? Building in-house can help ensure that an organization has the competencies to do these things without depending on someone else.

Good point, it definitely isn't hands off from what I have seen. Just fewer people needed to manage those integrations.

I hope this removes some people from the general dev pool who jumped because someone told them to "learn to code" and they thought it was a good way to get rich.

Please add the (2022) date

> All that being said, AI is not for every software engineer. I’ve seen about a 50-50 success rate of engineers entering this field. The most important determiner is a specific flavor of technical humility.

Oh, give me an f-ing break and spare me the self-congratulation.

You can't expect much from "It's time to become an ML engineer" written in the AI hype period.

The TFA's AI-generated image "Harnessing the most compute in the known universe" that captures the entire globe in a blue hue of AI compute is a bit disturbing, TBH.

Lol for some reason my company's IT team has deemed Greg's website is bad and it's blocked by Cisco Umbrella for being a "phishing threat"

Did you used to be a blockchain engineer?

You can find out a little more about Greg's background here: https://gregbrockman.com/

Rather than a crypto-guy, he used to be the CTO of Stripe, who were moderately successful.

And as we know, success in one area is never a source of overconfidence in another.

Like most CEOs he probably just asked the model ‘tell me about Greg Brockman’ and got a nice response. Thus they think it’s amazing.

C suite love LLMs because the make them feel important, but they don’t have to use them for anything serious.

In order to make more money?

I think I’ll pass

Reply on news.ycombinator.com