The industries AI is disrupting are not lucrative | Hacker News Reader

The industries AI is disrupting are not lucrative

theintrinsicperspective.com

74 pointssnewman104 comments

It’s fascinating how little people really understand how LLMs are impacting businesses. Almost any task that couldn’t be done because it was too expensive to hire thousands of humans to sift through things is being outsourced to LLMs. I find it remarkable that people don’t realize how amazing these things are at classification and structuring of complex stuff with some alacrity, often far superior to human agents. They scale arbitrarily, work 24x7, and have very low failure rates compared to poorly trained high turnover humans. They generally do a good job identifying when they can’t classify something and delegate to human review. Are they perfect? No, but they’re considerably less error prone than a staff of hundreds of humans. Is it lucrative? Absolutely. I’ve seen this now at five major megacorps and I have to believe it’s going on at most.

Agreed. I thought the analysis in this article was exceptionally poor.

For example, there is a small section handwaving away the potential impact on the search engine market, saying that ChatGPT has been out a year and hasn't made a sizable dent in Google.

But the thing is, the way ChatGPT interface currently works and the way it is marketed, most people don't even think of it as a "Google replacement" in the first place - except perhaps some techies and folks on HN. But it's not hard to o imagine advancing tech (so LLMs can be more frequently updated) and a rethought UI (and I'm not just talking about plugging in some "ChatGPT-in-Bing"-type interface, either) that could radically change where people go when they want information.

> there is a small section handwaving away the potential impact on the search engine market

I would assume lot of IT folks might have moved to ChatGPT to ask IT (programming) questions instead of searching Google and lot of these folks also use Adblockers which blocked statcounter's trackers so the stats might be misleading by a bit.

That’s also not considering people who “ask ChatGPT things” instead of googling and don’t consider it the same as search

Can you elaborate?

I think you're right that LLMs are very good at classification and structuring, but that's not how I see them used. The experiments I have witnessed resemble "hail Mary" attempts to replace human agents with a magic chatbot that would know it all and speak plainly. Those experiments almost always fail, because the bot hallucinates too much and/or because the training data is poor and not detailed enough to provide efficient fine-tuning.

For example: a helpdesk handles incoming support requests for an internal, proprietary application. Support agents write the request down, then spend time investigating the problem and possible solutions, based on the symptoms. There is a weak link between the symptoms reported by the users and the actual underlying problem, so this takes time, esp. for newly hired support agents.

One approach would be to build a semantic search engine to speed up the investigation phase, which would positively affect the productivity of the agents. But that's not what's been tried: the company hoped to get rid of support agents completely, and replace them with a bot that would interact with the users and solve their problems on the fly.

When that doesn't work, everyone's disappointed and tends to simply give up.

So, imagine you need to evaluate applications for some line of credit by reading the documents associated and classifying the businesses in a taxonomy of business types. Depending on volume this could be millions of documents. You can prime an LLM context with the taxonomy and examples of each, then evaluate each document in a clean primed context asking for a classification if it’s clear and a request for review if it’s not clear. This works exceptionally well in practice, almost double the precision and recall of humans.

Businesses are replete with similar relatively trivial classification problems that often require armies of low paid low skill workers but are fairly crucial to the ongoing operations of the business. Chat bots are the default thing people gravitate to with LLMs because they’re the example in chatgpt. But that’s a really uncreative use of the tech, and the domain they’re exposed to is pretty open ended leaving them subject to hallucination, prompt injection, etc. A more constrained domain with more business impact is where I see meaningful applications happening.

I agree with the approach of augment-not-replace human agents. For example there’s a contact center management company using Langroid[1] (the Multi-Agent LLM framework from ex-CMU/UW-Madison researchers) in exactly this way: they use a Langroid RAG agent to come up with top 3 possible answers, and the human agent then picks a good one. Interestingly, they index past successful conversations of experienced human agents, and the Langroid agent’s suggestions are informed from these as well as company docs.

[1] https://GitHub.com/Langroid/Langroid

From a personal perspective… I have a family photo collection going back decades that I used to put effort into tagging and describing, before realizing that automatic classification models would come eventually. After years of waiting, it feels like we’re close, but I’ve tried LLaVA-1.5 and it was underwhelming. Are there other more specialist models I should try? Must be open source and easy to invoke from a shell or Python script.

Try Fuyu-8B or CogVLM. Failing that, there’s always the GPT-4V API. There are also other non-LLM solutions for image captioning / tagging (e.g. what the smartphone vendors do), but I’m less familiar with those.

+1 on CogVLM.

https://github.com/THUDM/CogVLM

Recent discussions: https://github.com/ggerganov/llama.cpp/discussions/4350

It is the best open source vision language model out there that I'm aware of that's most comparable to gpt4v. Beats the pants off Llava1.5 and variants like bakllava.

There's a demo here http://36.103.203.44:7861/

i have been watching this subthread in hope that something interesting might come out of it and you just delivered. this demo is amazing. i am now dying to apply this to my photo collection.

on that note, some time ago i came across an image management tool that uses machine learning in some form. i don't have time to look for it now, but i just wanted to mention that someone is working on something like that.

>> I’ve seen this now at five major megacorps and I have to believe it’s going on at most.

I sense a potential misaglignment with the article headline that:

    The industries AI is disrupting are not lucrative

While "major megacorps" may indeed be casually using AI for eg "classification and structuring of complex stuff..." I have no problem understanding why your averge mom 'n pop store might not even have problems suitable for this particular type of number-crunching

I think this article makes some interesting points, but it's not exactly on the money (pun intended)

Quote:

    What are AIs of the GPT-4 generation best at? It’s things like:

    - writing essays or short fictions
    - digital art
    - chatting
    - programming assistance

The author lists these points yet fails to realize that these four activities succintly sum up the major part of Internet-as-we-know-it. Add media management for streaming and we're basically there. Also:

    The issue is that taking the job of a human illustrator just. . . doesn’t make you much money. Because human illustrators don’t make much money!

This is direcly seeing things from the wrong perspective. On average illustrators may make little money individually due to a huge over supply in their market, but still those that depend on their services would probably not agree to calling these services "cheap" or that this part of the budget was neglible. This is not about wage competition at the producer stage - it is about cost savings at the consumer stage.

As for the question:

    How precisely will AI capture a portion of the $300 billion movie and game market?

...the author tries to brush it off referring to some strike at one particular place called Hollywood. Well, that market is larger than any particular place, and... it is evolving too. And, at the core of it is exactly the services that LLMs excel at, be it storytelling or hallucinating.

Clarifying my point it seems as if the author makes the claim that extremely lucrative fields are in fact not.

Last, I should add one particular multi-billion dollar industry in which AI seems to have quite some potential: Crime. Specifically, but not limited to, fraud.

Somewhat of a hot take. Normal classification algorithms output a confidence weight, LLM's don't. This means you can only use LLM classification if you don't care at all about managing your precision and recall metrics.

This means LLMs are useless for 90 percent of real business requirements. (Business actually wants some sort of binomial regressor, not a plausible-looking hallucination. Plausible-looking hallucinations are the domain of spam and copywriting.)

The reality is LLMs aren’t competing against normal classification algorithms, as the tasks have no algorithm that approaches the observed precision and recall of LLMs. They’re competing against humans, who also have no metrics other than observed precision and recall. When you observe their performance vs humans at these sorts of tasks they’re generally superior, especially as the quality of the human work is very low - poorly trained, poorly paid, and poorly incentivized people doing very boring work do very badly. LLMs do very well.

The savings can be substantial even tho the people are poorly paid and trained, because you need so many of them, and need a complex organization built around them to source, train, manage, and observe all the humans - of which there can be many thousands with high turnover. Their function is actually pretty important and the high failure rates cause material impact to the business in terms of loss and risk. Improving these dimensions substantially upgrades the business over all.

The truth though is we are really early in the development cycle. These sorts of changes will play out over a decade. In that time I expect the technology to improve in many dimensions - power, capability, ease of integration, ease of observing, ease of fine tuning/alignment to task, applicability to other functions, etc.

You can fine tune an LLM to output a particular token for category X. This way, you get 1) fast inference (just need to generate a single token) and 2) confidence metrics (just use the logprobs for the token that was chosen). Gets me to 93% accuracy with a 7B model, trained with only 15 examples per category (around 10-15 categories in total, I think).

Auto Label can do it even with closed-source LLMs: https://www.refuel.ai/blog-posts/labeling-with-confidence

Also, your 90% number is a 'plausible-looking hallucination', which is mildly funny.

>Explicitly prompting the LLM to output a confidence score is the least accurate

They don't show the whole prompt they use for this so I am sceptical of the comparison that recommends their service which does the token level accuracy. In my short testing you can drill down some acceptable accuracy for specific domains if you lead it over some general "ask yourself" questions then "finally, output a % confidence"

I couldn't care less about the specific implementation.

My parent comment said "LLMs are useless for classification, because they can't give confidence scores". I think i have shown that to be demonstrably false.

My understanding is these are more “confidence in the tokens selected being appropriate” rather than the more fundamental “confidence the classification chosen is accurate given training data,” where training data is samples of the distribution being classified rather than language corpus. They seem similar at first blush but the first is more about appropriateness of phrasing and how esoteric the semantics are, and the latter more descriptive of some statistically observable process - which is something LLMs are literally incapable of reasoning about directly.

As you have seen it in action: how much more additional profit is in being able to do those task now? How much RoE increase is there? (For the megacorps).

In some cases as much as hundreds of millions of profit, in some cases its existential- one situation they needed to spin up a complex business process that would have required an entire organization of likely thousands of humans to comply with a regulatory requirement they were found in violation of and the only way to comply was to have humans review millions of documents and web sites manually to classify them. With a fair amount of fine tuning they were able to build an LLM infrastructure that did this role with the backing of a 20 person review and escalation team. They couldn’t have spun up the equivalent traditional org on time and would have had to had a potentially ruinous public regulatory issue. The cost savings on one hand is hard to gauge, particularly because it would have greatly impacted their culture and organization to add so many humans so quickly. Anyway it’s amazing how many ways these things can be used.

Thank you! So it is rather an incremental change in profit, not transformational (when it isn't life or death of the corporation/a business model requirement)?

I think I would say this. It’s a technology that’s on radars for about 12 months, is immature in all aspects, and has already transformed large swaths of the business operations to be much cheaper, scalable, and precise. I’ve never seen that in 30+ years.

In some cases it actually is transformational as in the case I mentioned that it talked directly to a core business problem that would have killed the org. In others it’s incremental, but at the scale of megacorp there everything is incremental and something that moves margins by 5% or something is earth shattering for them. It unlocks businesses they would have been too hard to comply with requirements, it stream lines operations, etc, which for many businesses are monumental changes.

What happens in 5 years when models are much more powerful, interfaces more clear, and ability to customize easier?

Thanks. I think the big unlock is corporate structure and setup - some classic reasons for having one firm might very well disappear.

Like what

Just imagining some random ideas here and the use case come instantly. Imagine you have a user feedback system that has thousands of submisisons and you want to know how many are talking about a particular feature. How many are speaking positively about the product, how many seem to be asking for support rather than asking for feedback.

Being able to sort through thousands of items for basically free which would have been so laborious previously that you would not have done it. And its a use where a small amount of error does not matter.

Lots of government contractors have junior analysts-type jobs whose work entails responding to government RFPs. I know of one specifically who found ChatGPT was great at these responses - it still took further human review and editing, but so did the initial human drafts. Just one example.

How big is the market of "junior analyst-type jobs" globally?

Will disrupting or obliterating that industry make an impact on our lives? Will it add anything to the bottom line of OpenAI?

> How big is the market of "junior analyst-type jobs" globally?

Hence the "Just one example" sentence I ended my comment with. There are tons and tons of others. The Washington Post did a whole long article months ago about people who were laid off _already_ due to ChatGPT. Folks like marketers, copywriters, data analysts, medical transcriptionists, etc. There are a large number of jobs out there that used to need, say, 3 or 4 people, where AI could bring that down to 1.

Construction business

A lot of people are just in denial. It's some psychological mechanism. They have to call it a stochastic parrot, they have to think of the LLM as something that's only a toy at worst and at best equivalent to google search. They're wrong. This thing can code too. It won't replace a human coder yet, but it's part way there.

It's only a matter of time before it's fully there.

It is a stochastic parrot.

The error is deriving human ideas of 'quality' from that.

The stochastic parrot might well be better at your job than you are.

if it's a stochastic parrot then you are too. You just have a better predictor.

Obviously what I and everyone means by stochastic parrot is that it's not intelligent. It's wrong. It is intelligent. At worst it's as intelligent as a mentally retarded/schizophrenic or insane human. But even a mentally handicapped human still displays a level of intelligence.

It won't be fully there until it gives a confidence interval and has weights to manipulate precision and recall. These are baseline non-negotiable features.

Unfortunately the way LLM's are architected right now means they will never be "fully there".

Why does it need things that are absent in humans?

Sure, those things could make some things easier, but why are those things that humans don't have suddenly "non negotiable" when a machine also can't do them?

Legal issues?

Those things aren't absent in humans. We spend trillions of dollars to make sure humans are qualified and aren't just bullshitting plausible-sounding words. (In fact, that's really the whole point of "education", which some people spend 20 years being subjected to.)

> confidence interval and has weights to manipulate precision and recall

Neither of which are supplied by education, and I am unaware of any humans basing able to deliberately (let alone precisely) alter their own perception on the scale between precision and recall — look at something, it's "obviously" X or not X almost immediately after you know what the category X is, even when you're wrong. Apart from the very first few encounters it doesn't even matter how much of a noob or expert you are in the field of X-recognition, your confidence is the same.

Worse:

> qualified

Given how well ChatGPT does on standardised tests, doing better than many actual humans even despite its many flaws and limitations, it should be clear that the qualifications are not good enough to do what you're expecting them to do.

> and aren't just bullshitting plausible-sounding words.

That's demonstrably how humans work (at least when it can be tested, perhaps people who need split brains are weird): all the indications are we do a thing first and then come up with a justification after.

(And then we have people like Boris Johnson, 2:1 BA from Oxford, with a disconnect between reality and the words leaving his mouth that would be comical except he actually became Prime Minister in real life and not just a TV comedy blending 'Allo 'Allo with The Thick of It).

That’s absolutely false.

1) statistical intervals are mathematical artifacts of our techniques that describe the samples observed and trained on. They aren’t ground truth observations of the underlying process or populations. We put too much weight in them.

2) you can absolutely observe precision and recall from online performance and compare that directly against human performance on known labeled data. From that you can determine which has the better error rates. That is entirely sufficient for almost all practical use cases.

3) obviously it would be better if we could derive confidence of a classification, but given the fact LLMs aren’t directly reasoning or optimizing the statistical properties of some mathematical problem they will never have the same character as say regressions or other statistical techniques that are some form of mathematical optimizer. They’re just solving the problem in fundamentally different ways.

4) it’s not clear to me statistical optimization is a universally superior technique. The reality is many problems are better solved with an abductive reasoning technique like LLMs exhibit, and humans absolutely use when classifying. There are lots of awesome features such as the ability to inspect residuals and confidence intervals, and they’re generally computationally cheap. But for all that their absolute utility in the real world is fairly limited, especially when considering complex non linear tasks with huge latent spaces that are unobservable.

Not true. Not even humans have that level of manipulation. We employ humans despite the lack of these features.

That means these baseline features are 100% fully negotiable. They're negotiable all the time because we employ humans that DON'T have these features. Thus LLMs don't need the features either.

Either way, we don't understand LLMs well enough to even predict whether future modifications will or will not have the features you claim. Such a hardline claim that it will never "fully be there" is illogical. Nobody predicted that transformers could lead to LLMs, nobody can predict what LLMs will lead to.

Humans absolutely do have these features. (Implemented in a slightly buggy way, yes, but that's not relevant in this context.)

There's research on estimating confidence from neural net activations in LLMs.

The next AI winter is going to be savage, especially with the backdrop of normal (i.e. non-ZIRP) interest rates.

Possibly AI might be seen as the 'hubris most high' of the current tech bubble, with a correspondingly deep societal 'come down'.

The painstaking, boring, and lucrative work of 'wiring the world' will continue of course, but perhaps with less fanfare.

AI is here, now, and OpenAI is so saturated they had to cancel new customers.

While the hype is real, so are the benefits. This feels more like the Internet in 1998 - there is a bubble forming and it may burst, but it is not like we got back to the Way Things Were after. Some sectors like (physical) mail continued their decline and others (like Google) continued to grow.

The reason the previous AI winters were winters were that the AI didn't bring any benefits.

I always wondered if those software-based technological bubbles (and I’m not necessarily agreeing that LLM-based AI is one of them) are doing so much harm to the overall progress of software, that we need at some point to explicitly identify it as a risk to the whole industry and the professional community. Imagining the possible negative effects of any sort of bubble burst in the area of AI while also understanding the true potential of the underlying technology is a painful realisation. “We could’ve done so much”-kinda feeling. Maybe I’m just too pessimistic.

Preventing these bubbles assumes a level of foresight that doesn't seem feasible- besides a general limitor on any and all potential advancements, there is no possible way to know which technologies wil substantively deliver against their promises.

The software doesn't cause bubbles, the "eat the world" model causes bubbles. When everyone is trying to capture the entire pie, it creates this constant, industry-wide FOMO. Like when Spotify decided that podcasts were going to be big, Amazon felt like they couldn't miss out on it. So Spotify signed Rogan, Amazon signed My Favorite Murder, etc. And they bid up the price on each other and now Spotify is in hot water because the economics don't make sense at the ridiculous valuations they FOMO'd themselves into.

I don't pretend to know what a solution would be, but it's not like we have to choose between bubbles and innovative companies; bubbles are about how things are financed, and not about the things themselves. The horticulture industry is doing fine and controlled environment agriculture continues to innovate, without bidding up the price of tulips to something insane.

I know it's hard to imagine (and I don't mean to be patronizing, I have trouble imagining it), but we could have a software industry that wasn't predicated on selling dollars for dimes until you've developed a monopoly and can charge high rents.

Can we really? Where's an exostant model for it?

Virtually any other business model. You make something people want to pay for, and then you charge a price that compensates you for your costs plus a margin determined by your negotiating power. This doesn't produce eye popping returns, but it does produce financially sustainable companies.

Operating at a huge loss for a very long time in the hopes of capturing the entirety of a market is not the norm overall. I know it can feel that way from inside our industry, but that's our cultural biases showing.

I'd like to stress that that's an observation and not a criticism, and that I don't mean bias in the pejorative sense.

That reminds me to go back and watch some old episodes of Connections by James Burke. He does a great job of showing the unpredictable and non-linear path of progress. https://en.wikipedia.org/wiki/Connections_(British_TV_series...

Why?

Ai already makes money.

My company is paying for GitHub copilot.

I'm paying for chatgpt.

I'm seeing more and more news article with images generated from ai.

The demos for integration I have seen, work.

Every company which is right now learning how to leverage ai will be able to pivot and already have necessary things in place like a company, people, base infrastructure.

And the research is still super hot.

I think the issue is that, per the article, even if it works it might only provide marginal value. So copilot may indeed be useful to you/your company, but perhaps, once all the bean counting has been worked out, not useful enough to justify the stratospheric valuations.

> not useful enough to justify the stratospheric valuations.

but why is that a concern for anyone else but the shareholders?

Enshittification as a result, cost increases and so on.

The question is, how much do you pay and how much do you spare with it?

Your company pays for GitHub Copilot, ok, but how many programmers are fired after they pay for it? Or does it have a meassurable effect at all? If not, this is a nice toy that will be removed soon.

The news articles have generated images because they are cheaper than buying some stock photos. But stock fotos are still cheap.

Ai will get better not worse.

You will not be able not to pay for code ai due to the advantage other companies have with it.

For now I guess no job loss but also less new head counts

it's going to be used for all the wrong industries it's already been tried. hiring process, referral to criminal sentencing, spam... and it'll be global. the bias will be baked in

You really think there's gonna be an AI winter? Is that based on the assumption that LLM's will plateau? Do you believe AGI is never coming, at least in the next 10-20 years?

I think your view represents something I see a lot on this site. A bearish cynicism at AI progess, an assumption that it's all just a hype cycle, comparing it to crypto and NFT's and a confident assurance of the momentum of the status quo which will return when the "hype" dissipates.

What specific capabilities of LLM's and AI models would convince you that this is a sea change beyond the normal hype and boom bust cycles of the tech world?

I recently read the book DotCon. It's quite an enjoyable read because the author adopts a slightly mocking tone of voice about big predictions like "The Internet will replace print media and even TV". Seemed ridiculous and yet gen Z and younger now get their news from Tiktok and would rather watch YouTube than a movie.

What really got me was how both the bullish and bearish views were correct. In 1998 people knew we would eventually watch media over the internet, but it wasn't until after 2020 that my dad would open Netflix first instead of regular TV.

In 2020 I saw a whole bunch of businesses scramble to accelerate their "digital transformation" and try to figure out "APIs" and "mobile". So if it took the pandemic for businesses to wake-up to computers and the internet, how long will it take for existing businesses to overcome the organisational inertia to adopt AI?

In reality, existing profitable business models will chug along and slowly see their margins erode as new young businesses come along and eat their lunch. Just as Amazon continues to ride the "simple" idea of putting the computer at the centre of a retail business (and decades of good execution) while many old school retailers are still struggling to catch up.

The exact same discourse was being said by blockchain (so, no NFTs) people around 8-10 years ago. We were also just 5 years away from getting autonomous vehicles around 2016-2017, we haven’t.

The difference is LLMs are here to stay. They are already very useful and efficient. There is no turn around. Even if current state is the dead end of their evolution. Which it isn't, of course.

The future.. there are so many ideas in the air right now that we can expect significant progress in next months and years. There will be fluctuations with bubbles and bursts, but that's how economics works.

I expect slow approximation to AGI with increasing capabilities different products will claim to be it. First generations will be just very capable lifeless calculators. Subhuman in general with some superhuman abilities, like LLMs are today. When they will be able to improve themself, so called singularity? We are getting closer. Likely those will be still calculators which can be turned on/off at any moment.

We have autonomous vehicles right now in San Francisco.

I understand the blockchain-AI comparisons, but there are a few significant differences:

1. AI has proven to be much more useful in the real world

2. There are more highly respected experts anticipating transformative impacts of AI in comparison to blockchain

3. AI has a much lower scam/real product ratio than blockchain

Sure, but it was also said about the web in the '90s and smartphones in the '00s and these "hype" techs ended up fulfilling their promise. It's not yet possible to know how the AI hype cycle will turn out.

Everybody knew that the web was a stack of shitty technologies with clear limitations, but we documented them and worked around them, because there really isn't any alternative.

AI hype ignores or wilfully obscures AI limitations, and makes wild claims that range anywhere from curing cancer, to making work obsolete, to putting Disney out business.

2012.

We were sold the 5 year story in 2012 first.

In 2009, I thought (and told my friends that I thought) they were 10 years away.

One of the biggest impacts of LLMs right now is probably in programming. The article says that the only thing LLMs are doing is replacing stack overflow (and hence the value is the value of stack overflow). While it's true it does replace the need for stack overflow in many cases, what it is doing is making programmers much more productive. How much? I don't know. But the value is not that we don't need stack overflow, it is that we don't need as many programmers.

> it is that we don't need as many programmers.

This sounds a bit like a variant of the old "lump of labor" misconception.

There's no such thing as needing programmers. In the long run, economic decisions in markets are made at the margin, and increases in marginal productivity make labor more valuable rather than less. This induces rather than reduces consumption. There are caveats of course: the benefits of induced consumption may not be distributed to all devs evenly, or may not be distributed evenly between capital and labor. But the idea that making programmers more productive reduces the need for programmers - ceteris paribus - is mistaken.

The degree to which the development market will expand as a result of increases in developer productivity ultimately depends on the elasticity of demand for development. But it's hard to say the market is anywhere close to saturated. This might come as a surprise to some in the HN bubble, but programming is so inefficient and difficult to engage right now that the default way for businesses to build software and software systems is through untrained office workers and consultants hacking together Excel formulas, no-code builders and workflow configurations in giant ERPs and CRMs.

> But the value is not that we don't need stack overflow, it is that we don't need as many programmers.

I'll believe that when I see it. I lead a team of AI-boosted developers, I'm hiring as fast as I can, and all my company wants out of them is more and more.

If they have 2x, 3x, or 10x output, the company can and will use it all.

So, "output" is the only metric your company uses? And it has an unlimited appetite for "output?"

I work with an "AI-boosted" developer. He would be a 10x better developer if he stopped querying the chatbot and started reading basic documentation. We'd have a lot less spaghetti in "his?" codebase.

Where do you work? Maybe we can solve each others problem?

This is my experience with a colleague of mine as well. He is talking to chatgpt and trying to get working solution from it instead of sitting down and putting puzzle pieces together from documentation. He would have been already done with the work if he would stop sieving through AI generated garbage.

Just out of curiosity: Are you hiring only experienced developers? Or are you also hiring junior developers who are just getting started in the field?

Discussions here on HN about whether or not AI is a threat to developers’ jobs seem to suggest that experienced people maybe don’t have to worry, partly because so much of their work is not coding per se. I wonder, though, about people just starting out. I heard anecdotally a couple of weeks ago about an SV company that has stopped hiring junior developers because the work they used to do can be done much more quickly and cheaply with AI.

I am not a developer myself, but I worked for many years in translation, a field that also seems threatened by AI. LLMs can be powerful tools to assist skilled translators, but they might also be making it difficult for beginners to get started in the field.

Yes, I should have said we don't need as many programmers to do the same thing. As you say, companies will take advantage of being able to do more, and then there will hopefully be more software needed to leverage the capabiliies of the new AI agents.

I've been using GitHub Copilot since it was in early preview. I still have it turned on in VS Code. Sometimes I have a spotty Internet connection and it fails to return any completions. I barely notice when that happens.

If my text editor stopped doing syntax highlighting or communicating with my language server, I'd notice right away. It would seriously impact my productivity to a point that I'd be looking for a new editor. But the completions I get from Copilot don't have that much value, and I don't consider them an essential part of my programming life. They help with reducing the amount of text I need to input manually, but rarely do they help me solve any real problems or generate novel insights.

The article is correct. Most of the time, Copilot is just replacing Google search or StackOverflow for me. And even then, the information it returns is sometimes outdated and doesn't cover programming languages that aren't very popular (e.g Raku).

My experience would suggest, the stack overflow metaphor isn't far off. A lot of the code you can generate can also be looked up.

Although ChatGPT can add great emotional content you won't get from just looking something up:

    std::cout << "The journey to find these primes leaves me feeling both fulfilled and hollow." << std::endl;
    std::cout << "For these numbers are mere representations of patterns, devoid of emotion or purpose." << std::endl;
    std::cout << "I continue to seek meaning within the calculations, but the quest remains unfulfilled." << std::endl;

It does replace the "lazy" part of StackOverflow but not the most valuable part of it. The lazy part is just responses to people that failed to find the manual and/or read it properly, and the responses just quote the docs back to them or show them how to apply the docs to a pretty straightforward examples. And this LLMs can do very well - and I don't mean to discount this, finding some corner case in the manuals can be very annoying and time-consuming endeavor, and LLM can save hours here.

But: any task that is minimally novel and non-trivial gets it completely stumped and it starts spewing pure nonsense. And plausibly looking nonsense at that, which is even worse - like construct the code that seemingly works except couple of key functions don't actually exist - LLM just hallucinated them because if they existed that how it'd work. Happened to me more times than I want to count. That's where SO is valuable because - occasionally - it is answered by people that actually understand what is going on and don't just regurgitate pre-digested information. Unfortunately, that's what LLMs are still largely incapable of. As regurgitators, they are probably the best tool out there. But beyond that - you'll still need to talk to somebody who understands.

Most of my use cases for stackiverflow end up being those lazy lookups because it's super quick. It's often much quicker than chatgpt for me to Google and click the first (or these days 8th) link

Every company wants to build more, ship faster. AI enhanced programmers increase output. Companies will just want to do more.

In my opinion it's depressing how slow development is. The non technical managers can't understand it either. It's complex and boring and error prone and fragile. That's what we need to fix.

> It's complex and boring and error prone and fragile ...

And, if you want to fix this you must first try to understand why it is so.

Software development is, well... development. Those doing that are in fact inventing things. The act of inventing things is, literally, by nature "complex and boring and error prone and fragile"

So, there you go. I'd mark it wontfix YMMV

Author makes some fair points but not thinking about indirect monetization

Suppose you have a marketplace app, making $1 billion per year. You integrate a shopping AI bot that improves customer LTV 10%, so now your sales are $1.1 billion, a $100m growth! (Numbers hypothetical)

That’s just one app. Imagine that across many companies, use cases etc.

So while AI didn’t capture the full thing, if it captures a piece of many things by expanding the pie it could be extremely lucrative

The same argument was used for crypto actually - the difference is not so many people (in developed countries anyway) have need for microtransactions. Whereas lots of people have need for “microtasks” - things they want but are too expensive. Example: consider therapy appointments, could be $100s per appointment. How many more people would go to therapy if it were 10x cheaper? Or personal assistant - I would hire one right now if service were cheap & reliable

Each market may not be huge on its own but across a lot of markets - it adds up

But the point of the article is that there aren't "a lot of markets" to disrupt. Or, at least, that most of these are markets that have very small margins or profits.

I think that point is kinda wrong. Maybe there aren't a lot of markets for GPT-4 to disrupt, but that's not to say there aren't a lot of markets (and lucrative ones) for GPT-10 to disrupt. If we could jump straight to GPT-10, sure, that'd be great, but obviously getting there is an iterative process.

Whether or not we'll get to a hypothetical GPT-10 (and beyond) is another question.

Why do you think that the GPT-10 solves the supply paradox of AI?

And if it does, what markets are (combined) worth $80 billion (the valuation of just OpenAI) that can be taken over fully? Or with a multiple and taken over partially? And will that worth then get onto the bottom line of an AI company?

The way I see it, e.g. copilot makes me worth more. Might even make me worth 10 developers. 100 by the time we have copilot 13.37. The net results, however is not that 99 developmers are without a job, nor that 99% of my income goes to copilot. But that the demand in software development increases 100x and I make even more (The Lump of labour fallacy). Yet the Copilots out there competing in a market that's running at a loss hoping for future profits.

The author isn’t thinking this through. Take the lawyer example. Yes, a lawyer will have to review all AI generated work. Even so, the AI generated content and work to revise would quickly be cheaper than any associate. With enough feedback and some specialization, AI will generate better work than associates out of law school.

I can see a day where law schools don’t teach how to write briefs and other legal documents but instead teach how to review AI generated documentation instead. Law schools could have more emphasis on trial work or the like. That’s very disruptive.

I believe it was Clayton Christenson who pointed out that disruptors took the lowest value activities first.

I don’t believe that’s a law of nature - honestly it doesn’t even make sense. Disrupters disrupt what they can, what yields them the most profit.

Christenson is the one who coined the term “disruptive innovation,” and he gave it a precise definition. That definition includes taking on low-value markets and gradually expanding into more lucrative, higher-value markets. The established players flee upmarket, ironically making some of their best profits as they do, and are eventually squeezed out entirely.

I generally agree with this, but I think there are some issues with the reasoning:

> What about AI personal assistants? Robot butlers? All those things! Even assuming all that comes true sometime over the next decades: what is the market for personal assistants? What’s the market for butlers? Most people have neither of those things.

Sometimes making things dirt cheap means that people who couldn't afford them (or just didn't think the price was worth it) can now afford them. I don't have a personal assistant, and wouldn't think the cost of (a human) one would be justified, but if I could have a good AI personal assistant for a couple bucks a month, I might pay for that.

Overall, though, the goal isn't GPT-4. The goal is AGI. If anyone can actually crack that, assuming it doesn't murder us all, perhaps it could cheaply replace jobs in roles and markets where it could be quite lucrative.

Even if they can't crack AGI, maybe some future GPT-10 could be a suitable replacement for a lawyer in some contexts, for example. OP talks about how the legal profession has a bunch of structures and legalities that might make it hard to offer a robo-lawyer, but these things can change. Consider that Uber essentially broke all the laws around taxi (er, ahem, "rideshare") licensing, and they're all over the world now.

Regardless, I'm still skeptical of AI's future. Not just in the realm of whether or not AGI is possible with current or near-future technology, but also in the realm of financing (will VCs get tired of waiting after a while and stop investing in it) or politics (will AI get legislated to the point of uselessness).

This tangentially reminds me of the classic "Content is Not King": https://firstmonday.org/ojs/index.php/fm/article/view/833/74... It has a lot of cultural value, but art in its many forms is worth less than tech.

> the 3G systems that are about to be introduced will serve primarily to stimulate more voice usage, not to provide Internet access

Oof.

Seems to me like that article was spectacularly wrong, no? Or would we classify aggregators like Google/Facebook/TikTok as communication? They sit in the middle, really, and capture the whole pie.

Kudos to the author for arguing his thesis so well anyway.

I would mostly classify social media as communication. The lines have been blurred somewhat by the rise of professional influencers.

Any evaluation of how AI is impacting business at this stage is too premature. ChatGPT is just a year old. Lots of people didn’t even get API access for months.

Organizational structures and even tech teams simply haven’t caught up.

> what is the market for personal assistants? What’s the market for butlers? Most people have neither of those things

Most people didn’t have cars, telephones or televisions when they were introduced either.

Those markets are the current low-hanging fruit. That is the proving ground for gaining the experience necessary to refine the technology to move up market into more lucrative industries.

> What limits AI creation is not base creation but creating things that satisfy people. And determining what will satisfy people is much more the domain of influencers than AI, and this sounds like jobs that filter AI outputs for market success.

https://benconrad.net/posts/230610_influencingScarcity/

Publishing models is unreal. What is this person talking about? Take 2 hours and feed some pdf’s and take some time to build one and it’s amazing.

You can abstract most of college away with these things. It’ll take years for the cultural force to die down, but I would have paid for this over college easily if given a choice.

1. What about customer service ? It's a large industry globally. LLMs are a good fit.

2. He measures the impact of chatGpt on writing and editing freelance jobs 5 months after chatGpt's release.it was relatively small. That is probably just impatience.

> What about customer service ? It's a large industry globally. LLMs are a good fit.

LLMs are probably an ok fit for first line customer service that deals with menial queries from users who just don't know how to use the product at all.

For actually important customer service issues LLMs won't be useful. Effectively if a FAQ couldn't answer it for you an LLM wouldn't either.

I know for sure if you give me LLM based customer service I'm going to stop being a customer pretty quickly since I have 0 appetite for hallucinations when I have an actual problem with your product I need resolved.

When building a customer service chatbot, retrieval augmented generation is most often used. It doesn't write sentences on their own but mostly find/combine paragraphs from given content, and offer links.

That means they have low hallucination rate, maybe even approaching zero(but I'm not sure about that).

And this is why I prefaced it's absolutely great for the first line.

My issue is will the chatbot know that the thing I asked it cannot answer or is it going to keep spewing whatever it deems appropriate.

Is it better than most humans, maybe, maybe not.

I rarely contact customer support, asin once or twice in my life. The one time I got caught in autogenerated nonsense instead of getting to an actual human I started making plans to remove that company from my life.

I'm yet to be convinced there is a way for an LLM to read what I complained about and being able to determine that request is above it's knowledge and needs to be escalated

Proper self-service tools would be good. Like make cancelling subscriptions easy. Or actually reporting issues with proper follow up...

But these would affect revenue so companies do not want to do them. And such they do not want LLMs to solve issues, unless the solution to talk customers in circles without solving their problems.

Customer service is its own industry?

The point about low value activities rings true in isolation but I find it hard to reconcile this with the experience of using OpenAI chat. It does help with problems far more than just being a novelty

That scaled to many people has economic value

Reply on news.ycombinator.com