Hacker News Reader

Comment by silverquiet | Hacker News Reader

It's still not clear to me that LLMs are actually useful, but I'm perfectly willing to accept that I'm some sort of neo-luddite (how deliciously ironic a term by the way). I wonder, do you think there are examples of companies that have good strategies now? And if so, what utility are they getting out of it?

View on HN

I'm in finance and we're using LLMs to summarize a ton of soft data on company performance. We're making a killing and beating the market substantially. Within the next five years I don't think there will be anyone left standing who isn't worth a trillion dollars or doesn't have all their quants augmented by LLMs.

Before this I was building a matching engine for online retail which used (then) state of the art multimodal models to decide which products would match your query to the lowest priced item. The dirty secret here is that we've solved search, it just costs too much to provide it for free. I think we'll soon see an online Costco show up which can provide this service to members and filter all the bullshit dropshippers.

I see that as a continuation of a long arc of using machines to read things in finance (not just on the buy side, btw.), don't think it will change the competitive dynamics between the more dominant players. (There might be some more concentration in some areas, but the dynamics for that pre-date LLMs or even AI/ML in some cases.)

The question is if their holdings will melt faster than they can develop the capability to use the new AI better than the small fries going after them.

So far this year the main thing slowing down development for us has been the SEC seizing our development boxes for insider trading once a month or so. Which given that each cost $50,000 has been something of a sore point.

Some of the "small fry" might get big, but similarly some of the big players will stay big - even pre-AI there have been sizable shifts in AUM.

> Within the next five years I don't think there will be anyone left standing who isn't worth a trillion dollars or doesn't have all their quants augmented by LLMs.

What an interesting comment. So, your thesis is that a few players that come out ahead will be worth substantially more than the biggest players now and they will all be using LLMs? What's driving the consolidation? High barrier to entry for making a foundation model for finance?

>I'm in finance and we're using LLMs to summarize a ton of soft data on company performance. We're making a killing and beating the market substantially.

so are you saying that you are okay with LLM hallucinations causing your company to make possible huge losses, even if rarely ?

I should mention that I am an LLM newbie, or rather a zerobie. all I have is a very high level sketchy overview of the field. but I am a dev with many years background in many areas, though not AI at all, except for reading a little bit now and then about it.

feel free to blast me, but preferably with data-backed statements.

I think customer support will be replaced or severely downsized within the next 5 years and that's a pretty big number of jobs.

But are you actually a luddite objecting to the impact that technology (AI in this case) is going to have or just a skeptic?

I honestly don’t know. AI is so young; it’s like asking who had good internet strategies in 1994; in hindsight we know some did, but nobody really knew at the time (I would argue those who say otherwise are speaking from survivorship bias).

But take a look at GitHub’s Copilot Workspaces for ideas of how transformative AI will be.

I don’t have all the pieces (if I did, I’d be rich, right?), but I think AI will deliver mass customization of goods, software, and content by reducing up front costs to near zero. So if I were looking for good strategies, I’d start with companies that currently spend a lot of money on software or content creation and who are investing in automating that.

From own experience with ChatGPT, I asked it some months ago to go through an EU regulation, and create a checklist/audit programme for IT Security. Work that would have taken me a couple of weeks, it took 'it' a few seconds. I had to go through the text to make sure it's clean and doesn't contain hallucinations. Total work effort 1-2 hours max.

This doesn't mean that the human is no longer required, but for certain tasks a company can save a lot of time/money.

So I don't know if they dream to drop a dept of 50 people and replace it with a single machine, but there are some tasks that cannot be outsourced to machines (e.g. decisions impacting the lives cannot be made by automation)

"the existence of automated decision-making, including profiling, referred to in Article 22(1) and (4) and, at least in those cases, meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject."

So good luck 'convincing' the regulator that an LLM can replace mortgage-decisions makers (it is considered a significant activity in a person's life).

If the regulations are so extensive that you would have needed a couple of weeks to create a checklist, how did you manage to see if the list was complete and correct within a couple of hours?

This was my first thought? The amount of effort I have to go through verifying what I get from gpt makes me wonder. On the other hand, there I’ve been thinking about how it might help me read datasheets. There are details that are easy to miss when skimming, that may be easy to verify through simple text searches.

That said, I know someone who almost took a job writing the actual text of laws, and they were excited about it because, while they didn’t control the laws in general, comma placement allowed them to subtly change the law’s meaning.

I currently would‘t trust an LLM to understand subtle comma placement.

LLMs are very good tools for automating tasks which shouldn't exist at all.

You mean automating bureaucracy? That would make room for more bureaucracy albeit in the form AI bureaucracy. I don’t think we’re ready for that though.

No, I really mean things that shouldn't exist at all. I see it used a lot by marketing people generating press releases nobody really cares about, online media generating images for short useless pieces meant to be filled a space between few really useful articles etc.

> generating press releases nobody really cares about, online media generating images for short useless pieces meant to be filled a space between few really useful articles etc.

I can't see any evidences that those aren't really important or useful given your comment, do you have any data to support your argument?

LLMs are tremendous force multipliers. They don't replace humans but increase the productivity of people quite a bit

That’s not a certainty but an AI salesman’s pitch.

Lived experience for me.

I've seen that claim made but I've not seen it in reality.

There are examples of where LLMs are useful for information retrieval in that they can take vague queries and produce good summaries, but they're just as likely to produce reasonable-sounding nonsense that needs to be carefully verified.

I've also seen examples where LLMs can produce code that implements well-defined toy problems but when used on a real software system with millions of lines of context they do not produce code that can remain coherent with the system.

From what I've seen LLMs have yet to graduate from "impressive demo" and everyone characterizing them as being a revolution are still describing some potential future state with no real evidence that we'll get there with this technology or any time soon.

We live in different realities then. I can't talk about details for my previous employer, but I can say there is an implementation involving LLMs saving Data Scientists a lot of time right now. I witnessed impact first hand and advised on the implementation and refinement for the group that built it.

I agree that LLMs are less useful for smaller scale use cases. I’ve found success using them for creating mock data, reformatting unstructured text into structured data, and a few other menial tasks that would take me a while to do on my own.

I think so much of the hype is about potential larger scale applications but the models just don’t seem reliable enough yet for that.

Motte: "AI will make humans more efficient and help scale small businesses and creators"

Bailey:" AI can replace your entire human labor (the most expensive cost of business) by scraping the content made by laborers on the internet without compensation"

Unfortunately, businesses are being sold the Bailey. And the sad part is that some companies aren't even pretending to hide behind the motte. I'm sure some, many businesses won't fall for the hype, but many others will and it will take a huge toll on the workers as a result.

Reply on news.ycombinator.com