Comment by jazzyjackson | Hacker News Reader

Comment by jazzyjackson | Hacker News Reader

Controlling [unjustified futurist hype's] Growing Energy Needs

Haven't seen a case study yet where a company outside of spammers and scammers increased their productivity using LLMs to do any kind of classification or unstructured-data-to-structured-data translation.

I've tried having it build up a table of contact information from email signatures, llama3 hallucinates phone numbers when the one I wanted was in the prompt, would have been better off with regex.

Looking up film developer recipes with Perplexity Pro/GPT4o, the answer comes back with a confident 1:119 ratio of HC110 for my P33 film, when I click through the source it pulled those numbers from a completely different film stock, since naturally the forum thread drifted from discussing one brand to another midway through conversation and doesn't matter how big your context window is, LLMs cant keep associations straight.

I'm going to keep trying to find uses for these garbage generators because the appearance of omniscience is so seductive, but I'm more cynical by the day, the investments and nuke fast tracking seems to be coming straight out of faith that AGI is just around the corner.

(Yea yea I know, I'm holding it wrong)

Our engineers feed a LLM engineering plans as a quality control. Not as a replacement of any of the regular process, simply as an additional step. It discovered what would’ve been a fairly serious error in some component design for one of our energy plants that nobody else had caught. Similarity it “fact checked” one of our subcontractors on something with cable sizes that I honestly don’t understand. Finding a better option than what the subcontractor was suggesting. I say “fact checked” because you shouldn’t think of it as that since everything it outputs need to be actual fact checked.

I mostly view LLMs as advanced auto-complete for what I do and haven’t been very impressed by them, but some of the examples our engineers show us are wild.

Thanks. I actually do have a workflow that would benefit from feeding an LLM the data corresponding to 4 'stages' of an order in varying formats. Having a computer flag inconsistencies sounds feasible.

I just wrote a multilinear algebra research paper using natural language to generate all the latex. This used to take me several hour of tedious mind numbing boiler plate. I finished in in 10 minutes today.

People who aren't on board with llms as the biggest driver of growth since internet search were the type of people who still thought the yellow pages were safe from Google in 2004.

How do you generate "all the latex" in a latex document with "natural language"? What specifically did you generate? The whole research paper? What "boilerplate" are you talking about?

Are you familiar with LaTeX?

I was writing a lot of rather complex augmented matrices.

Instead of having to manually type them in with all the formatting done by hand I literally gave ChatGPT a python nested list and told it what to subscript, what to bold, etc.

In previous papers I have done this by hand if there's only a few matrices that I need to show, or by writing a script to auto-generate them in whatever format I need if there will be many.

This time I just talked to a chat bot and got it working in minutes.

Then it turns out I made a mistake at the start. The fix was just to tell it what the error was and propagate the changes. It did that flawlessly.

Yes, I'm familiar with latex. That's why I was asking.

Using an LLM to reformat/restructure your matrix data makes sense to me and this was the missing detail from your original post.

I would be too paranoid about hallucinations and opt for a script still, but that might just be personal preference.

The question is: how is more likely to make a stupid mistake and not spot it? Me or an LLM? My money is on me.

With a simple script? I'd bet on the LLM making more mistakes.

> I'm going to keep trying to find uses for these garbage generators

Is it really worth your time? Sometimes the simplest explanation is the right one. If you've already mastered your craft past a certain point and care about the quality of your work, this will only drag you down.

This reminded me of a conversation I recently had at work. Someone was turning over a project to me so he was explaining the context and history of the technical design decisions they've gone through. One stage of the project was basically linting through someone else's source code, evaluating possible code branches.

"So, we want to know all the possible states this could go through. We gave that task to OpenAI with the following prompt."

He shows me a sample prompt they would've sent to OpenAI. It started with the premise "You are a senior Ruby developer with ten years experience. Read through this source file and give me a list of all the code paths which involves global variables".

"Really?" I said, amused. I knew ahead of time that this project they were turning over is LLM-heavy. I have misgivings about LLMs but I try to keep an open mind so it was really impressive that it was used at this stage. "How well did it work?"

"It didn't. Instead we parsed it to a syntax tree and just performed static code analysis."

(Usual disclaimer that details of the anecdote were changed/obfuscated but the spirit of the interaction has been preserved.)

For all tech people, LLM are an absolute boon: it brings productivity by 10x (at least!) onto the next level, esp. for develops.

I developed an financial servic application with 1MB+ source of code (not including 3rd party libs) within 3 month - thats more than 1 million keystrokes that created a system that works perfectly fine and produces exactly the results as specified. (for comparison: my masterthesis had around 110.000 letters, and i wrote it over a couple of weeks, while the app is slightly more complicated: its a complex system where each part relies on each another, while the masterthesis is just a dumb document in which i could write in theory everything and nobody would care/check if the descript is just bs)

I’d be sweating furiously if told I was to inherit a new app to maintain and reading the text above

I'd be looking for a new job.

People like OP are the reason why there is so much skepticism about the new generation of AI.

>I just copy pasted all the answers I found on stack overflow into our live trading database at it works like a charm.

-- OP from a previous wave of computing.

No, actually not:

The thing is - i have 20+ years of IT & SWdev experience, i'm an old guy in your eyes: If you can check and QA, what the LLM gives you, you are very-pretty save.

Also, just sharing some of your current thoughts usually leads to new ideas when applying LLM; its very inspiring because they usually point you out to new contexts & ideas.

Again, that's also true for copy/pasting anything from stack overflow.

by far not! :-D

on stack overflow, i do not get "context-adjacent" ideas & inspirations - thats the difference.

Hahah, i heard that sub-tone :)

Well, the thing is, the application does only a few things and it is very well structured, i'd say - i could line out the architecture with a few sentences. For sure: To understand how the internal mechanics are working, you need to have some background knowhow on several layers.

> I developed an financial servic application with 1MB+ source of code (not including 3rd party libs) within 3 month

The timeframe + loc + financial sector seems .... Off (I'm not touching that).

Wanna join me for a code/QA flight, i'm open for external audits :)

But actually you are somehow right: Without LLM this wouldnt have been possible, e.g. The platform uses Hibernate - for this, you need to create your entity class, a mapping class and a SQL script, and these things have to be somehow in line to each another to make it work; before LLM, you had to type those files.

Its just about automating those things which you had to type earlier & timecostly on your own.

> The platform uses Hibernate - for this, you need to create your entity class, a mapping class and a SQL script, and these things have to be somehow in line to each another to make it work; before LLM, you had to type those files.

Do you really need an LLM for that? Sure LLMs can do that but they're overkill, no?

My internship and first proper job (2011-2012) was at a consulting firm that used Hibernate. They managed to auto-generate all these things with an Excel macro. I'm willing to bet some JetBrains plugin can do that too without LLMs.

This was just a very simple example which came to my mind: you could also use active mapping etc. (though i prefer the XML files), its just one example out of many: i see LLM not as a "problem solver" but as a helper/sparringpartner

> I developed an financial servic application with 1MB+ source of code (not including 3rd party libs) within 3 month - thats more than 1 million keystrokes that created a system that works perfectly fine and produces exactly the results as specified

How do you know it "works fine"? How do you know there aren't tons of caveats as soon as some boundary conditions are exceeded? Certainly not because you actually understand it all.

Just yesterday we had "Engineers do not get to make startup mistakes when they build ledgers".[1]

I sure hope it's just for your personal use because it would be completely irresponsible otherwise, to the point of criminal neglect. This is a Knightmare[2] waiting to happen and I can only hope it will only your own money that's effected and that you won't ruin anyone's lives with this.

[1]: https://news.ycombinator.com/item?id=42269227

[2]: https://dougseven.com/2014/04/17/knightmare-a-devops-caution...

Interesting why i get downvoted for appraising LLM and clapping-up that it grows my productivity? Very interesting...

Good one: I read the article about this Fintech yesterday as well :) To your questions: It works because it can see it? (For sure, every software has some bugs which may be never enconutered, but in this case its a fairly simple application, doint exactly 4 functions for me)

The link to the Knight story, thanks for pointing out on that, interesting read! Well, they are some magnitudes above what we are doing here with our own trading infrastructure, apart from that: we are using our own money on our own risk.

You are almost certainly downvoted primarly for the tone, not content. "For all the tech people" is very likely to bring downvotes. If you wrote it as a personal data point it would be fine.

Also, advocating for a general availability fintech app with 1MB of llm-generated code makes me shudder. I want my finances (x-rays, car software, etc.) to not be done that way.

If you had been clear that you made your own trading infra for your own money -- good luck! I have zero problems with this. My 2c.

Well, with "all tech people" i actually meant ">everyone working in tech (or like the HN community<", and i was serious about that: I havent met one person in the business on the tech side who didnt profit from it?

Hm, because of which wording do you imply "general availability"? Interesting take! :)

Do you mind revealing the name of your "financial servic application" so we can stay clear of it?

Its a proprietary trading app which is used for our own activities only:

For a "standard user" its too complicated, its not made for the retail world. Would be tricky to sell this, esp. as it works only with one specific broker currently.

But if i make a retail version, i will post it here :)

Reply on news.ycombinator.com