JBAnderson5

To me, LLMs struggle with the most important aspect of technical writing - conciseness. As Einstein put it: “If you can't explain it simply, you don't understand it well enough”

> Opus went off for ~2 hours and wrote 5500 lines of code that (procedurally) rendered the story.

> it's an example where we go from "no one would ever do this" to "sure, why not, it's ~free".

How is two hours worth of token generation free? With tech revolutions things get cheaper/faster/better/doable, but there’s still real world limits. The advent of railroads made it feasible for the average person to cross the country, but it still cost a lot of time and resources. People weren’t crossing the country every weekend for fun just because it was now doable.

Why do we treat LLMs as ~free when we are generating things that weren’t doable before but have to invest more money than the Apollo program to build AI data centers let alone account for the operating costs?

Will they ever come out with a new mini? If they don’t, is it possible to buy a new 13 mini anymore? Or just replace the battery?

> Realistically, serving 300 users per GPU you'll spend a lifetime cost of about $133 per user, plus the datacenter/upkeep bill.

What is the operational cost and when does it become more expensive than the upfront capex?

The B200 tops out at 1000W and idles around 140W. It averages around 600W. https://www.lightly.ai/blog/nvidia-b200-vs-h100 U.S. average electricity cost is $.14 per kWh in March. https://www.eia.gov/electricity/monthly/epm_table_grapher.ph...

600/1000 *.14 =$0.084 per hour. $2.01 per day. $60.30 per month. With 300 users, $.20 per user per month. Seems fairly cheap for the electricity.

Does anyone know how to estimate colo/data center rent costs? Where did I screw up my estimates?

I’m still waiting for another small form factor iPhone like the iPhone 13 mini before I upgrade. I find it a lot more ergonomic to use

Multiple times I’ve rejected an llm’s file changes and asked it to do something different or even just not make the change. It almost always tries to make the same file edit again. I’ve noticed if I make user edits on top of its changes it will often try to revert my changes.

I’ve found the best thing to do is switch back to plan mode to refocus the conversation

> Now consider what the same analyst does with an LLM agent: "Show me all software companies with over $1B market cap, P/E under 30, and revenue growing over 20% year over year. Build a DCF model for the top 5. Run sensitivity analysis on discount rate and terminal growth."

While I think LLMs can improve the interface and help users learn/generate domain specific languages, I don’t see how a professional can trust an llm to get a technical request like this correct without verification. Wouldn’t a financial professional trust the Bloomberg llm agent that translates their request into a set of Bloomberg commands more?

Why can’t you store your agents.md file in a database?

They built it as a railroady board game instead of a sandbox video game. The rumors from their experimental workshop test and latest announcement make me hopeful for a big update in the spring. Until then, it doesn’t feel worth playing it more than a couple times through. Every game feels the same.

I’m currently working through the Linux from scratch project and think it would be cool if I got good enough to contribute to the Linux kernel or drivers.

I think part of the issue here is that software engineering is a very broad field. If you’re building another crud app, your job might only require reading a ticket and copy/pasting from stack overflow. If you are working in a regulated industry, you are spending most of your time complying with regulations. If you are building new programming languages or compilers, you are building the abstractions from the ground up. I’m sure there’s dozens if not hundreds of other sub fields that build software in other ways with different requirements and constraints.

LLMs will trivialize some subfields, be nearly useless in others, but will probably help to some degree in most of them. The range of opinions online about how useful LLMs are in their work probably correlates to what subfields they work in

I mostly agree with your assessment of the industry. However, I think there are still more new and useful products to be built. They are not “the next big thing” though. Big tech Management has been screwing this up in a couple of ways though.

1. prioritizing bets for things that could be as profitable as social media or e-commerce instead of betting on more incremental improvement products.

2. Focusing on pricing everything with reoccurring revenue and thus increasing the lifetime cost for end users instead of selling products at a discrete costs and providing end users value

3. Optimizing for growth and controlling the vision of products instead of letting small groups of talented people slowly build products.

4. Treating people as fungible resources and moving them around all the time rather than letting people develop unique expertise skillsets.

As a result, any product that can’t achieve $10+ billion annual revenue within a couple of years with a ship of Theseus team is deemed a failure and scrapped.