lumost

There is a big divide between “application ai” and “model ai” startups.

Model ai startups start from OSS models, and use them extensively for different purposes as their work would usually be banned by proprietary labs.

Application ai startups don’t want to fight the model game, so they either pick the best or let the user control it.

The fear is that we can’t tell when the ai advice is bad on these subjects, and as such probably accept confidently terrible advice.

How often do managers just regurgitate ai advice rather than consulting their experts? How often does a person question an expert because the ai said so?

Naturally, the ai will be right some of the time - but it’s really hard to correct for the times the ai is wrong.

I unfortunately have to use grok via tesla. The grok voice chat is objectively decent.

I think most nuclear folks would rather divert the "big coal, and nat gas" plant building budgets to "build nuclear."

I understand the motivations for solar/wind, but there are real limiters that aren't addressed yet. Nuclear is the only option that is carbon neutral and lacks those limiters making it appealing. If I need steady state gigawatt scale power in a specific location, Nuclear is the only green option.

This is a potentially strong indicator of how institutional investors view spacex. Given that Spacex is in a high depreciation/capital intensive business, a high cost of capital and potentially difficult capital terms is problematic.

Spacex will raise more money again, they have no known path to structural profitability.

The risk here is that either AI commoditizes the software "why ask duolingo to ask chatgpt when I can do so directly?" or simply adds cost to an existing service "why does every costco employee now cost 10% more?"

So far, there hasn't been a clear win for "software wrapping an existing LLM" - if the AI is good enough, then the users can go directly to the source.

I do wonder if the productivity gains control for increased bug findings and token costs.

I think the counter point for these projects is that you may not need a deep understanding if you can measure the outcome. While this may not be true every time today, it plausibly will be in the future - making the activity worthwhile.

When I was in a large org, it was a common failure that a manager would come in from a smaller firm - and assume that our design practices etc. were meaningless overhead.

Inevitably they'd try to do a "rapid prototype" of a feature that had been tried a hundred times before and declared "too hard/unworkable/uninteresting." as they wouldn't talk to those who had done this before, or look at any past work... the invariable result was a project shut down within 6 months and the manager would be shown the door some time later.

The main takeaway from the above is to listen to people around you when you join an org, and observe the habits of successful leaders. Odds are good that at least within this particular org under these circumstances... those habits are the ones that work.

I think the theory is that they can expand the infrastructure enough that conventional fiber etc. stops being competitive.

if super chargers were always within 5 minutes, located around desirable locations to spend an hour, did not charge idle fees, and everyone had 30 minutes of free stuff to do at these locations... then it might just work!

This math is bad and shouldn't be used as a justification. It presumes that every charger is always available, that variable battery conditions and traffic conditions make reaching that charger every time, and that drivers have time to drive to the fast charger and spend 30 minutes there every time their car has low battery.

I drive a new EV in the Boston metro, I do not need to drive most days. Charging without an in-home charger is a massive pain even with Superchargers within 20 minutes drive and a 300 mile range.

A trip to the super charger takes about 1.5 hours assuming its available when I arrive, I can only make the trip when the car needs charging or I am wasting time and energy. The exact time I will need to make this 1.5 hour trip work is variable. It depends on battery conditions, traffic conditions, and what I need to do. I cannot assume that the car will be charged when I need it to be - if I need to depart on Friday evening, or unexpectedly needed to get groceries during the week - I am obliged to spend the 1.5 hours at that moment.

To avoid these problems, I generally need to charge the car when it has ~30-40% and for battery protection I can only charge to 80%. Turning a car rated for 310 miles range into one with ~100-120 miles of usable range and a 1.5 hour weekly maintenance schedule. That range could be further reduced to 50-70 miles in winter conditions. The stalls at charging station charge by the minute, so you can't exactly wonder off.

With a reliable home charger, these problems mostly go away, even a slow 5kw charger should charge the car overnight - and a typical tesla installation delivers 11kw. A renter with no ability to plugin and charge overnight will have a tough time with an EV compared to a gas car.

the counter point is that building or selecting the specialized model may cost as much as the lifetime inference costs of the task with the specialized model.

If I need to pay someone 300k to make the model and infrastructure... then I would need to process many documents to recoup my OCR costs compared to asking claude code nicely.

Perhaps the model zoo is becoming good enough that the cost to find a specialized model is not so high?

remote inference should be sufficient for most robotics applications with potentially a small model for safety critical actions running locally.

Unless you are in military robotics or automotive of course :)

The Niche model story is still fairly week. Evidence points to general models being equally capable to niche models at a more attractive capex (risk is spread across multiple verticals rather than concentrated in a single model capability)

I suspect the concern is that model serving is a stateless “simple” problem.

As yet, no one has identified a reliable moat in inference. If the moat is performance, then prices will collapse. Unlike traditional cloud moats around state, operations, and capex management - I can host a model reliably with less than 30 minutes effort.

If I understand correctly, there are more people employed in manufacturing today than there were a century ago. The hollowing of western manufacturing due to policy choices created a false perception that the sector was in decline globally.

it's a common missconception that engineers spend most of their time producing code based on documented requirements in jira tickets.

I'd believe that a complete automation of this aspect of our industry would only be enough to provide a 10-20% boost in productivity. Still impressive, but within the range of "Our team improved our CI, build times, development process etc."

so all we need is someone to leak a sufficiently large amount of claude generations onto the open and private web for all other LLMs to mimic the same marking style?

wouldn't this happen due to the massive amounts of spam/slop being released?

Context is more available locally. You can have the LLM operate for arbitrarily long periods, use your credentials to access services (if desired), store memory locally etc.

Whether such a model exists or not is a different question.

The big question for local LLMs is whether there is a 100 tok/s model which requires less than 16 GB of memory and is competitive on most tasks with the cloud models.

There is some signal that this is possible through both hardware innovation and training/data improvements.

Cloud models have their own constraints - I can’t have opus4.8 spend 4 hours on a deep research question I had in the shower without spending money. I can’t do real time video game upscaling and graphics work in the cloud period.

A laptop is about an order of magnitude cheaper than a cloud server thanks to economies of scale, uptime requirements, and other factors.

The s curve won’t inflect until it becomes difficult to allocate additional resources due to economic limitations. There is no sign that training a model on 10x the compute won’t lead to at least an equivalent improvement as the last order of magnitude increase.

If we define the Pareto frontier’s input in terms of a magic “compute equivalent unit”. We get a free order of magnitude from nvidia hardware improvements every 2-3 years. We get another order of magnitude from capital expenditure every 6-12 months. Kernel improvements to the models themselves likely yield an order of magnitude gain at some periodicity.

I don't think this is really true, there are plenty of engineers/managers who rotate through major tech companies. Many Meta folks will head off to new companies which would pay at similar levels.

This is true if you take the ai market as equal to the market for labor discounted to 5-10% penetration.

It’s not a totally unreasonable assertion, it’s the implication of the assertion that we are uncomfortable with. There is no reason for the models to stop their improvements in the near future.

The abstraction of capital and money get a bit funny when wealth is sufficiently concentrated. If there is a monopsomy (one buyer), then they can largely dictate the price of anything. If they also control violent coercion via a captured state or other means, then they can compel production at that price point.

The idea of capitalism only really makes sense when wealth is reasonably distributed such that there is still reasonable competition in both the marketplace and control of the state.

its worse at code compared to qwen 3.6 coder.

This just looks like a capex problem. There is no evidence that Anthropic has secret sauce above and beyond access to capital. If there is secret sauce, it's unclear that it changes the required amount of capital by all that much.

China will spend all of the money required to catch up, Google and OpenAI will both spend money to catch up as well. NVidia and others will not allow a frontier lab to become the AI bottleneck.

bear in mind, elon musk now has the wealth of 10k "100 millionaires" - we truly lack comprehension of how wealthy the wealthy have gotten.

I really think the environmental movements were a red herring. It was always impossible to make a meaningful dent in your personal emissions while still existing in your location. There was never any reduction proposal which could mitigate this.

Government mandates for e.g. large nuclear construction, geo-engineering, BEV adoption, or other similar proposals would have had an impact. These all exposed the real tradeoffs which would need to be accepted of cost, hardship, or whatever the opposition to nuclear was.

The environmental movements of the last 60 years focused on impossible goals which were easy to rally behind.

This is either a complete own goal by Amazon… a play to consolidate compute/model access.

Will Chinese models be allowed on the market… at all? Will startups be banned from training models of equivalent capacity?