I think the main problem is the investment in gpus delayed the adoption of ddr5 based platforms for servers.
1,086 karma · joined December 21, 2010
To deal with the refresh loop.
I don’t think this is true. People take airplanes and rent a cars all the time. The same could be true for train travel. All of the ways someone would leave a train station generally exist.
People don’t take trains for two reasons: 1. They take too long. 2. They are too expensive.
For example sf to la takes at least ~9-13h and costs between 50-80 dollars. Versus a southwest flight for ~140 that takes an hour. For most people that extra 60 dollars for 7-11 hours is worth it.
I would be surprised if everyone got these emails, and I would also be surprised to hear they sent lots identical to this.
Wages is pretty obvious, and the article cited the changes in California. It is important to keep in mind that we’ve seen wages jump in the golden state 66% for fast food workers since 2019. That’s huge, especially since labor it’s the largest cost for these restaurants.
Delivery apps added a new fee that showed consumers weren’t as price sensitive as they thought. So I think we’re seeing some increases related to that.
The reason for this is if you take lower rent than previously your building value changes and your lenders make you cough up the difference to cover additional principal.
Since the loans come due every 5ish years we’re now seeing buildings unable to justify their previous value, and having to walk away or get valued at their current rent.
It’s going to be ugly for a while.
If you have a job, it less important. If you want to change jobs it still valuable. More and more employers want folks near by.
I feel that BYD has taken steps to enable this. The replaceable battery and swap system they developed makes it orders of magnitude easier to get the battery pack out of the car.
Polestar in theory can have the battery pulled, but I believe it involves pulling cutting the car apart. This raises the cost making recycling less attractive.
Semi analysis did some cost estimates, and I did some but you’re likely paying somewhere in the 12 million dollar range for the equipment to serve a single query using llama-70b. Compare that to a couple of gpus, and it’s easy to see why they are struggling to sell hardware, they can’t scale down.
Since they didn’t use hbm, you need to stich enough cards together to get the memory to hold your model. It takes a lot of 256mb cards to get to 64gb, and there isn’t a good way to try the tech out since a single rack really can’t serve an LLM.
The cloud provider path sounds riskier since that’s two capital intensive businesses, chip design and production and running a cloud service provider.
I’m not convinced latency matters as much as groqs material tries to claim it does.
MCR DIMM is like 1/2 the memory bandwidth that is possible with HBM4, plus it requires you to buy something like 2TB of memory. It might get there, but I'd keep my money on hbm and gpus.
I do think long term there gets to be more hope for CPUs here with inference largely because memory bandwidth becomes more important than the gpu. You can see this with reports of the MI-300 series outperforming h100, largely because it has more memory bandwidth. MCR dimms give you close to 2x the exiting memory bw in intel cpus, and when coupled with AMX you may be able to exceed v100 and might touch a100 performance levels.
HBM and the general GPU architecture gives it a huge memory advantage, especially with the chip to chip interface. Even adding HBM to a CPU, you are likely to find the CPU is unable to use the memory bw effectively unless it was specifically designed to use it. Then you'd still likely have limited performance with things like UPI being a really ugly bottleneck between CPUs.
Great idea, and love the security context, but I feel like you loose so much.
Logging for example, is it all syslog? How do you manage them? What if you need them in splunk, how does that work?
I love the idea, but can’t figure out how to use them in practice.
US companies are going to be weird about this for the next few years. They want to do the profitable thing, but have spent the last decade making workers feel like they have a say. In the end, management and shareholders are in charge, the rest is an illusion.
It seems like it would answer so many questions.
I guess the q part is the query parameter for the prepared statement. I’d still be a bit antsy here given the ease of a mistake.
I just don’t get what it’s providing over using my db’s module. Usually the dialect is close but different enough to require changes to queries.
When I look at what occurring, I think it has more to do with their cheapest labor force is in China. So the scale and profit is hard to leave.
Taiwan is definitely caught between a rock and a hard place. They watched what happened in Hong Kong and no longer is interested in rejoining Xi’s China. So now they are threatened with invasion unless they rejoin China. They are reliant on the US to prevent that’s from happening. So given the choices, it’s not surprising that they are choosing the US to continue existing.
It’s crazy to me that you haven’t see more divestments in China from Taiwanese companies.