Hi HN. I'm an independent researcher and I've been working (completely self-funded) on my inference engine since January 2025. It's built from scratch and uses highly optimized fused kernels and I've been testing it against vLLM and SGLang. Thank you, Victor
Damn. I just bought a maxed out MacBook Pro M5 Max 128GB 8TB, still waiting for it to be delivered. I could get 256GB RAM M5 Ultra 1TB for roughly the same price, and it's double the memory bandwidth. Which one would you recommend? I do plan to run local LLMs.
I bought a server with 768GB RAM a couple of years ago, for cheap. Today just the RAM itself is worth more than I paid for the entire server. DDR4...
Same thing with SSD, two years ago you could buy 4TB SSDs for like 250 USD, and today its more than $700. Madness!
The problem is that 95% of the products on Amazon are just chinese garbage. Most people do not research the products they are buying, they are buying because of the convenience. The "default" quality of products on Amazon are utter complete overpriced shit. Even if you have done your research and ACTIVELY look for a specific locally manufactured product on Amazon you're flooded with sponsored products with unpronounceable names. There's a whole hidden industry about Chinese companies have a streamlined process to create a shell company, apply for various trademarks to skirt rules even if their product is obviously copied. It's horrible if you look into it. I try really hard to only support companies that manufactures products in the US/EU/UK. You should do, too! It's good for the economy.
reads like Salesforce to me, ugh! Enterprises are paying so much to blatantly vendor-lock in themselves using hundreds of "Salesforce engineers". It's baffling to me.
It’s a cost I’m willing to pay. You’re paying for a fully domestic supply chain. The cotton is grown in the U.S., then cleaned, spun into yarn, made into fabric, dyed, and finally cut and sewn in Los Angeles. Every step happens in the United States, supporting local farming, manufacturing, and labor.
You can notice the quality difference when things are actually built to last. I work a lot with electronics and need good work-holding tools. I didn't use to focus on that and just bought cheap crap. I then heard of PanaVise (made in Reno, NV) and got their tools, absolutely SOLID. Over-built, heavy and built to LAST!
Support brands with values and local manufacturing. For example: American Giant, Origin, Crye Precision, Randolph Engineering, American Optical, and many more.
There’s a lot of wasted compute with stateless inference. We set out to solve that with a new computational model for transformers, and only process the delta between requests. That’s how we can achieve crazy low latency tool calling with LayerScale. Check it out https://layerscale.ai and technical whitepapers out next month!
You’re not alone. I’m a self-funded startup founder and I still buy Made in USA goods (clothes, appliances, tools, supplies, equipment, etc). For me the price isn’t the main factor, it’s simply that I want to support the countries I like. Been doing this for 10 years now. Based in London so I also buy Made in England things too. Never ever support authoritarian regimes!
I never ever buy any crowdfunded product, because the moron that makes it always offshores production to China. I support local businesses, with a local supply chain. Please do that too, it’s how we get a better future!