edude03

Yeah, I'm probably splitting hairs here but as far as I understand (and honestly maybe I don't understand) - Rubin CPX is "just" a normal GPU with GDDR instead of HBM.

In fact - I'd say we're looking at this backwards - GPUs used to be the thing that did math fast and put the result into a buffer where something else could draw it to a screen. Now a "GPU" is still a thing that does math fast, but now sometimes, you don't include the hardware to put the pixels on a screen.

So maybe - CPX is "just" a GPU but with more generic naming that aligns with its use cases.

> why he regretted introducing Neo4J

Even for use cases Graph dbs knock out of the park Neo4j (historically, I haven't used it in like 10 years) didn't work very reliably compared to modern competitors.

But as always it's about picking the right tool for the job - I tried to build a "social network" in mysql and neo4j, and (reliability aside) neo4j worked way better.

> I wonder if it will spur nvidia to work on an inference only accelerator.

Arguably that's a GPU? Other than (currently) exotic ways to run LLMs like photonics or giant SRAM tiles there isn't a device that's better at inference than GPUs and they have the benefit that they can be used for training as well. You need the same amount of memory and the same ability to do math as fast as possible whether its inference or training.

Feels incredibly dismissive, if you look outside your own bubble for sec, there are people who've had CC drop their prod databases, delete their home folders, uninstall system dependencies etc etc.

And yes, these are all "skill issues" - as in, if they had known better this wouldn't have happened to them, however I think it's fair to call these possibilities out to counter balance the AI is amazing and everyone should use it for everything type narratives as to instil at least a little caution.

Ah, I figured something was up - I had sonnet 4 selected but it changed to "Legacy Model" while I was using the app.

I feel like it's not intelligence or lack there of, it's that implementing sort with a[i] < b[i] is the simplest way to do it. Putting 9 before 10 would require some kind of windowing since otherwise you'd be comparing 9 and 1, and of course 1 is smaller.

> Essentially, the drives being HDD are the only real bottleneck

? on the low end a single HD can deliver 100MB/s, 80 can deliver 8,000MB/s, a single nvme can do 700MB/s and you have 4, 2,800MB/s - a 10Gb link can only do 1000MB/s, so isn't your bottle neck Network and then probably CPU?

The qwen thinker/speaker architecture is really fascinating and is more in line with how I imagine human multi modality works - IE, a picture of an apple, the text a p p l e and the sound all map to the same concept without going to text first.

So there are two ways to look at this - both hinge on how your define "consumer":

1) We haven't managed to distill models enough to get good enough performance to fit in the typical gaming desktop (say, 7B-24b class models). Even then though - most consumers don't have high end desktops, so even a 3060 class GPU requirement would exclude a lot of people.

2) Nothing is stopping you/anyone from buying 24ish 5090s (a consumer hardware product) to get the required ~600GB-1TB of VRAM to run unquantized deepseek except time/money/know how. Sure, it's unreasonably expensive but it's not like labs are conspiring to prevent people from running these models, it's just expensive for everyone and the common person doesn't have the funding to get into it.

Integrating linen[0] might be a good way to backup the messages and provide an off ramp in the meantime

0: https://www.linen.dev/

You don't need it, it's essentially a hosted version of their tool bake[0], which is actually pretty slick - without any setup you can go from a system with just KVM to running any random docker container. I can think of a few use cases for this, especially anything where you want to use containers as the delivery mechanism

[0]: https://github.com/losfair/bake

add "we're a meritocracy"

A s a technologist I think this is obviously a move in the wrong direction but let’s be real - Synology is going to be fine. The type of people who buy synology are the same people who buy Apple and are the same people who buy prebuilt gaming PCs, they’re actively trying to avoid knowing how it works so they can focus on the task to be done.

I have a DS1823 for what it’s worth, but I also have a home built NAS from ten years ago and a Ugreen running nixos. I explicitly use the Synology stock for things that just need to work

Not sure why I got downvoted - literally the first result (for me) says the best result[0] is 11t/s at Q3. Everything else is single digits, like 2-8t/s. Also considering that its not supported anymore[1] (It's Compute Capability is 6.1, not supported by cuda anymore) and it's power draw, I'd highly recommend anyone interested in ML stay far away from it - even if its all you can afford.

While the memory bandwidth is decent, you do actually need to do matmuls and other compute operations for LLMs, which again its pretty slow at

[0]: https://old.reddit.com/r/LocalLLaMA/comments/1dcdit2/p40_ben... [1]: https://developer.nvidia.com/cuda-gpus

Like 4. Definitely single digit. The P40s are slow af

Most SaaS these days are really about gatekeeping some resource (typically data, or a marketplace) so while you can replicate the gate, it's really unlikely you can replicate the resource on your own. Like random example, I could vibe code a social network, but I can't make people switch to it.

> I worry that we are being over sold on a promise that LLMs will magically make up for a lack of proficiency in an area

I saw a post on twitter about how game devs were using ChatGPT for localization and when you translated the text to English it said something like “as a chat assistant I’m unable to translate this concept” or an explanation instead of the translation.

This is exactly the sort of future I imagine with AI - not that the grunts on the ground will be sold on it but management will be convinced they can fire the people who know what they’re doing and replace them with interns armed with a ChatGPT subscription

More likely because otherwise companies would have no reason to support Safari, they'd tell everyone to download chrome like they do on desktop - then google would have no incentive to optimize chrome for iOS because what would you use otherwise?

Sounds like the roughly do the same thing as ablation - run the network in a way that’ll get the undesired result and multiply it with vectors that prevents it from going that direction

You might have better odds buying scratchers - assuming you even have $400/m to invest in an enterprise without cash flow

First time someone I know in real life has made it to the HN front page (hey sharnoff, congrats) anyway -

I think this post (accidentally?) conflates two different sources of slowness:

1) Building in docker 2) The compiler being "slow"

They mention they could use bind mounts, yet wanting a clean build environment - personally, I think that may be misguided. Rust with incremental builds is actually pretty fast and the time you lose fighting dockers caching would likely be made up in build times - since you'd generally build and deploy way more often than you'd fight the cache (which, you'd delete the cache and build from scratch in that case anyway)

So - for developers who build rust containers, I highly recommend either using cache mounts or building outside the container and adding just the binary to the image.

2) The compiler being slow - having experienced ocaml, go and scala for comparisons the rust compiler is slower than go and ocaml, sure, but for non interactive (ie, REPL like) workflows, this tends not to matter in my experience - realistically, using incremental builds in dev mode takes seconds, then once the code is working, you push to CI at which point you can often accept the (worst case?) scenario that it takes 20 minutes to build your container since you're free to go do other things.

So while I appreciate the deep research and great explanations, I don't think the rust compiler is actually slow, just slower than what people might be use to coming from typescript or go for example.

Many docker users develop on arm64-darwin and deploy to x86_64 (g)libc, so I don't think that'll work generally.

I think the student example is wrong but I'm struggling to articulate why - here's my attempt:

Students often cheat because they claim "they won't need to know this in the future", but school isn't about memorizing facts as much as learning how to research, learning how to communicate, learning to manage their time etc. When I think of someone who's "smart" by my definition, I would expect them to have all these skills that these students are (semi purposefully) avoiding.

Furthermore, - and this might be the sleepwalking bias - the conclusion that "I won't need to know this" is subtly against their best interests, because imagine for example, you spend $200,000 and 10 years to go to medical school, you "cheat" on everything, and pass only to find out, that DRs aren't in demand anymore because AI knows everything (and more that) you would have learned in school, so now the lay person just uses that instead. Wouldn't you have preferred to avoid wasting your time and do something niche that the AI doesn't know instead?

And of course - what are students typically doing instead of learning? TikTok, instagram, fantasy football, youtube shorts - all things that "we as a society" have decided are brain rot.

So it's hard to say that someone who choses things against their best interest for no real upside, who hasn't learned the skills to survive in society is smart by whatever definition

> We tell ourselves it’s substance abuse, so we can pretend it’s their own fault and couldn’t happen to us

I don't disagree that many people disregard the problems of the homeless however I want to be abundantly clear that that I'm arguing against

> by simply losing income and not being able to afford rent anymore.

It's not simply that, things have to be going wrong in your life (not necessarily your fault) that losing your income puts you on the streets. Which goes back to the gradient - for some people losing their income will have no visible impact, for some they'll be under a bridge but which one it ends up being will depend on other factors in your life (having savings, friends, etc)

> but if you give someone keys to a home the problem of them not having a home is solved.

I guess this is the heart of the debate though - is homelessness caused literally by not having a home? And bonus question - would changing zoning laws as OP suggest solve this?

I'm arguing no - more often than not there are contributing factors other than mental health/abuse/drugs that cause homelessness to be your only option that wouldn't be solved by someone giving you keys. Most obviously - food, utilities, is the house close enough to work that you can walk there, who is paying for maintenance of the house, will you be housed with people that are actually homeless because of mental health issues - and will they do things that cause your own mental health to be impacted etc.

> However, what people miss are the tons of people living in their cars, wearing regular clothes, not panhandling and showing up to work best they can.

Sure, this is what I'm talking about when I say there is a gradient. But for argument sake/illustrative purposes though - what would housing have to cost to end homeless? And bonus question - what would zoning laws have to look like to get to this price?

I imagine the answer would have to be "free" but we know for a fact that wouldn't end homelessness because homeless shelters exist that cost $0 and homelessness still exists (yes, even where there isn't overcrowding).

And sure yes, I'm sure there are people that are hanging on by their finger nails - if rent goes up $100/m they'd have to live in their cars but I'm skeptical that zoning law reform is the thing that's going to save them (or end homelessness as the OP suggests)

> If those jobs don't pay enough to afford to live in LA

There's a huge gap between "I can't afford to buy a $700,000 house" and "I live on the streets" though. Renting is an option, renting (or buying) and having room mates is an option. Not living in Beverly Hills is an option etc.

> I will move away from LA") will never be enough to fix it.

I disagree - people generally go towards where the money is. People want to be a hair stylist in LA instead of North Dakota because of the high population density and general wealthiness of the population. During covid, we saw an exodus of people out of the HCoL areas *because* the "non essential" jobs they relied on also moved out (not clear which came first, but the point remains). So generally yeah, if you can't get your hair cut in LA because all the stylists moved out of the city, you'll either commute, or move where everyone else did (which I guess is gentrification in a nutshell)

Either way. I don't think the author is talking about the average starbucks employee here

knative might be what you're looking for

bko in the sibling comment said nearly exactly what I was going to say. While I do agree that you can "fall into" this start and it's very difficult to get out of - I'm very skeptical that with no other contributing preexisting factors you wouldn't be able to figure something out. You'd have to be extremely unlikely to

- not have any family

- not have any friends

- not have any savings

- not have anything they could sell

- not have any ability to do even temp work (IE, had some traumatic debilitating accident)

- Not being able to move somewhere that's cheaper

- Not being able to take advantage of any social programs (I'm canadian, it's pretty easy for example to get a large portion of your school paid by the government)

And if _all_ of this was true, other than potentially avoiding the impacts of being homeless - how would being given a place to live (especially in a High cost of living area) fix any of your other pretty serious issues?

Now, this is a whole other philosophical debate - but why does anyone need to live in LA? I really doubt that people of sound mind are living their homes willingly to be homeless in LA. Sure - there is the crowd that think LA is where they need to be to make it - and I'm sure more than 0 of them believe in the grind so much that they do decide to be homeless on purpose HOWEVER (with no data to back it up) I'm confident that's not the majority of homeless people in LA.

I'm saying of sound mind not as pejorative to be clear but I think for most people of "sound mind" they could figure something out that's not living on the streets (getting roommates, moving somewhere cheaper etc)