9,708 karma · joined March 29, 2013
That just kicks the can down the road to "Why should we fully trust the IOMMU?"
Granted, it does defend against the vast majority of actors.
One workup indicated it was theoretically possible to modify a piece of SGLang's routing layer to support JIT predict-ahead expert swaps from Gen5 NVMe storage straight into GPU memory.
I'm hoping that proves true. The setup relies on NVIDIA Dynamo, so NIXL primitives are available to support that.
Curious if anyone's tried this already.
Yeah, the way I look at it is product managers and everyone above them in the reporting chain make more money for their respective companies the more they optimize short-form content delivery. Pretty much what you just said.
So, what we're left with is a hyper-optimized content pipeline over the years that's pretty rough to get away from when quite a large number of people are already accustomed and/or addicted. In other words it's really hard to close up Pandora's Box again, but fortunately not impossible.
>Nice chat, apologies if my response was off-putting. It was intended to be self-deprecating humor.
No worries, wasn't sure and didn't want to read into it wrong. Wasn't trying to be snarky on my end. Cheers.
>>The problem, to me, is deeper and is rooted in our education system and work systems that demand compliance over creativity. Algorithms serve what Users engage with, if the Users were to no longer be interested in ragebait, clickbait, focused on thoughtful content -- the algorithms would adapt.
Technically that's true. Thing is, the UI/UX isn't built for long-form content. The platform, interface and algorithm when taken as a whole represent more of a dopamine delivery system heavily biased towards short-form content.
That dynamic in turn ends up being deleterious to cognition to the point it ends up fighting any external factors that which could change user behavior for the better.
In other words the algorithm is part of a larger format, and that format is arguably the real drag. Of course, the algorithm being properly transparent and accountable to its users would certainly help.
Does the #1 spot confer some type of Mad Max-esque villain role?
Like you've contributed to the depletion of clean water so hard that you're put in charge of controlling the supply of Mother's Milk.
Ostensibly also given a custom desert rig with massively oversized tires, and a posse of devotees to ride with, shiny and chrome.
Stupid mainstream science.
>... and leftist politics ...
>... nor do they see that their views are political in nature.
You don't say. Personally, I respect comments that prove their own claims.
That's not the norm. We're not the norm.
I recommend against putting HN on a pedestal. It just leads to disappointment.
Candidate: That's the hotel.
HR: What?
Candidate: Where I live.
HR: Nice place?
Candidate: Yeah, sure. I guess. Is that part of the test?
HR: No. Just warming you up, that's all.
Perhaps AI time is the inverse of Valve time.
So according to you, the concept of attack surface doesn't exist. A 100MB binary is equivalent in risk to a 1KB binary. Got it.
If both are highly-audited, their risk is equal despite their size and protocol complexity. Got it.
>...its false to state that one piece of software has a “principle risk” of vulnerabilities that another piece does not.
That's like the third or fourth time you've scare-quoted the word principle. You're aware that principle and principal are two different words with different meanings?
The word I used, principal, in that context means the foremost or primary risk.
Anyways, I'm just telling you how major corporations think about it. Their underlying rationale is exactly what I've explained thus far, and hence why it's best practice.
Keep shooting the messenger I guess.
I want my C-suite and V-suite LLMs to feel like they earned their positions through hard work, values, and commitment to their company.
* = (Not to be confused with a famous poem by Dante Alighieri)
Civ III in my opinion had some of the best art of the entire series. The 3D feeling of the successor games are kind of off-putting by comparison.
WireGuard is 4k LoC and is very intentional about its choice of using a single, static crypto implementation to drastically reduce its complexity. Technically speaking, it has a lower attack surface for that reason.
That said, I've been on your side of the argument before, and practically speaking you can expose OpenSSH on the public internet with a proper key setup and almost certainly nothing will happen because it's a highly-audited, proven piece of software. Even though it's technically very complex.
But, that still doesn't mean it isn't best practice to avoid exposing it to the public internet. Especially when you can put things in front of it (such as WireGuard) that have a much lower technical complexity, and thus a reduced attack surface.
>So what’s the difference in risk of ssh software vulns and other software vulns?
I proceeded to explain how large companies think about the issue and what their rationale is for not exposing SSH endpoints to the public internet. On the technical side, I compared SSH to WireGuard.
For that comparison, the chattiness of their respective protocols was directly relevant.
Likewise complexity: between two highly-audited pieces of software, the silent one that's vastly simpler tends to win from a security perspective.
All of those points seem highly relevant to your question.
>... but thats not going to make you correct in the original question.
If you can elucidate what I said that was incorrect, I'm all ears.
Or how large companies actually think about this risk in the real world. Expose SSH ports to the public internet willy-nilly and count the seconds until their ops and security teams come knocking wondering what the heck. YMMV of course, but that's generally how it goes.
Are critical SSH vulns few and far between, as far as anyone knows? Yes.
Do large companies want to protect against APT-style threats with nation-state level resources? Yep.
Does seeing hundreds if not thousands of failed login attempts a day directly on their infrastructure maybe worry some people, for that reason? Yup.
You call it consultant distraction speak, I call it educating you about what Wireguard actually is, because in your original reply you suggested it was password-based.
>Further, they serve two different purposes so its comparing Apples to oranges in the first place.
Not when both can be used to protect authentication flows.
One is chatty and handshakes with unauthenticated requests, also yielding a server version number. The other simply doesn't reply and stays silent.
>Simple software can have plenty vulns, and complex software can be well tested.
In this case, both are among some of the most highly audited pieces of software on the planet.
For vulnerabilities, complexity usually equals surface area. WireGuard was created with simplicity in mind.
>So, the alternatives to ssh you suggest are all reliant on passwords but ssh, in the case, is based on secure keys and no passwords.
WireGuard is key-based. I highly suggest reading its whitepaper:
Hey, thanks for doing the right thing.
It warms my heart to hear that Sam is against authoritarianism. Hopefully he doesn't hang out with anyone that supports that kind of thing.
https://www.wsj.com/tech/ai/the-real-story-behind-sam-altman...
If you must, you'd typically use a bastion host that's configured just for the purpose of handing inbound SSH connections, and is locked down to a maximal degree. It then routes SSH traffic to your other machines internally.
I'd argue that model is outdated though, and the prevailing preference is putting SSH behind the firewall on internal networks. Think Wireguard, Tailscale, service meshes, and so on.
With AWS, restricting SSH ports via security groups to just your IP is simple and goes a long way.
Sure why not, what could go wrong?
"Siri, find me a good tax lawyer."
"Your honor, my client's AI agent had no intent to willfully evade anything."
Moreover, most executives don't require blackmail; they tend to go along to get along.
I've had that same dream at various points over the years, and prior to AI my conclusion was that it was untenable barring a very large, world-class engineering team with truckloads of money.
I'm guessing a much smaller (but obviously still world-class!) team now has a shot at it, and if that is indeed what they're going for, then I could understand them perhaps being a bit coy.
It's one heck of a crazy hard problem to tackle. It really depends on what levels of abstraction are targeted, in addition to how much one cares about existing languages and supporting infra.
It's really nice to see a Rust-only shop, though.
Edit: Turns out it helps to RTFA in its entirety:
>>Our approach differs in two key ways. First, we target Rust's std directly rather than introducing a new GPU-specific API surface. This preserves source compatibility with existing Rust code and libraries. Second, we treat host mediation as an implementation detail behind std, not as a visible programming model.
In that sense, this work is less about inventing a new GPU runtime and more about extending Rust's existing abstraction boundary to span heterogeneous systems.
That last sentence is interesting in combination with this:
>>Technologies such as NVIDIA's GPUDirect Storage, GPUDirect RDMA, and ConnectX make it possible for GPUs to interact with disks and networks more directly in the datacenter.
Perhaps their modified std could enable distributed compute just by virtue of running on the GPU, so long as the GPU hardware topology supports it.
Exciting times if some of the hardware and software infra largely intended for disaggregated inference ends up as a runtime for [compiled] code originally intended for the CPU.
There's also that possibility.
It certainly is at scale.
Yes, Harvard and The Lancet are just wildly political.
>Beyond the veracity of those numbers ...
In addition to being incompetent slouches.
Unfortunately, we can multiply any given figure by 0.01 and still get something that amounts to mass murder.
>... it is a political decision whether or not the US should be spending $150B on foreigners or Americans.
A proper political decision wouldn't have involved an abrupt rug pull on a bipartisan program that's been operating for the better part of a century.
There's this thing called continuity that's usually taken very seriously. Especially when hundreds of thousands if not millions of lives are hanging in the balance.