Hacker News Reader

Comment by foobiekr | Hacker News Reader

It's not just Intel. Open initiatives and consortiums (the phase two of the same) are always the losers ganging up hoping that it will give them the leg up they don't have. If you're older you'll have seen this play out over and over in the industry - the history of Unix vs. Windows NT from the 1990s was full of actions like this, networking is going through it again for the nth time (this time with UltraEthernet) and so on. OpenGl was probably the most successful approach, barely worked, and didn't help any of the players who were not on the road to victory already. Unix 95 didn't work, unix 98 didn't work, etc.

View on HN

You're just listing the ones that didn't knock it out of the park.

TCP/IP completely displaced IPX to the point that most people don't even remember what it was. Nobody uses WINS anymore, even Microsoft uses DNS. It's rare to find an operating system that doesn't implement the POSIX API.

The past is littered with the corpses of proprietary technologies displaced by open standards. Because customers don't actually want vendor-locked technology. They tolerate it when it's the only viable alternative, but make the open option good and the proprietary one will be on its way out.

> It's rare to find an operating system that doesn't implement the POSIX API.

Except that it has barelly improved beyond CLI and daemons, still thinks terminals are the only hardware, everything else that matters isn't part of it, not even more modern networking protocols that aren't exposed in socket configurations.

Its purpose was to create compatibility between Unix vendors so developers could write software compatible with different flavors. The primary market for Unix vendors is servers, which to this day are still about CLI and daemons, and POSIX systems continue to have dominant market share in that market.

Arguably the dominant APIs in the server space are the cloud APIs not POSIX.

Many of which are also open, like OpenStack or K8s, or have third party implementations, like Ceph implementing the Amazon S3 API.

Also all reimplementations of proprietary technology.

The S3 API is a really good example of the “OSS only becomes dominant when development slows down” principle. As a friend of mine who has had to support a lot of local blob storage says, “On the gates of hell are emblazoned — S3 compatible.”

> Also all reimplementations of proprietary technology.

That's generally where open standards come from. You document an existing technology and then get independent implementations.

Unix was proprietary technology. POSIX is an open standard.

Even when the standard comes at the same time as the first implementation, it's usually because the first implementer wrote the standard -- there has to be one implementation before there are two.

Nah, POSIX on servers is only relevant enough for language runtimes and compilers, which then use their own package managers and cloud APIs for everything else.

Alongside a cloud shell, which yeah, we now have a VT100 running on a browser window.

There is a reason why there are USENIX papers on the loss of POSIX relevance.

> Nah, POSIX on servers is only relevant enough for language runtimes and compilers, which then use their own package managers and cloud APIs for everything else.

Those things are a different level of abstraction. The cloud API is making POSIX system calls under the hood, which would allow the implementation of the cloud API to be ported to different POSIX-compatible systems (if anybody cared to).

> There is a reason why there are USENIX papers on the loss of POSIX relevance.

The main reason POSIX is less relevant is that everybody is using Linux and the point of POSIX was to create compatibility between all the different versions of proprietary Unix that have since fallen out of use.

Hardly as many recent TCP/IP features aren't fully exposed by POSIX socket API, rather OS specific APIs.

Security and asynchronous servers are also hardly implemented with raw POSIX APIs.

POSIX also doesn't have anything to say about hypervisors, containers, kubernetes infrastructure, or unikernels.

Nor it says anything about infrastructure written in Go, Java, .NET, Rust, C++.

> Hardly as many recent TCP/IP features aren't fully exposed by POSIX socket API, rather OS specific APIs.

It's the same API. Some of the calls have OS-specific flags that can be passed as options. The way this often goes is that the flag then gets added to a future version of the standard.

> POSIX also doesn't have anything to say about hypervisors, containers, kubernetes infrastructure, or unikernels.

You keep listing different layers of abstraction. When the host goes to open the disk image file for the guest, pretty good chance it's using open(2) etc.

> Nor it says anything about infrastructure written in Go, Java, .NET, Rust, C++.

It's extremely common for C++ code to use C system calls directly, or thin wrappers around them, e.g. to have a socket class that calls close(2) in the destructor. Likewise for other languages:

https://docs.rs/posix-socket/latest/posix_socket/

You can say this "isn't POSIX" because the POSIX standard describes using it in C, but meanwhile the same library implementation and anything using it would be able to run on any POSIX system.

Maybe it's POSIX that is holding new developments back because people think it's good enough. It's not '60s anymore. I would have expected to have totally new paradigms in 2023 if you asked me 23 years ago. Even NT kernel seems more modern.

While POSIX was state of the art when it was invented, it shouldn't be today.

Lots of research was thrown in recycle bin because "Hey, we have POSIX, why reinvent the wheel?", up to the point that nobody wants to do operating systems research today, because they don't want their hard work to get thrown into the same recycle bin.

I think that people who invented POSIX were innovators and have they live today, they would come with a totally new paradigm, more fit to today's needs and knowledge.

I think OP's point was less that the tech didn't work (e.g. OpenGL was fantastically successful) and that it didn't produce a good outcome for the "loser" companies that supported it.

The point of the open standard is to untether your prospective customers from the incumbent. That means nobody is going to monopolize that technology anymore, but that works out fine when it's not the thing you're trying to sell -- AMD and Intel aren't trying to sell software libraries, they're trying to sell GPUs.

And this strategy regularly works out for companies. It's Commoditize Your Complement.

If you're Intel you support Linux and other open source software so you can sell hardware that competes with vertically integrated vendors like DEC. This has gone very well for Intel -- proprietary RISC server architectures are basically dead, and Linux dominates much of the server market in which case they don't have to share their margins with Microsoft. The main survivor is IBM, which is another company that has embraced open standards. It might have also worked out for Sun but they failed to make competitive hardware, which is not optional.

We see this all over the place. Google's most successful "messaging service" is Gmail, using standard SMTP. It's rare to the point of notability for a modern internet service to use all proprietary networking protocols instead of standard HTTP and TCP and DNS, but many of them are extremely successful.

And some others are barely scraping by, but they exist, which they wouldn't if there wasn't a standard they could use instead of a proprietary system they were locked out of.

> The point of the open standard is to untether your prospective customers from the incumbent.

That is the point.

But FWIW the incumbent adopts it and dominates anyway. (Though you now are technically "untethered.")

> But FWIW the incumbent adopts it and dominates anyway. (Though you now are technically "untethered.")

That's assuming the incumbent's advantage isn't rooted in the lock-in.

If ML was suddenly untethered from CUDA, now you're competing on hardware. Intel would still have mediocre GPUs, but AMD's are competitive, and Intel's could be in the near future if they execute competently.

The open standard doesn't automatically give you the win, but it puts you in the ring.

And either of them have the potential to gain an advantage over Nvidia by integrating GPUs with their x86_64 CPUs, e.g. so the CPU and GPU can share memory, avoiding copying over PCIe and giving the CPU direct access to HBM. They could even put a cut down but compatible version of the technology in every commodity PC by default, giving them a huge installed base of hardware that encourages developers to target it.

If the software side no longer mattered, I would expect all three vendors would magically start competing on available RAM. A slower card with double today's RAM would absolutely sell.

> A slower card with double today's RAM would absolutely sell

Absolutely, SQL analytics people (like me) have been itching for a viable GPU for analytics for years now. The price/performance just isn't there yet because there's such a bias towards high compute and low memory.

Windows still uses WINS and NetBIOS when DNS is unavailable.

In a business-class network, even one running Windows, if DNS breaks "everything" is going to break.

Open source generally wins once the state of the art has stopped moving. When a field is still experiencing rapid change closed source solutions generally do better than open source ones. Until we somehow figure out a relatively static set of requirements for running and training LLMs I wouldn’t expect any open source solution to win.

That doesn't really make sense. Pretty much all LLMs are trained in pytorch, which is open source. LLMs only reached the state it is now because many academic conferences insisted that paper submissions have open source code attached to it. So much of the ML/AI ecosystem is open source. Pretty much only CUDA is not open source.

>Pretty much only CUDA is not open source.

What stops Intel to make their own CUDA and plug in into pytorch?

The same shit as always, patents and copyright.

CUDA is huge and nvidia spent a ton in a lot of "dead end" use cases optimizing it. There have been experiments with CUDA translation layers with decent performance[1]. There are two things that most projects hit:

1. The CUDA API is huge; I'm sure Intel/AMD will focus on what they need to implement pytorch and ignore every other use case ensuring that CUDA always has the leg up in any new frontier

2. Nvidia actually cares about developer experience. The most prominent example is Geohotz with tinygrad - where AMD examples didn't even work or had glaring compiler bugs. You will find nvidia engineer in github issues for CUDA projects. Intel/AMD hasn't made that level of investment and thats important because GPUs tend to be more fickle than CPUs.

[1] https://github.com/vosen/ZLUDA

You didn't have a choice when it came to protocols for the Internet; it's TCP/IP and DNS or you don't get to play. Everyone was running dual stack to support their LAN and Internet and you had no choice with one of them. So, everything went TCP/IP and reduced overall complexity.

TCP/IP was neither a consortium nor an open initiative. You’re missing the critical point. When TCP showed up there weren’t losers or anything else. It has no relationship to the problem I am describing.

Routing is what killed IPX/SPX and that was going to happen anyway. I am a former Netware admin and it was the post-LAN environment that killed IPX as well as pre-IP NetBEUI and all the rest.

Reply on news.ycombinator.com