OpenBSD was right to disable hyperthreading [video]
youtube.com
youtube.com
To some extent, you might say they are like Cassandra; speaking the truth but not believed or listened to.
Its not really meaningful, because the firmware could be pretty much any arbitrary OS and it would make zero difference to any end user.
Tannenbaum himself didnt even know about Intel using MINIX in their ME firmware until recently, so that should show you how much relevance it has.
Not at all. MINIX was actually Intel's second choice, they tried first to fit Linux into their new x86 based ME. But the maintainers were uncooperative:
https://www.phoronix.com/scan.php?page=news_item&px=MTY4MzM
Intel then submitted similar patches to the MINIX kernel, which subsequently got accepted.
BSD licensed projects see plenty of contributions, they're just not as popular as Linux because of historical reasons. I and most BSD fans blame the AT&T lawsuit for BSD losing popularity and Linux gaining popularity. That being said, BSD is still quite popular, though somewhat niche.
Companies not upstreaming code will happen regardless of the license. Plenty of companies maintain Linux change sets because they're not obligated to release them, but plenty more upstream their changes when not strictly necessary. It just depends on the value proposition of releasing improvements.
IMO it's not that Linus has more charisma than Theo, it's simply the network effect of one project over the other.
The emergence of a GPL-licensed kernel was inevitable. If Linux didn't appear when it did, some other kernel would appear. Maybe folks would have more motivation to work on Hurd, and it would be the main kernel for everything now.
Right now, only the people worried about paying more for performance, dev time, or security engineers are listened to. We need the legal teams inside companies to have something more substantial than possible negative publicity with which to motivate the CEO and CTO as a countervailing balance.
Just like the majority of industries, we need real negative consequences for when we dump incompetent code out into the world. We've tried the "no consequences at all" plan for a long time and it's gotten us, well, continual data breaches via the easiest possible things to control. S3 buckets and databases open to the world. An inability to patch known CVEs in under 3 months (hi Equifax!).
It seems like it might create some perverse incentives as the risk escalates.
If not, why do you think harsher punishments are needed here but not for crimes?
That's where we are atm with security breaches.
That's pretty exclusively the purview of white collar crime behind a corporation though.
Street crimes have a far different cause and should be treated differently. I'm surprised I even have to type that, it seems obvious.
With that as a barrier to entry , the only solution I could see working for security is: public domain hardware and software.
The only solution I believe
I only ask because that all makes perfect sense to me, but I see a lot of negativity about GDPR on here, that all it ever does is stifle innovation and produce ever more cookie-agreement popups.
I suspect when that happens the companies will launch a massive PR campaign and fight it in court but eventually lose. If they pull out of the EU or pay I have no idea.
Edit: seems like 4% of alphabets 2018 global revenue [1] is "only" 5.44 billion dollars. Wonder if it can be applied multiple times.
[1] https://www.statista.com/statistics/266206/googles-annual-gl...
There is a real social stigma with regard to committing robbery, burglary, breaking and entering, etc. I feel like there isn't so much with online crime. As a community we really pile the blame on the victim for not be prepared and seem to give the perpetrators a pass for taking advantage of the situation.
Also, there is a real tension between anonymity on the Internet and the ability to identify perpetrators. It is a difficult tradeoff.
It's not a tradeoff we can make because the nature of computer security is that unless you fix the software and networks, you can't even identify the criminals, let alone catch them, presuming they're even in your legal jurisdiction. There's a tremendous asymmetry between attacker and defender in terms of cost+benefit, and it heavily favors the attacker.
In any event, computer crimes are punished with an iron fist in the U.S. What's not criminally prosecuted and punished very well is harassment. Yes, if social media platforms offered less anonymity, we could deal with harassment easier. But organized criminal organizations don't need the anonymity of Twitter to pilfer and fence credit card numbers; they have the anonymity of zombie networks and stolen accounts. And you can't address that with harsher penalties. If you penalized that activity with summary execution, the problem would substantially remain. And in fact in some respects it could get worse by deterring security research.
We have no choice but to fix the vulnerabilities. We have to make it more difficult to execute these attacks from a technical perspective, dramatically increasing the likelihood of identification and capture, before we can even hope of using criminal penalties as a substantial deterrent. We're a long way off from that day.
Imagine if any other field said that. "Not burning people's houses down with electrical wiring is just really hard and we're really bad at it." "Keeping bridges standing is just really hard and we're really bad at it." "Flying across the country without killing any passengers is just really hard and we're really bad at it."
You also end up creating a lot of really perverse incentives, like nefarious companies not disclosing data breaches because disclosing them would result in liability even though that's necessary for the victims to take steps to mitigate the damage. There's a reason the NTSB does no-fault investigations.
And a lot of mediocre but still harmful incentives like cargo culting decades-old security checklists to satisfy compliance requirements even though they don't actually result in improved security, but do create a false sense of security.
More than that, the problem is that humans are fallible, so even if you do 99.9% of everything right you can still make a mistake. A company with one security vulnerability can get just as compromised as a company with ten thousand. Does it really make sense to destroy OpenBSD with fines as soon as they have one security vulnerability? Or every random company that uses OpenSSH on a day that a not publicly known 0-day is being exploited in the wild? Or a company that updates to the latest version of some software that claims to have fixed a CVE even though it didn't?
The real problem here is architectural. It shouldn't be possible for someone to breach Equifax and get all your information because they shouldn't have that information to begin with. They shouldn't exist. Your data should be yours, on your device, so that it isn't possible for someone to get it by breaching a third party because the third party doesn't have it.
Also:
> and google suddenly getting religion about you being able to mass-download your data.
They had that even before the GDPR.
And the smaller-than-FAANG companies... too many checklists, contracts and theater ("GDPR requires us to disable autofill on this form") and not enough actual rethinking what they're doing and if they should change their approach to data... so we'll still be seeing plenty of breaches where they shouldn't even be having the breached data
It'll probably be a decade before we see real effect from the GDPR...
For example, if Equifax faced a fine of $5B (more than 1/4 of their market cap) instead of $500M, you can bet they'd be more serious about audits in the future. However, we've conditioned business to expect minor consequences for breaches, so security becomes an afterthought. Likewise, the $5B fine against Facebook is unlikely to change anything, though a $200-300B (20-30% market cap) fine would be much more convincing.
The point isn't necessarily to ruin companies, but to set a precedent that says these types of issues will not be tolerated. It'll force companies to get insurance, and the insurance will have an incentive to avoid collection on the policy.
It also doesn't make any sense to base fines on market cap because the two things have nothing to do with one another. All that would really do is cause corporations to restructure their operations to separate the entity that does all the dirty work from the one that owns all the assets, so that the entity that exists in your jurisdiction and is susceptible to being fined is renting/leasing everything and has only a nominal market cap, whereas the one with all the assets is a totally independent company that isn't even in your jurisdiction and never does anything "wrong" because all it ever does is lease and license things to a different entity.
It also seems kind of obvious that even if you could try to impose a fine equal to 20-30% of a company's global market cap, all that would do is cause the local entity declare bankruptcy, dissolve and abandon your jurisdiction without actually paying the fine, because that large of a fine would exceed the long-term value of operating there. Especially when there isn't any guarantee it won't happen again if they stay. For that matter it would tend to make companies not want to operate there to begin with, because it's possible to do your best and still fail, and that kind of uncertainty is precisely how you drive businesses away.
But most importantly, it still generally isn't the large tech companies who are the ones with poor security. It's the other industries, especially finance and government, that are collecting just as much data but then doing a much worse job of securing it. What does a fine mean to the DMV or OPM?
TLDR is that prophecies are self-fulfilling. Tiresias is the OG self-fulfilling prophet.
1. Cassandra is usually at the center of attention, an object of desire to Agamemnon and his troops, an object of hatred and jealousy to Clytemnestra and Aegisthus. Tiresias isn't nearly as ostentatious, and mostly exists passively in the background, waiting until someone else asks his opinion on something. Tiresias gives off a kind of awkward vibe, much like Stallman, compared to Cassandra, who's totally a social butterfly.
2. Cassandra is cool and sexy, Tiresias is a blind old dude. Richard Stallman eats stuff off his foot and often looks like he hasn't showered since the last emacs release.
3. Cassandra's prophecies are very straightforward, Agamemnon and his buddies understand what she says and even listen to her to some degree, they just don't care enough to do anything. Tiresias is much more cryptic and is always derided until the denouement when it is revealed that he was right all along, just in a way that nobody else could have foreseen. Likewise, Stallman's insights into the future of our technological dystopia seem absurd and maniacal until they inevitably come true a few years later.
I like your comment though =)
Your assessment of charisma is foreign to me. I cannot imagine anybody who is more charismatic that these two men, theo and rms.
I mean, maybe it's not too much charisma but I'd be glad if I had only like 10% of that...
But I believe grandparent didn't mean they lack charisma, but that they didn't have enough to swerve the general public.
impact = charisma * funding
funding = charisma * suitability for capitalism
1) He's not a hypocrite in any way. He's honest and you can tell that he has truly thought about his opinions.
2) In my country, I get bombarded with a ralentless stream of leftist ideology. The worst kind of leftism: the lazy 'slogan' leftism, from people too stupid to realize the full implications of their discourse. In comparison to them, Stallman is a moderate, thoughtful, sweet person.
3) I see his leftism as orthogonal to his software freedom ideology. I use and support free software, and I'm more pro-capitalism than Milton Friedman.
They aren't meant to be widely adopted, but mean, rather, as a critique of the status quo.
You need somebody as guideposts to stand at the extreme ends. I am glad RMS stands at his end.
I think that's crucial. I remember he was discussing one of the aspects of software freedom with my friend and she said "I don't agree." He answered: "No problem, you have your view, I have mine, we don't have to agree on everything."
It struck me as I had expected he'll try to convince and win her over his ideas.
Charisma means being able to persuade all kinds of individuals, regardless of their inclinations and initial positions.
He also caveats it by saying they were right for "a little bit of the wrong reasons" but at least in this clip doesn't expand on what he meant by that or what those wrong reasons were or why they were wrong.
EDIT: can't see the video so if it is just a default rather than a forced disablement that's fine.
It would be especially awesome if this could be enabled at runtime so it could be turned on when thing specific tasks, like rendering or compiling something big. Even better, it would be nice to lock that to specific processes (i.e. enable it on a few cores and lock privileged processes to those cores).
If the desktop PC has wrong default the performance is bad. Still functional though.
If in case of VM the default is wrong we will read another headline about how N million customers of $company got their personal data leaked.
I'm actually wondering if there should be some sort of premade "profiles" when it comes to default settings. Debian for example is used in a lot of contexts so perhaps it'd make sense to have a way to ask you what sort of usage you'll do when installing and provide different defaults based on that (not just at the initial installation time but also when installing some package, the default settings would be based on the profile you chose).
Defaults should be set for the lowest common denominator. A lay computer user shouldn't have to understand and make decisions upon things like this.
amount of untrusted dll injections to mod, by default, unmoddable games; 3rd party VR tools and drivers; video and audio multiplexer drivers; compatibility drivers for normally unsupported console cameras/controllers etc; ultra demanding last gen console emulators; macro tools that are essentialy keyloggers; anti-cheat daemons running as admin to read memory of other processes;
Windows gaming is wild! I literally sell my soul to gain a few more fps or immersion. Browser js looks almost too innocent in this whole mess.
If someone is patching DLLs they can figure out how to enable hyperthreading.
Hyperthreading allows you to queue up additional instructions (in a different thread) that the executor can switch to when it would otherwise be just waiting for the next instruction in the primary thread.
In making my previous claim, I was limiting the concept of 'encoding' to the more old-school definition: lossless compression of media bytestreams. This would include things like HuffYUV, ProRes, etc. But to be fair, these days it is quite likely that even a few of the newer intermediate/mezzanine codecs benefit from hyperthreading. I'd edit my post to clarify but the edit window has passed.
Hence I believe hyperthreading is an option disabled for sane defaults.
If you don't trust yourself to use the web safely, then put limits on what a naughty script can do. One option is to disable hyperthreading.
Iirc OpenBSD actively disabled it for you, making it the default.
Nobody else did that.
I haven't been following this conversation closely; is there any serious change of Linux (some distributions, or the kernel upstream) disabling HT by default?
They have, as of this commit[1] on 2018-06-19.
On the image that you're pushing out to your fleet of production servers? Maybe a couple.
If you know enough to know whether up should be using HT, you can enable it yourself.
...when is something like this going to happen to intel?
We've bought CPUs with excpectations of promised performance (like people did with emission expectations and gas milage expectations), they messed up, and we get lower speed, and now no hyperthreading and still no refunds? If i bought a 60" TV and the picture was only 50" with a black border around, i'd return it immediately... why isn't there some action regarding CPUs?
There is no such thing as perfect security. It is a cat and mouse game that will continue until the end of time, requiring ever greater resources. Therefore... all software and hardware should be free because all software and hardware is defective?
If you bought it yesterday, why wouldn't you be able to get a refund? I don't know of any major vendor that would deny you a refund on grounds that the unit is defective.
Is this a manufacturing defect in CPUs?
(The defect is baked into hard silicon out in the world, so the analogy is plausible.)
Lock manufacturers can't advertise that their locks are hardened against specific yet-to-be-discovered attacks.
Intel can't advertise that their CPUs are hardened against specific yet-to-be-discovered attacks.
They can only provide mitigations after the fact.
In case of design defects in highly regulated fields (cars), there is often a campaign to make things right. When Intel processors couldn't divide properly, they had a campaign to replace them. In this case, it looks like we're not getting much.
Intel took shortcuts to make their CPUs faster. At least some of the chip architects working on their implementation of hyperthreading should have understood that they sacrificed security for speed - without telling anyone.
And what if they didn't?
It's pretty much exactly like that. Intel has been making CPUs for well over a decade that are vulnerable to various side channel attacks, and the only thing that has changed is the community's understanding of the vulnerabilities (i.e. there's a new way to pick the lock).
Hyperthreading/SMT is a trickier issue because it had obvious and even proven side-channel potential from the beginning. But 1) everybody had to hold their nose in order to compete with Intel on SMT performance, and 2) technically the operating system communities should have made the effort to keep unrelated processes from sharing an SMT'd core. And that still needs to happen--we need smarter schedulers.
I don't agree.
Meltdown: Intel, IBM, some ARM
Spectre v1: Intel, ARM, IBM
Spectre v2: Intel, ARM, IBM, AMD
Spectre v3a: Intel, ARM
Spectre v4: Intel, ARM, IBM, AMD
L1TF: Intel, IBM
Meltdown-PK: Intel
Spectre-PHT: Intel, ARM, AMD
Meltdown-BND: Intel, AMD
MDS: Intel
RIDL: Intel
That doesn't look to me like "everybody had a rough idea about how far they could go."
It is really easy for me to believe that a ton of designers could add optimizations without consideration of side channels. Nobody appreciated the vulnerabilities that speculation introduced.
(And keep in mind Intel has probably 90+% market share in the search for exploitable behavior.)
> The problems are just too deep and pervasive
One could also say that it strains credulity that the entire community failed to realize the existence of these vulnerabilities that are so fundamental to speculation, and yet here we are - that's exactly what happened.
For example, Meltdown exposed severe negligence in Intel's design. For ARM Meltdown was limited to values of a single register, for which there's no reason to believe it was anything other than an unintentional bug--i.e. you don't get any substantial performance benefits from permitting speculation through that single register, though it perhaps simplified some other aspect of the chip.
Basically, if you go down the line Intel's issues were both more severe and pervasive, as-if they just didn't care about preventing speculation across privilege domains.
Notwithstanding the ARM's Meltdown mistake, both ARM and AMD very clearly had designs that attempted to prevent speculation across privilege domains. And they mostly succeed. The major issues are at syscalls where intra-privilege (not cross-privilege) speculation can indirectly be exploited by unprivileged callers. But like with SMT, it was always sort of understood that it was the operating system's responsibility here; there really are no good hardware mitigations.
Basically, the exploits for AMD and ARM (notwithstanding the lone register issue) are intrinsic to speculative execution, period. And everybody sort of understood this, especially in the cryptographic community with work on constant-time algorithms. It's just that everybody was too lazy to take it seriously more generally until Meltdown/Spectre lit a fire under everybody's pants. And once they began to pay attention, it immediately became clear that Intel's designs made patently and grossly unsafe design choices.
The details on IBM Power chips are spartan. I think their Meltdown issue was similar to ARM--a bug with a register--but I can't confirm that. My impression is that Power pushed the envelope more heavily than AMD and ARM, but not like Intel. Power went all-in on SMT, though, and though SMT is fundamentally anathema to cross-privilege confidentiality, Intel's and IBM's SMT implementations seem to leak more than AMD's.
https://a.sellpoint.net/a/Qo3wL1no.jpg (via NewEgg)
https://www.intel.com/content/www/us/en/products/processors/...
Has Intel ever said "we guarantee that hyperthreads are entirely isolated from one another?"
https://www.intel.com/content/www/us/en/architecture-and-tec...
> By combining one of these Intel® processors and chipsets with an operating system and BIOS supporting Intel® HT Technology, you can:
> * Run demanding applications simultaneously while maintaining system responsiveness
> * Keep systems protected, efficient, and manageable while minimizing impact on productivity
That statement is a long way from an actual guarantee that there is no way for one logical thread to extract information about another.
We were given certain benchmark numbers and performance target and it all went to shit with a single microcode update.
Somehow most people expect CPU not to give random javascript in the Internets a private key from encrypted file system.
Intel got off so easy from that drama. Imagine a car marker selling you a car 4 seats, but when backseats are used you might lose steering? Would that be okay? No where it says you get 4 usable seats.
Kryptonite did exactly this when someone figured out they could open their U-locks with a Bic pen barrel. Full recall of vulnerable products, with free replacement, regardless of age.
That's the same as buying a car from CarCompany(TM) with an A/C and a Android Car touchscreen interface/radio/..., and an automatic android updates disables your A/C and changes the engine paramters so you have 20hp less... wouln't you expect "them" to fix it? As a consumer, you shouldn't have to worry if it's googles fault or CarCompanies(TM) fault, you should be able to take it to the dealer and have them fix it or give you a refund, or atleat 'do something'?
So there's a strong argument that Intel, which is currently marketing their chips this way, is committing fraud. Maybe it could be argued that Intel didn't previously commit fraud. But as soon as the bugs became known, and Intel continued to market their chips as having hyperthreading, from that point forward they were committing fraud.
One of the things the Linux kernel developer says in the interview is that researchers are going through Intel patents to find "security bugs".
He says this is "fun". Twice. He seems pretty nonchalant about these issues. Like it is great to fix them, but not like it is too important or anything to worry about. He says what is most important to him is that Linux "succeeds". The attitude is reminiscient of Microsoft in their heyday. Drunk on success. He even calls out a Microsoft employee he is working with. He says the companies contribute "selfishly". This is no different from BSD. Users do not determine how much security is prioritized, the contributors do. However, what happens when the biggest kernel contributors are companies?
He also indicates he does not agree with Stallman philosophically on technology issues. We do not get any details of the specifics of their disagreement.
As a user, I think is it somewhat easier to keep track of Net/OpenBSD kernel contributions than it is to keep track of Linux kernel contributions. I might be wrong on that. As far as I can tell, the biggest contributors to Net/OpenBSD kernels are still individuals and are not acting directly on behalf of corporations.
In my experience as a mathematician building parallel compute servers, hyperthreading generates more heat than it is worth. I can overclock further without hyperthreading, to more than overtake the faint advantage that hyperthreading offers at a given clock speed. So I now buy binned, delidded processors from Silicon Lottery, choosing the best reasonably priced speed of the best cpu without hyperthreading. That would today be the i7-9700 @ 5.1GHz for $400.
The vast majority of consumers aren't running compute heavy workloads that are more amenable to SIMD work (which it sounds like this might be) than the sort of highly branching, often stalled work that general purpose programs do.
OpenBSD devs are likely open to it, but there are other inefficiencies in the kernel like locking that have priority.
Two threads in a web browser that are assigned to different websites for instance.
The kernel has a way, and it is process isolation. The kernel doesn't care if you want two threads in the same process to be isolated from one another - that's your problem, not the kernel's.
Anyway, thread isolation already doesn't work even without hyperthreading-specific attacks: "we have discovered that untrusted code can construct a universal read gadget to read all memory in the same address space through side-channels. In the face of this reality, we have shifted the security model of the Chrome web browser and V8 to process isolation" (from https://arxiv.org/pdf/1902.05178v1.pdf).
My point is that the OS has no responsibility to isolate threads from one another, regardless of how many applications may try to do it themselves anyway.
If an attacker is pinned to one hyperthread, and the victim is pinned to another which isn't a sibling hyperthread, none of the spectre attacks will work since the cache state isn't shared.
As an attacker with code exec on a core, you can theoretically play games with the OS scheduler until you're running on a sibling core with your victim thread/process/vm.
The most recent round of spectre variants measured the latency of the line fill buffers and other parts which are local to a physical core's memory subsystem.
Also:
Do not run not your own code, like eg. JavaScript.
There is way too much short living code.
Why we not build software libraries as we learn using computers and then use it for the rest our life ?
Programmers should switch to programming human domain problems, not constantly reimplementing Start Menu hierarchy.
Btw. anybody uses Tripwire ? Eg. with apt-get or pacman and saving hashes to ro device ? ;)
It cast a long shadow over the viability of Berkeley at the height of the UNIX wars, and just as Linux was appearing as a completely independent and unencumbered UNIX supported by the maturation & advocacy of GNU and the FSF.
We probably would have had BSD wars for real not long after, if it had become popular. To some extent, this is by design, as there's no culture or ethos that pushes people back together.
As far as BSD being a mess after base I completely disagree. Using and understanding a package manager makes life pretty simple.
That said if the Linux community did that they would probably realize how silly containers are :)
You also need to do this for any pipeline for any OS.
Seems to me this had nothing to do with licensing or culture, but rather that if you wanted your distribution you'd use the upstream kernel and build your own userland with blackjack and hookers and whatever direction you were interested in, so there's some measure of strong relatedness in the kernel everyone shares. You might add a few modules or patches, but few people bother (or need) to fork the kernel itself.
Since BSDs are systems, if you want to go your own way you fork the entire system (that's literally the genesis of both OpenBSD and Dragonfly).
In general, *BSDs take more time to work out technical details, make sure that the design is right.
The approach in the Linux communtity is much more, release something that mostly works now and fix it later.
Little orthogonal: It sort of rhymes with the mono repository versus collection of micro repositories mindset. Does BSD have more code reuse since it is whole system?
Outside the server scope, OSX is mainly BSD with a different kernel.
FreeNAS is based on FreeBSD too.
* https://en.wikipedia.org/wiki/List_of_products_based_on_Free...
If you follow the commit logs, you'll regularly see "Sponsored by" messages:
* https://www.freshsource.org/commits.php
Not just for the core OS, but also in ports and also drivers (Intel, Chelsio, Mellonox).
FreeBSD in particular has always been persnickety about acknowledging work done on behalf of others. Something that would have prevented the IBM-SCO lawsuit if Linux had used commit/patch tracking from the beggining.
One of the huge reasons people use FreeBSD is simply licensing. If you dont want to release any source code simply build your custom app on FreeBSD and only include BSD licensed software. Makes being proprietary simple.
Note, this does not mean any companies that do this do not give back to the project, they do in the way of code commits and sometimes donations.
Isilon did not contribute back for a while, and then the FreeBSD project kept moving forward, and so the patches they kept in-house kept getting bigger and bigger, which was overhead in their development.
They've basically caught up now:
* https://en.wikipedia.org/wiki/OneFS_distributed_file_system#...
Isilon and their OneFS was a stand-alone company (like Panasas still is). EMC bought Isilon. Then Dell and EMC merged.
People like Google can't pay enough? Do all the BSD developers work as front-end quants or something? How are they all earning so much that nobody can afford them?
Yahoo! did but they also had BSD developers on the payroll.
Any Silicon Valley company will pay enough but that is why when you leave Silicon Valley or a big tech hub everything is windows. The pay attracts the talent.
Linux has become mainstream so there are more people in the talent pool to pay to support it.
Windows ... has more so it is cheaper
BSD simply doesn't have enough people who know the system well enough to support it
Paying 5 developers to build something cool doesn't mean you have the support system to run it.... You actually need people who understand your product to use it as a business system
In some ways & depending on the situation, moreso, since linux distros are often trying to do things drastically different from each other in order to differentiate themselves, whereas BSD's are often sharing code because of the small developer base, sharing-compatible license, and lack of bottom-line oriented corporate sponsorship.
Used to be a system admin could modify a kernel module in linux. These days its getting hard to find one who can use the CLI properly
https://undeadly.org/cgi?action=article&sid=20140506132000
Google also has donated to the OpenBSD foundation historically in relatively small amounts: https://www.openbsdfoundation.org/contributors.html
Meh. Probably for superficial stuff, yes. But would Linux kill cat(1) if it did some network calls? I bet not. See https://www.openbsd.org/innovations.html
I'm sure you could engineering something to kill it also with currently kernel functionality - though I would need to do more research to say what.
Those two made an effort to meet directly with business leaders, attend all the trade shows, and gave their product away for free. Their model was at least as shocking as their license; hitherto, hardly any business software was free and paired with consulting services. It caused a storm, and gave birth to a whole industry that wasn't possible under the expensive BSD model.
Solaris was when Sun made the jump to System V. AFAIK part of that was because Sun had financial difficulties at the time and AT&T offered to help them -- in exchange for Sun to rebase on System V.
[1] e.g. https://minnie.tuhs.org/cgi-bin/utree.pl?file=SunOS-4.1.4/us...
It used to be fashionable to run BSD on security focused machines (like firewalls). I'm not sure why Linux won there too.
State actors are now targeting whole populations. It takes less and less time to enumerate the entire IPv4 address space. Knowledge about hacking is becoming more and more accessible to a larger group of people. This problem is not going away - it is growing larger for every year.
If you don't believe me: Just attach an object to the internet and watch the logs.
Yep, but you have to grade it based on likelyhood. Which costs more, a thing that doesn't work right now or a thing that might not work in the future, maybe. If you can make it work now and be secure, good, otherwise people will, quite rationally, prefer that it work.
OpenBSD has much more complete and accurate documentation, which is a plus when you're debugging a problem.
I would opine that Linux won because it was familiar but not ideal - see also people trying to shoehorn Windows into embedded systems where a unix variant would be a more ideal pick.
Sure, the pool of currently experienced Linux kernel developers is large compared to the BSDs, but if you hire a smart person (like a Linux kernel developer), and provide them with means, opportunity, and motive you'll have a BSD kernel developer soon enough.
Commercial support and network effects and costs of running multiple operating systems are more likely to be a deciding factor than developer availability. If you need to run someone else's software, it probably runs on Linux or Windows. If you run a BSD for your stuff, and Linux for theirs, that means you'll probably need more people to understand both.
If you're going to run on other people's hardware, Linux is almost always supported, although there's been some movement on BSD in clouds. Like with other niche platforms, if it's not currently supported and you want to do it, you might need to do the work -- there's a lot less community to rely on to magically make things work.
It's not just about the kernel. You're talking about entire ecosystems of core utilities, filesystems, network stacks, security mechanisms, virtualization and resource-control subsystems (e.g. cgroups), performance profiling and tuning, etc. They're different systems from top to bottom, just like when I switched from 4.3 to V.2 thirty years ago.
> Commercial support
Stop right there. FAANG companies aren't going to buy support contracts. They'll hire the maintainers instead (like they did with me and hundreds of others like me at my current company). They're not going to split that effort across multiple platforms. They'll focus on one, then hire thousands of developers who don't even realize the tools and interfaces they use are Linux-specific. Then all of the wannabes will copy them, for the reasons I already mentioned. The network effect has gone way beyond any chance of reversal. Sorry.
That is the exact same question I was asked ~20 years ago regarding Linux vs Windows.
The barrier to entry is higher on *BSD then it is on Linux. But with the appropriate skills/time/energy it is very much worth the effort.
ALL of my edge devices run OpenBSD(since 2011). Most of my Internal servers run FreeBSD (90+%), with the remainder on OpenBSD. I made the decision to migrate away from Linux when SystemD was made default in Debian. In my mind they make more sense. I can grok the config files and Init process. MAN pages are much easier to understand. Network config is brain-dead simple and powerful; that's a combination that shouldn't be overlooked.
I freely admit that I'm an old-fart. My manager thinks I'm a hippy, and my co-workers think I'm a Unix-greybeard. In reality it's just this simple: It does what I want, and gets out of the way.
Therefore, bedroom hackers would be more likely to be able to install Linux on the hardware they had, not the hardware they wished they had, so they'd reach for Linux when their employer wanted some kind of backroom system that didn't cost an arm and a leg.
Therefore, there was more Linux out there beyond the explicitly techie companies, the ones who'd have already bought Solaris or IRIX or HP-UX, and the current userbase drives both features and future userbase.
...this can be a show stopper
I can run same set of docker containers that make up an app, including scaling multiple instances of each container, on (1) my macbook, (2) my ubuntu linux laptop, (3) my centos server (4) my windows laptop, (5) my other windows laptop, (6) my client's windows server.
I can do this without bothering to configure my app specifically, or without even knowing or learning what the dependencies of my app are!
BSD's are the "loner children" playing by themselves while everyone expects to have everything running everywhere and expect it by default...
Sorry, but unless effort is made to embrace kuber and docker, the niche will shrink.
One way out of the problem is to have entities like the FreeBSD foundation the OpenBSD devs invest effort into Docker development to make it capable of running BSD containers inside it... but this will never happen because of endless ego :|
writing a gui to spin up the VM flavor du-jour with one hipster command that you can show in an animated terminal in your bloated website doesn't make something cross platform underneath the hood, and that same VM runtime could run just about anything else.
and no, i don't miss the point. you miss the point - you are arguing market share as if it is technical merit. I'll take 1 true school unix hacker over 5 million 'noders' who cant debug their super hip k8s clusters they provisioned 'in the cloud' with wget|sh when they break.
There are many reasons Linux-based systems are generally much more popular than the BSDs in the server and workstation spaces. Here's why I think that happened:
* GPL vs. BSD license. Repeatedly someone in the BSD community had the bright idea of creating a proprietary OS based on a BSD. All their work was then not shared with the OSS BSD community, and the hires removed expertise from the OSS BSD community. In contrast, the GPL forced the Linux kernel and GNU tool improvements to stay in the community, so every company that participated improved the Linux kernel and GNU tools instead of making their development stagnate. This enabled the Linux kernel in particular to rocket past the BSDs in terms of capabilities.
* Bazaar vs. Cathedral. The BSDs had a small group who tried to build things elegantly (cathedral), mostly in "one big tree". GNU + Linux were far more decentralized (bazaar), leading to faster development. That especially applies to the Linux kernel; many GNU tools are more cathedral-like in their development (though not to the extent of the BSDs), and they've paid a price in slower development because of it.
* Multi-boot Installation ease. For many years Linux was much easier to install than the BSDs on standard x86 hardware. Linux used the standard MBR partitioning scheme, while the BSDs required their own scheme that made it extremely difficult to run a BSD multi-boot setup. For many people computers (including storage) were very expensive - it was much easier to try out Linux (where you could dual-boot) than BSDs. The BSDs required an "all-in" commitment that immediately caused many people to ignore them. I think this factor is underappreciated.
* GNU and Linux emphasis on functionality and ease-of-use instead of tiny-ness. GNU tools revel in all sorts of options (case in point: cat has numerous options) and long-name options (which are much easier to read). The BSDs are often excited about how small their code is and how few flags their command lines have... but it turns out many users want functionality. If tiny-ness is truly your goal, then busybox was generally better once that became available circa 1996 (because it focused specifically on tiny-ness instead of trying to be a compromise between functionality and tiny-ness).
Some claim that the AT&T lawsuit hurt the BSDs, but there was lawsuit-rattling for Linux and GNU as well, so while others will point to that I don't think that was serious factor.
Here's one article discussing this:
https://www.channelfutures.com/open-source/open-source-histo...
For me, I ran FreeBSD for about a year around 1999. I lasted about a week before breaking down and installing a GNU userspace. Excellent CLI ergonomics for the day.
Linux had quite a few "easy to install" distros where, if your critical hardware was fully supported, you had something that was easier to get up and running than Windows 95/98. X configurations and sound drivers were a sticking points back then though. BSD had no such "easy mode" gateway drugs.
That's the first time I've ever heard GNU associated with the bazaar. The original thesis of the book (CatB) is the observation that GNU is a cathedral (with saint rms at its head) and Linux (the kernel) is a bazaar.
Do you mean the SCO lawsuit against IBM? Because I would argue that 1) that happened long enough after Linux had established itself that people were too invested to be immediately scared off and 2) people put a lot of faith in IBM and there legal team to defend Linux. I think the community around Linux was able to basically laugh the whole thing off once SCO started presenting their actual "evidence".
Without a large company to defend it and the fact that no one had really started using it for anything serious yet made the AT&T lawsuit against the BSD's look a lot scarier at the time.
All good points by the way, but I do think AT&T rattling their sword did have a pretty chilling effect on BSD adoption as well.
No, pretty sure the parent meant the the USL vs BSDi one. Though I disagree with the parent, and believe it _was_ impactful on BSD adoption.
https://en.wikipedia.org/wiki/UNIX_System_Laboratories,_Inc.....
I don’t remember anything before SCO so I was curious if there was something before that I missed.
I must admit I was downright obsessed with that case and Groklaw’s coverage at the time so I guess it’s not surprising that it was the first thing I thought of...
The SCO vs. Linux/IBM/the universe travesty that attacked Linux came a little later; that started in 2003 and seems to never really end. There were also legal accusations around that time (circa 2004) about Linux raised by Kenneth Brown (from the Alexis de Tocqueville Institution). Both were focused on Linux and not on BSD, and both failed to slow down Linux.
Reasonable people can definitely disagree on whether or not the USL vs. BSDi legal case seriously impacted BSD adoption as compared to Linux. I don't think it was a key factor in the long run. The USL vs. BSDi case was raised in 1992, and resolved in 1994, so the legal issues didn't stick around very long. In addition, the BSDs had a head start and plenty of time to recover after the legal issues were resolved... if legal issues were the only problem. But again, reasonable people can disagree. We'll need two universes, where it did and did not occur, to really answer the question :-).
The SCO lawsuit was a joke and everyone knew it. A bare shell of a company, a mere coat rack they could hang a lawsuit on, was going up against IBM with evidence it wouldn't even release for an embarrassingly long period of time, and when it did, it was laughed out of Slashdot and Groklaw. Microsoft really didn't get its money's worth out of that little venture.
As for lawsuits against GNU, I don't know of any off the top of my head. Can you name one?
When the AT&T lawsuit happened it had the direct effect of steering people from *BSD to Linux at a critical time for both, so I'd say that it was definitely a factor.
It spread FUD and prevented people from developing on BSD, and steered them towards Linux.
This is from memory, maybe someone else can fill in the details.
Linux just had "more stuff". Without any other particular reason to pick one over the other, most people picked the one with "more stuff". Nobody wanted to install an OS for a server just to find out it couldn't run the latest software, or didn't have the latest drivers. In addition, the userland of the BSDs was different from GNU tools; if you were already installing GNU tools on every other OS you adminned, you might as well run the OS that's based on it. Finally, if you wanted Enterprise support, Linux was the only Open Source choice, afaik.
Some at the timed claimed performance benefits from one or the other, but various benchmarks showed Linux and BSDs each had their respective performance strengths that could generally be overcome by tweaking.
https://marc.info/?l=openbsd-misc&m=156700281107546&w=2
The most recent one I saw gave a reasonable sense of some tradeoffs:
https://marc.info/?l=openbsd-misc&m=156750048426578&w=2
I'm sure they welcome donations. :) Software they have written have benefited many, directly or indirectly (like OpenSSH).
...but the whole thread was interesting I thought.
Is anybody actually attacking AWS this way? It seems a promising attack vector; you get to run your own code on the same machines as others.
[1] https://aws.amazon.com/ec2/instance-types/#instance-details
So... there's more to come.
The important thing to remember is that I am intentionally making those choices and considering that trade-off. Most users are not. It is somewhat better to leave things like ASLR enabled and hyperthreading disabled by default, because that is what is best for the average user.
Lastly, it is of not that if something such as this is disabled and there is a resulting security breach, every one will run around screaming, "Linux is insecure; ahh!" This will happen, even if the user is informed he is supposed to do this for things which need to be secure.