MacOS 10.12 Open Source Components
opensource.apple.com
opensource.apple.com
The general idea is to have a site that can track various projects that release their own copy of GPL (especially, but other licenses as well) software. So Apple ships their version of {bash, etc.}, and so does your router, and someone else's refrigerator. And this site ("GPL Hub?") just links to all the places that are hosting their own version of {bash, etc.}. Maybe it could even show how they've diverged, although that might not be so easy.
Recently it has become very trendy to host all code in one centralized location like GitHub. But GitHub isn't really that great at showing you the "forks graph" anyway. Many companies that ship GPL code have archives like this available on their own websites. This could help increase visibility and give users an easier time finding the source code. I'd love to know how many copies of different GPL projects are out in the wild. Maybe for the companies that don't publicly host their GPL code, someone who makes a successful GPL request could share the code on this same site.
Those are some of the ideas, I'd still love to finish the post sometime but it's a bit too vague for me to say anything concrete yet. Any direction to related works or explanations of why this is a bad (or good) idea would be much appreciated!
Happy to chat more. Details are in profile.
It could also work as a (perhaps curated) reference. Sort of like Rosetta Code, but for more high-level features – “how have projects X and Y chosen to implement this feature?”
Besides, it could serve an archival function. A quick search on the web reveals a couple of projects on this vein, but they tend to rely on storing official releases. The way I see your idea, it could be made to work in a way closer to what archive.org does for the web, archiving everything it’s permissible to, with the addition of meta information about the projects, their relationships, etc.
Not a small undertaking, but a very inspiring one!
Looks like they've only released the x86_64 and x86 versions of the code? I don't see any ARM in there.
Anyways, found gems like this [1]:
/*
* File: sched_prim.c
* Author: Avadis Tevanian, Jr.
* Date: 1986
*
* Scheduling primitives
*
*/
[1] https://opensource.apple.com/source/xnu/xnu-3789.1.32/osfmk/...
> to the HAL by the NT god. Hail Caesar!
../tdi/tcpip/tcp/tcpsend.c:// STUPID FUCKING COMPILER generates incorrect code for this.
../fastfat/allocsup.c:// God knows what state we left the disk allocation in.
../dd/histgram/cmd/perfhist/perfhist.c:// the heck? we are gods anyways!
../nthals/halntp/mips/xxinitnt.c:// to the HAL by the NT god. Hail Caesar!
../w32/ntgdi/client/dcquery.c:// if cjotma == 0, this is HONEST to God unicode font, can not convert
../w32/ntgdi/fondrv/tt/ttfd/fontfile.h:// honest to God msft unicode font
../w32/ntgdi/fondrv/tt/ttfd/tt.h:// uint16 reservedPad; // only God knows why
../se/ctlpcqos.c:// Appease the compiler Gods.
../cntfs/create.c:// Here is the "M A R K [REDACTED]" hack from hell.
../afd/disconn.c:// Hack-O-Rama. TDI has a fundamental flaw in that it is often impossible
../afd/disconn.c:// Yet another hack to keep it from crashing
../fastfat/read.c:// Deal with stupid people who open the volume DASD with
../afd/recvdg.c:// Bomb off if the user is trying to do something stupid, like
/* We don't support deferred ASTs; everything is candycanes and sunshine. */
/*
* Author: Avadis Tevanian, Jr. */
*/
But I'd argue that beyond code, probably his most important Apple role was establishing a steady release schedule. After the disasters of the Pink (later spun off as Taligent), Copland and Gershwin megaprojects, Avie managed to get Apple to focus on making small steady releases.
And while MacOS X was a megaproject, it too was released in steady steps: Rhapsody, Mac OS X Server 1.0, MacOS X Public Beta and then finally Mac OS X 10.0
Here are some hints of what they are doing to process the original sources:
https://opensource.apple.com/source/hfs/hfs-366.1.1/make_ope...
# Check for remaining bad file names
BADFILES=`find . \( -name "*.arm*" -o -name "arm*" \) | xargs echo`;
...
# Check for remaining bad file contents
if grep -iEr '([^UD_]_?_OPEN_SOURCE_|XNU_HIDE_SEED|XNU_HIDE_HARDWARE|CONFIG_EMBEDDED)' .; then> MIRROR ONLY, UNOFFICIALLY
Someone took it and uploaded it to GitHub. This was not published by Apple. Also has not been updated in years.
They did publish an updated Darwin core / kernel a few releases back, but it seems to be closed again.
Too bad, I think someone with some determination could have gotten am hypervisor based container tech running on this. Of course the test would not run, but APIs shims could have made whole software compile (but not link). This in itself would have been very useful for a lot of people. Being able to compile in a container then link in the "real" macOS could also have been implemented, making this much more useful.
[1]: It has been open sourced in the past:
https://torrentfreak.com/microsoft-takes-pirated-windows-nt-...
http://www.theinquirer.net/inquirer/news/1030335/leaked-wind...
I've built the WinXP kernel a few months ago just for nostalgic purposes, here's a VM running the custom kernel with a KDBG hook on NTQSysInfo [1]. The neat thing is that you can debug on the system itself, or via a serial port.
I also know that some people had success merging the /base/ code from the above licensed kernel (but the x86 version) and the leaked WinNT4 code. One even got so far as to manage a recompile of the NT setup, building a custom NT OS from scratch. Though that's violating the WRK and NT license left and right.
This isn't just pedantry, we are scientists from different countries, with many of us having learned English as a second or third language (I'm from Romania, hi), so it's better to be precise in language.
And in this instance "open source" is a good term to be precise about, because the OSI definition specifies exactly what you can and cannot do with it. And in particular (1) you can use it for any purpose, even for evil ones and (2) you can fork it and modify it in any way you wish.
So when I hear "open source", I don't have to go and read the license and validate that these requirements are true, because OSI has already done that for me and they have lawyers that know how to read licenses. This saves me time, headaches and money.
And academic, non-commercial licenses do not qualify. Microsoft itself has introduced an alternative term for such academic, non-commercial licenses and I think it's a good one: Shared Source. Lets use it.
Most of us have some vague understanding about Licenses.
Eg: MIT/X11 specifically says "...FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT..." but it is "open source"
That is, if you earn too much money from some MIT License software, the author itself is free to sue you the way he/she can (eg, patents). One example could be dotNet from M$.
So better stick to software pieces that are licensed under GNU [A]GPLv3 or Apache v2 (If you care patent issues). Also, note that you can trust "free software" guys more than "open source" ones.
edit: Also note that this can also be a reason why Apple didn't update any GNU packages once they were relicensed to GPLv3+. (The main reason could be the GPLv3 requirement that the user should be able to replace the software if [s]he wish to. Apache doesn't have this requirement, and Apple used that for swift)
If you ignore the patent grant, sure.
That's not actually true. In law, at least in the US, there's such a thing called Estompel [1], and licenses like BSD / MIT do have an implicit patents grant [2], unless the software is also accompanied by an explicit patents license (like Microsoft is doing), in which case that license takes precedence.
Of course, you can't be totally safe with an implicit patents grant, because its applicability depends on jurisdiction, plus it's always unclear if you can also count on it for derived works, etc. But then again, this is why GPL v2 is still safe. Remember that the Linux kernel is licensed under GPL v2? So what, are we all screwed?
> note that you can trust "free software" guys more than "open source" ones.
With all due respect, but many "free software" guys tend to be zealots and I can't take them seriously anymore. Not when the FSF starts doing stupid shit, like preventing Emacs on MacOS from supporting emoji just because the "free operating systems" would be at a disadvantage. Talk about crazy.
Also, in my opinion AGPL does not pass the OSI definition and it was probably accepted only because it would have brought out the crazy, but that's a personal opinion.
[1] https://en.wikipedia.org/wiki/Estoppel
[2] http://en.swpat.org/wiki/Implicit_patent_grant
[3] https://github.com/emacs-mirror/emacs/blob/emacs-25.1/etc/NE...
Those in the scene may recognise the names Roy and Wengier, 911cd, and the China DOS Union.
Here is an interesting article from those days: http://pasotech.altervista.org/mirrors/michael_kh_au-yeung/n... (file links are mostly dead though)
I am really hoping that you were joking.
https://opensource.apple.com/source/IOUSBFamily/IOUSBFamily-...
They re-wrote IOUSBFamily (or at least they said they did) and it's buggy as all heck whenever you try use a USB 3 hub (since the 10.9 rewrite).
I'd fix it myself, but it's no longer open source. Plus, to really rub salt in the wound, there's no longer a USB debugging kext. Yay, Apple...
I also don't think a power problem on an attached USB device should give my computer a kernel panic.
It was often compacted to MacOS for various reasons, like brevity, keyword searchability and tagging.
If it was really meant to be "OS 10", then they wouldn't have versions like "OS X 10.6", they'd just have "OS X.6" or "OS X.VI". How many people do you hear saying "OS ten ten point six"?
"macOS", for all its initial lowercase goofiness is at least understood by most users.
You'd think Apple would want to invent the next cool kids bandwagon, maybe invite some community goodwill and crowdsource maybe some new features like fixing time synchronization to have a high-precision ability or working on another actual replacement for HFS+, or adding Windows compatibility. Maybe developer pressure could sway them, and wouldn't it be a good PR move to eventually relent after a long, drawn-out public debate which buys some time for the Apple Car or whatever else is cooking next to be readied? Ive and Cook have massive shoes to fill, which they and the Apple team helped fill, but many people don't think that right now, at least not yet.
The majority of open source is crap, like 80% of it. People build what they need and forget about it. Often the code has no cohesion to the rest of the project and often it is of poor quality. Not to mention the ego's and lack of sociability of most projects.
Apple didn't become the most valuable company by following that principle. Instead it took the approach of an egotistical maniac who demanded perfection and required that all the pieces fit together as part of an elegant puzzle. No small feat and not really replicated today -- except for Tesla and SpaceX.
If I'm missing something, please name one other company today has has a "hit" product that unanimously loved (aka has insane sales and really high ratings) to the extent of the iPod, iPhone, Model S, etc..
Furthermore, I'd challenge you to name one open source project that can be shipped AS-IS as a product. Don't even say the Linux kernel, that has like what 2% market share?
No doubt the majority of HN folks are sharp but there is a huge lack of emotional intelligence. I've found this at many tech companies except Apple. Even when talking with Apple engineers I find myself talking to another HUMAN who happens to appreciate technology and understands it isn't the be all end all. Its sad to see such a company lose it soul but it was inevitable.
Disclaimer: I'm a kernel developer and contributor.
Off topic, but Tesla and SpaceX are definitely not "egotistical maniac who demanded perfection and required that all the pieces fit together as part of an elegant puzzle"
GitLab. Postgres. Atom. Docker (debatable). I could go on, but to be fair I am a bit of a hoarder of interesting oss.
Re the Linux kernel: isn't it way higher than 2% if you count commercial and industrial usage? Also Android devices.
For the NDK ones, using the set of official APIs would be also enough.
The only ones that would fail, would be the ones that try to bend the rules of official APIs.
87% on mobile. Surely majority on servers. Lots in routers etc.
I don't reslly see your point, though. If something is available for free, obviously it won't sell a lot. If that must be your definition of success, it seems you just have a preconcieved notion.
Also, ovviously 99 % of all open source is garbage. But who cares? No one is forcing me to use it all.
Using Java and a subset of POSIX APIs that make it irrelevant which kernel Android is actually using, to anyone but driver writers.
Google likes this so much that Brillo has even less layers and Fuchsia is being built with their own inhouse micro-kernel.
How are you a kernel developer and then go on to only count desktop and laptop installs? Linux installs dwarf the other major kernels. Not big on desktop, but big on servers, embedded, and mobile.
You might be interested in fact, that even Model S runs on Linux kernel ;)
They don't care. Objective #1 is selling more iDevices. Objective #2 is selling more iDevices. Objective #3 is selling more iDevices. Everything else is a mean to that end. Take Swift: it only exists to make it easier to build iOS apps, so that they can sell more iDevices. APFS is being introduced so that flash-based systems can work better, so that they can sell more iDevices. And so on and so forth.
That's an admirable level of focus, mind; just don't expect they do anything because it sounds cool to geeks - Apple just doesn't work like that.
Wonder where that's used. Scripting in elisp?
Google as well.
If you want your code to be usable by people who don't know what "fsf" stands for, don't use gpl3.
I originally intended to point out more how people who use and invest in Mac OS X and claim they are for FSF are, in some part, contradicting themselves.
Instead they should actively contribute source-code to the BSD-ecosystem. Apple has a lot to gain long-term from the BSD-ecosystem being healthy.
Also, the whole Hackintosh thing is completely overblown. So what if some intrepid coder gets it running on a non-apple machine. It won't affect their sales the tiniest bit.
BTW, how useful is this code? Obviously you can't build anything with this. It's useful for auditing, maybe?
The idea is for Apple to release the source for components that are included in macOS, and the license requires publication. There is no obligation (or, I would argue, expectation) to do anything more.
For example, Android is not Open Source to me because the code is dropped after the fact.
But fair enough. I'll take the downvotes.
It's there to meet the liabilities that they have through the licenses for the used open source software. These licenses often say that if you use the software you also need to allow the public to retrieve it. Even more so if you make modifications to it.
I'm not sure what you think "open source" is or needs to be. We were passing around tarballs over ftp for almost all of our open source software in the past.
Unfortunately, CF is always released pretty late (it's "coming soon" for months), and it would be nice if much more was Open Source.
But I'm happy about every bit of code that's public!
I suggest editing the title to the following: "Open Source Components of macOS 10.12"
Of course, being open-source, you're more than welcome to put a copy there.
Think about what happened to SourceForge, and be glad that the Internet hasn't become totally centralised around a few major websites yet.
GitHub going down would be a major inconvenience.
If I asked a colleague at work if a project was properly backed up and they said 'well there's an enterprise backup solution that is appropriate for it', I'd metaphorically kick their but into next week. I'd want to know that they are actually backing it up and that the backups have been verified, not that they could be backing it up.
If you'd said something like 'there are options available to mitigate the loss of github and projects should be taking advantage of them, and here they are' that would be a good and useful comment, but I don't think a practical concern can be reasonably dismissed with a theoretical counter. I find it dismissive and lazy.
Now, an outage at github would be difficult because you would not only have to get your stuff available again, but you might also have to track down a long list of dependencies that are also hosted at github.
some people say "yes", some people say "nonsense".
It works really well, but, if it fails... there's plenty waiting right there to take it's place.
So, without further ado:
- obviously, services like Search, Youtube, Gmail are unavailable. That includes searching from the address bar in your browser ;)
- people, who have their DNS set to 8.8.8.8 should reconsider their settings,
- too many web pages use assets hosted on Google CDN, including analytics or fonts, and when unavailable, the pages will load very slowly, waiting for timeouts from CDN.
- Android phones will say, that Internet is unavailable, even if it is.
Otherwise, everything is fine.
The tool they used to create it is here: https://github.com/n-b/tatin
Not sure if it still works, but it's was pretty handy for doing diffs, etc...