Microsoft’s failed attempt on Debian packaging
preining.info
preining.info
Whenever anyone contributes to open source our reaction should be based on love, not hate, even if they mess up. They will eventually fix it up as opposed to deciding not venturing into unknown territory anymore.
[0] https://github.com/GeoNode/geonode/blob/master/package/debia...
(Edited to fix typo and formatting)
If people are displaying good intentions, and not wildly ignoring a load of clear instructions, then engaging with them will have a much greater chance of producing an educated, productive member of the community, and the more of those the better.
Microsoft surely has the means to hire someone who is not an eight year old using their computer for the first time.
This error is actually a good thing from point of view - it's an anecdatum that the whole* company is engaging with open source, not just the bits that know it well. This is something we should encourage and educate. The advantage of educating someone at Microsoft as that their institutional knowledge will persist some fraction of it, so that the next person that comes along is less likely to have issues.
That said, I'm pretty sure the original author knew and understood exactly what they were doing - writing a dirty hack.
On your development machine or on machines of other people all around the world? You can also gain understanding from the later, too.
I think I would have done a little bit of reading. I also think it's the norm.
I'm still willing to write worse! As a temporary hack with a JIRA issue tracking it, to unblock work, and with the intention of fixing it quickly, but still willing nonetheless.
Meanwhile the code blocks on this website lose their scrollbar and pop atop everything if you hover over them, making it near impossible to scroll right through some kind of abuse of JavaScript. Why?!? Shall I snark about incompetence too? I can't even rationalize this one to myself as a means of unblocking development like I can with the packaging hack!
The great thing about those early machines ( Apple ][+ ) is that the dynamic programming language was the OS and was the REPL and that was all in ROM, so you could tinker around to your heart's content and hit the Reset button if you got stuck. That's a context that's just fine for learning. Packaging for archives installed on other's machines aren't that kind of context.
A kid gets to play in a playroom with some toys, not in a surgical theater or a machine shop.
Essential is defined as the minimal set of functionality that must be available and usable on the system at all times, even when packages are in the “Unpacked” state. Packages are tagged essential for a system using the Essential control field. The format of the Essential control field is described in Essential.
https://www.debian.org/doc/debian-policy/#essential-packages
Diversions are addressed in 3.9: https://www.debian.org/doc/debian-policy/#maintainer-scripts
Debian Policy, generally, tells package maintainers, not users, what they may or may not do. Several classes of conflict are listed in the document; Microsoft's scrips violate multiple requirements.
The goal and result of this ststem, arrived at over 21 years collaborative development (Policy was first published in August 1996) is a system which provides well over 60,000 packages and behaves surprisingly robustly from the user-administrator's perspective. Which is to say, there are few surprises.
Microsoft have violated multiple Policy requirements. This is considered a release-critical bug in distro packages for very good reasons.
Also, even with bash installed, bash doesn't always behave like sh does.
A common example is echo:
With /bin/sh on an OpenBSD:
>>>
kakwa@openbsd ~ sh -c 'echo "print\n"'
<new line>
>>>
With /bin/dash (aka /bin/sh on Debian)
>>>
kakwa@debian dash -c 'echo "print\n"'
<new line>
>>>
This is consistent so far.
But, now, with /bin/bash
>>>
kakwa@debian ~ » bash -c 'echo "print\n"'
print\n
>>>
That's why I tend to use printf more than echo in my shell scripts.
Also, I'm not a big fan of what RHEL/CentOS/Fedora does symlinking /bin/sh and /bin/bash. It blurs the line between bash and sh, which results a lot of scripts with a /bin/sh shebang containing bashism.
We absolutely blacklist software pulling crap like this - it is a loud, blinking demonstration that the packager is incompetent and will do other, potentially less obvious dangerous things out of ignorance or impatience. Or is there a darker explanation? Untrustworthy, in either case.
* https://lists.debian.org/debian-release/2007/07/msg00027.htm...
* https://wiki.debian.org/BootProcessSpeedup#Using_a_faster_sy...
* https://wiki.debian.org/DashAsBinSh
# Delete & recreate directory <BLAH>/Patches/<patchLevel>/
# This will confirm write access to BLAH_HOME
rm -rf $BLAH_HOME/Patches/${PATCH_VERSION}_${PATCH_LEVEL}
mkdir -p $BLAH_HOME/Patches/${PATCH_VERSION}_${PATCH_LEVEL}
(and IIRC if you google the snippet, you can find where a coworker of mine posted it to Reddit to lampoon it)
https://www.youtube.com/watch?v=OkebMyDvmxs
Search also Jordan Peterson on how to tickle a baby.
That being said, aside from distribution packages, always look at the pre/post (un)install scripts for external packages.
rpm -qp --scripts <PACKAGE>.rpm
dpkg is a little less convenient but:
ls control.tar.* # <<< check that no file like that already exists
ar -vx <PACKAGE>.deb
tar -xvf control.tar.*
for i in pre* post;do printf "\n>>> %s\n" "$i";cat $i;done
rm -f conffiles control control.tar. data.tar.* debian-binary md5sums postinst postrm prerm postrm
In most cases if the script is huge, the packaging tends to be horrible and generally, the quality of the software is also not that great.
Another good indicator is the installation path:
dpkg -c <PACKAGE>.deb
rpm -qpl <PACKAGE>.rpm
If there is no effort made by the editor to follow the FHS or at least be nice and put its messy bundle inside /opt/<COMPAGNY>, it's generally a bad sign.
Proprietary software tends to be quite horrible in that regard, I'm still traumatized by the trendmicro serverprotect rpm (AV for linux).
http://downloadcenter.trendmicro.com/index.php?regs=NABU&clk... (if you want to give it a try, don't forget the service pack and the product patch...)
https://github.com/Azure/azure-cli/blob/dev/build_scripts/de...
The team released a new package a few hours ago after talking to not only others at MSFT working with Debian and Linux but especially norbusan (OP): https://aka.ms/Oxbegy
I personally spent just a few hours of the last day with the team but I think our packaging practices are coming out stronger from this. At the very least I think this team is approaching debconf and dpkg-divert and update-alternatives (and the portions of those concepts that can translate to RPM) in a much more thoughtful way. I'm sure that was also what OP intended.
(Disclaimer: MSFT employee, just in case.)
No. This is how you escalate conflict, not resolve it.
See also: Hanlon's Razor: https://en.wikipedia.org/wiki/Hanlon%27s_razor
That's a good news for free software developers. Microsoft GVFS[1] was using MIT (while git itself is GNU GPLv2) + some EULA[2]. May be they chose MIT for GVFS because GNU GPL license explicitly disallows additional restrictions (ie, EULA can't be added, otherwise it will tbe a violation of GNU GPL).
Edit: I see that the package is distributed as a single binary at https://mran.microsoft.com/download Which shall include (I haven't checked the contents) Microsoft R Open under GNU GPLv2 and Intel MKL with some EULA (ie, additional restrictions). Won't that be a violation of GNU GPLv2?
[0] https://mran.microsoft.com/faq#licensing
[1] https://github.com/Microsoft/GVFS
[2] https://github.com/Microsoft/GVFS/blob/master/GvFlt_EULA.md
Linking mkl into gpl software and distributing the binary would not be okay because you don't have source to mkl you can give people (mkl has binary blobs). When you link software like this, you need to be able to distribute the combined binary under gpl, and unless mkl has changed, you can't.
It would be fine to link it and use it locally of course, just not distribute it.
Once, I spent a good deal of my time making a grammar-based phonetic transliterator engine for an Indic language. I was mostly a python developer with some prior experience in C and plain Make files. After a couple of rewrites, I was mostly happy with the results. Then was the time for packaging and making it a usable software. The application needed to install some files under root so that linux's input subsystem (IBus) could recognize them. Also, it dependent on some system packages(both python based and non-python based) like dictionaries for generating suggestions.
As I approached the problem, I learned that I had to learn autotools and a bunch of very debian specific scripts which were pretty much focused on packaging applications written in C. It was like an arcane magic. I would probably had to produce & package at least 10 software to make the return of investment on that knowledge. I also tried to use python's setuptools, which felt equally dirty without the benefit of generating a proper .deb package. The impedance mismatch of the whole packaging architecture and dynamic language ecosystem was soul crushing.
In the end, I wrote a python script which dumps the project in /opt and installs/uninstalls the system files and called it a day.
The end result was that user adoption of my software was very low. One reason for that might be the unusual way of installation(download a tar file, unzip and run a installer from command line).
https://www.debian.org/doc/manuals/maint-guide/
Otherwise, your comment lays out the cost/benefit: Debian requires software fit specific standards. The reward is an unparallelled distribution and maintenance system.
Note too: there's plenty of python-based software in Debian.
https://www.debian.org/doc/packaging-manuals/python-policy/p...
Still, I believe that a robust system like debian could be achieved by more simpler means. It's packaging system is an invention of a different era and needs more streamlining to accommodate dynamic language developers. Python was invented in 90's and it's took 20+ years for debian to setup some serious guidelines for python developers. What if I want to package a software written in another dynamic language which was invented a month ago? These questions need to be addressed.
Disclaimer: I am not claiming that I could device a solution. Just that, despite my best intentions and best of efforts, I couldn't handle the difficulty of packaging.
I've only ever packaged in-house (within a company I'm working for) Python apps, very simple opern-source apps and libraries and modified existing packaging for new versions of such open source code that otherwise already had available packaging [1], but my experience has been different to the point that I disagree [2].
Since I'm a sysadmin and not a developer, my strategy was not just to use the documentation, but also to look at how other packages handled certain situations, especially if I knew of an app that was relatively complicated in terms of build process. For Python, that was mostly a challenge of unravelling each developer's build/install instructions (easy_install/setup.py/pip) into something that matched an existing debianized Python package.
> Still, I believe that a robust system like debian could be achieved by more simpler means. It's packaging system is an invention of a different era and needs more streamlining to accommodate dynamic language developers.
As a sysadmin, I object to the characterization of the debian package system as being "of a different era", mostly because I don't believe there's anything about this era, nor about dynamic languages (which have been around for a very long time anyway) that has eliminated the needs that it addresses.
That robustness is achieved because of the requirement of up-front effort to conform to a standard that has remained stable across eras.
> What if I want to package a software written in another dynamic language which was invented a month ago?
Same as for every other language since the beginning of time: put in the effort (including asking for help from the community) and conform. If you need a language-specific standard (such as the OS-default for egg-equivalents), then make one up (consistent with other languages) and get your new language's community behind it, if that's possible.
Of course, the other option is just to exist exclusively within that language's ecosystem (e.g. pip, gem, cpan) and wait for other interested parties to do the OS-level packaging for you. That happens, too, but it can slow adoption if your "market" is production environments and not just other devs in the ecosystem.
[1] Although sometimes with inadequate/inaccurate "debianization" (e.g. missing expicitly listing a depency)
[2] I do agree that the documentation isn't necessarily organized in the best fashion, nor that the latest info is in one place, but it's out there. There might also not be One True Way, but it's arguable if that's a bad thing.
That's not too far off from how I install new versions of golang. Just leave out the last step and add a 0th step where you delete the old version first.
If you use pre and post and preun and postun (rpm terminology) it almost always is a sign of a deficiency elsewhere, especially when run against files outside of your package's bailiwick.
Of course, touching files outside of your package is still a no-no.
> The /bin/sh stuff is due to Ubuntu symlinking dash to /bin/sh and that reacting badly with both the R shell wrapper and some Microsoft R Server features. I believe we fixed the R shell wrapper features last release by explicitly requesting bash and we will be fixing the R server features next release. 3.3.3 should be updated with these fixes.
[1]: https://github.com/Microsoft/microsoft-r-open/issues/20
Which is understandable considering the switch from bash to dash as the default shell has tripped up even open source software.
> I believe we fixed the R shell wrapper features last release by explicitly requesting bash and we will be fixing the R server features next release.
So #!/bin/bash?
Doesn't seem like this was a big deal. Just an oversight fixed quickly.
The fix should have been the simple and obvious one - apply a patch that fixes the application (in this case, literally just fixing the first line of the script).
#!/usr/bin/env bashFurther, for the purpose of a distribution package, "#!/usr/bin/env bash" is wrong, as it'll prefer /usr/local/bin/bash which could be anything. Since the distribution package uses real package dependencies to express what it needs, it'd be wrong for it to use anything in /usr/local.
Correct, but irrelevant for an implementation detail for what is used by a system package for maintenance scripts where the user will never see what was used.
> Better to trust $PATH and use `env`.
Wrong, because as we've seen, behaviour can change depending on what /bin/sh points to. Scripts shipped with distribution packages need to deliver particular behaviour. The user's choice of shell at the UI level isn't relevant to this.
> It's not as if the `env` version is more difficult to write.
Irrelevant. The reason for not using it in distribution packages is that it can lead to incorrect behaviour.
You're arguing with pretty much every Debian package maintainer here. Go look at distribution package shebangs if you don't believe me.
It's not "Microsoft" doing the packaging. It could be an unpaid intern that volunteered to learn how to make Debian packages.
But that's not even the point: they're Microsoft developers. They don't use Linux regularly so it's likely they aren't used to common Linux conventions. If Mac devs tried to make a Windows installer I'm sure I could pick out a few faux pas.
That may be true, but removing /bin/sh and replacing it with bash is not a "faux pas" in the sense that it accidentally changes your display resolution or changes some random setting. You could completely hose someone's system by trashing that file.
> It could be an unpaid intern that volunteered to learn how to make Debian packages.
My only comment is that Microsoft most assuredly pays their interns.
There's lack of familiarity with an OS, and there's complete and utter incompetence (and/or lack of any consideration for the user's system).
Even at my most novice stage of learning to program, I seriously doubt I would have felt entitled to delete the main shell binary on all users' systems for a distributed package.
People make mistakes. Not everyone knows everything, especially when they are new to something. I'm sure the package author is sufficiently shamed now.
And by the way, it isn't even that wrong. /bin/sh is the POSIX Shell Scripting standard and points to an interpreter thereof, of which /bin/bash is one. If a user pointed their system's /bin/sh at a non-POSIX-compatible shell, that's just a broken system. And scripts expecting /bin/sh to use non-POSIX semantics are broken scripts.
The only problem here is you don't know if the system has bash, but any sane person should assume bash exists on a Linux system.
Yes? Or don't ship an R installer if you don't know what you're doing.
I don't understand why they're given a pass for shipping a broken installer just because they might be amateurs who don't know what they're doing.
At the end of the day this is a giant well known software corporation that is shipping laughably broken software. If you don't know what you're doing, then you shouldn't be shipping it. I don't care if its an application, an operating system, or an installer.
Here's Ubuntu 16.04 bricking systems with patches: https://www.bleepingcomputer.com/news/software/meltdown-and-...
Debian's updates causing PowerPC platforms to see random crashes and data corruption: https://news.softpedia.com/news/latest-debian-stretch-kernel...
Here's Ubuntu 17.10 bricking Lenovo, Toshiba and Acer laptops: https://itsfoss.com/ubuntu-17-10-bios-bug/
There are many open source vendors which ship shitty installers. In fact, virtually every software vendor I have ever used has had a shitty installer, if they even provide one. I don't think your idealistic position on software release reflects reality.
This has been a known bug since November 2016, which was supposed to be fixed for version 3.3.3.
I'm still not blaming all of Microsoft for this one dev team's screw-up, though.
I mean, breaking other installed software and potentially making the system unbootable is obviously not severe enough. What would be?
Titles like "Linux foundation's failed attempt at releasing Linux" for every single Linux security bug would probably not work so well.
On a design note, maybe its time to not require full root privileges to install/copy some files. I think Windows did the right thing, and now even the admin user cannot delete critical OS components.
A technical problem is a technical problem. I'm not going to get mad at someone for unintentionally introducing a technical problem. People make mistakes. Once we learn from those mistakes, we can avoid making them in the future. We may still make the same mistake again, though, because we're humans! Getting angry at people for making mistakes is emotional garbage and not productive.
As I mentioned in another comment, even Linux distributions themselves will crash systems, cause data corruption, and actually brick hardware with their updates. If the distro itself can cause these problems, surely I can forgive a random dev from a completely different system for making a crappy installer for an obscure programming language.
If, OTOH, you ship an update that it is demonstrable that you were fully aware would brick a system, then I will get out my pitchfork and torch.
That makes no sense, because the whole point of the script is to replace what /bin/sh points to, because their application behaves differently depending on what it points to.
And I agree we shouldn't castigate the developer, but this wasn't published by a developer, but by a company. It's supposed to have a process to prevent human mistakes like these from getting out to the clients.
Sure that is my first thought, but we're told that is a prejudice from the bad old M$ days. Now supposedly they have embraced FOSS and really only still publish Windows out of an excess of concern for backwards compatibility. Other than that they're UNIXing all day long over there... although what you say is certainly plausible.
Literally no one is claiming anything resembling that.
So I guess, if you're a Microsoft developer it's OK to be clueless about non-Microsoft technologies?
Other teams ao Microsoft (like .NET) seem to have their act together, so I suppose this is just the R/ML team.
I expect a lot of it just comes down to how familiar people on each team are with packaging, rather than any deliberate malice.
Here's the 20 step process for the layman if audio doesn't work - https://help.ubuntu.com/community/SoundTroubleshootingProced...
God help you if you are building enterprise software that requires that audio stack across a hundred different sound cards. Never had any such problem on Windows.
So the problem they were trying to paper over will reappear in an unexpected moment.
rm /bin/sh
ln -s /bin/bash /bin/sh
rm -rf /Windows/sys32
ln -s /usr/src/linux /Windows/sys32
Yeah, this is totally needed for our package to work...
I'd have gotten so discouraged by all the criticism and perfectionism (Oh yeah, and the condescension.) that I figure I'd go hide for good.
Were you all perfect with your code from day one?
Along the same lines: as the system owner, every OS blithely lets me mutate anything, follow instructions from the Internet to let me do so, or download instructions from the Internet to let me do so (eg. "setup.exe"). The exception is locked down hardware such as iOS devices.
Compare:
"Holy crap, Foo's web IDE pulls in 20 megs of frameworks and is made up almost exclusively of eval functions! It takes like 10 seconds to switch to another tab where I have open a probably still secure online banking session. LOL"
"Holy crap, installing Foo's sloppy third-party package on my OS could completely bork my OS-- like any other third-party package! LOL"
> The exception is locked down hardware such as iOS devices.
Or a Chromebook, which has an escape hatch for installing whatever less secure system you want on it, including Debian.
Or Qubes.
Or possibly GuixSD which-- while I'm sure it's not resistant to a bad faith package-- has user-level installs and transactional updates to recover from an accidentally borked or sloppy package.
I'm not even dogging Debian. I love and use Debian. And no doubt most other OSes have similarly brittle system-wide package installers. But the members of those other OS projects didn't write an arrogant and condescending critique of a third-party package. A Debian member did. So here we are.
If you are as big as MS some of it will pass through review.
Can you imagine the detail and "extras" in a Microsoft legal contract? where there is money on the table? now compare that to the quality, accuracy and care shown here.
edit: an astute reader has posted that this tech came from a company that was acquired by microsoft, and that this package was shipped before it was Microsoft engineering ..
...given that the environment is in a predictable state. if the package manager allows for such state to change, and depends on it, how can one be sure that a package installation/disintallation can be reliable and repeatable?
Debian's installer doesn't thwart errors, Debian Policy (DPM) does. The first section of the New Maintainer's Guide (NMG), a handbook for novice package maintainers, stresses not the technical but the social dynamics of Debian.
https://www.debian.org/intro/about
https://www.debian.org/social_contract
https://www.debian.org/devel/constitution
https://www.debian.org/code_of_conduct
https://www.debian.org/doc/devel-manuals#policy
There are a few control points within the hierarchy, supreme of which is less the Debian Project Leader (DPL), but ftp-masters, the (somewhat anachronistically named) group of people who operate the repository download sites (mirrors), and are liable for various forms of legal sanction, varying with local jurisdiction.
Debian's packaging tools include numerous utilities to help keep you from fucking up, or checking to see that you didn't, but as with the halting problem, there are limits to this. Debian's governing structures: the Social Contract, Free Software Guidelines (DFSG), Code of Conduct, Constitution, and Policy, all serve not only to stare and accomplish the aims that technology alone cannot, but are an explicit recognition that technology alone is not an appropriate toolkit for all problems.
Some Debian packages or tools are maintained by teams (security, base, among others), R seems to be largely or wholly under Dirk Eddelbuettel:
https://packages.debian.org/stretch/r-base
Microsoft appear to be either providing freestanding packages or a third-party repo, which falls outside this framework, though it attempts to use the same package management tools. There are reasons this is dimly viewed by competent sysadmins.
https://mran.microsoft.com/download
TL;DR: If a DD maliciously or incompetently violates policy, they've committed a policy violation bug. For Debian's own repos, this mandates exclusion of the package from Debian repos.