MSYS2: Arch Linux on Microsoft Windows
msys2.github.io
msys2.github.io
FHS is just the split up at the top level into bin, lib, include, etc, share as opposed to having a folder called, e.g. MyApplication. Some people hate it, but it allows sharing libraries easily. Apple go for something of a hybrid with Frameworks. MSYS2 goes full Linux-style, even for our mingw-w64 (native) softare. "C:\Program Files\MyApplication" is the complete opposite. No sharing. No good for anyone.
Usually software is configured so that at `make install` time it will pull the DLLs of its dependencies beside its executables on Windows, so that, for example a Qt-based game would be packaged with the Qt DLLs in the same folder as the executable.
On MSYS2 we disable this code-path (ok, build-system code-path) and adopt the Linux path instead, so that our final package contains only the executable and a record to say "I also need the Qt shared libraries package version 5.5-3".
In fact, pacman doesn't allow us to make two packages with the same final files in it since it's a System Package Manager and it disallows such conflicts (unless the packages are marked as conflicting).
The other common thing you'll see with Windows software is people compiling and linking to static libraries because it's safer than DLL hell. It might be safer in that regard, but it's a security nightmare for the user (though they often can't know it), so we don't do that either. Our library packages do tend to come with static libraries, but our executables link to shared libraries by default which are also provided.
This way, when a security issue is found in a library, we don't need to update a load of packages containing executables that linked to that library statically, since none (or very few, some may slip through by mistake) do.
Windows isn't the only one non POSIX and even POSIX ones do have substantial differences.
OS X is POSIX certified, and Linux and BSD are obviously UNIX-like and mostly compliant.
Apart from the precise meaning of POSIX, windows is the only OS in widespread use not part of the UNIX-like family.
Not so niche are iOS and Android, which aren't fully POSIX compliant.
Then being POSIX compliant is only half of the story, because each OS tends to be certified to specific versions and there is room for implementation specific behaviours.
For example, Aix used to have Windows like model for dynamic libraries. OS X and its derivatives have Frameworks and so on.
Finally POSIX only applications are constrained to CLI and daemons only, as everything else falls outside POSIX.
That is not a small group, especially as it includes your build environment. In Windows, many times I had a problem that I cannot run ./configure (or worse: the package used a home grown build system that assumed unix-y environment; py2cairo, I'm looking at you), not only because of bash/sed/m4/awk/etc, but also not counting with cl.exe (VS compiler).
Side note: OSX has not only frameworks, but also dylibs. You can treat existing framework as dylib, it will link fine.
Assuming the build environment is about CLI and daemons as I mentioned.
Anything else isn't guaranteed to work. To pick on your example, Cairo needs more than just POSIX to compile. X11 isn't part of POSIX.
Have you experience with big UNIXes like Aix, Solaris, HP-UX, Tru64 and DG-UX?
It been awhile since I used most of them (1994 - 2006), but I clearly remember surprises with those scripts (autotools and friends) when running them outside GNU/Linux.
Maybe the situation has improved there, since like many of us, my UNIX like experience is nowadays constrained to GNU/Linux distributions and Mac OS X.
X11 for Cairo is optional. As is OpenGL or Win32 GDI. But yes, POSIX as it is, is a very limited API and for practical purposes, you need other APIs.
Yes, I have past experience with DX-UG/Tru64 and Solaris. At the time when Alpha AXP was a current chip, it was a little wonder to get to compile the same source code with different compilers, as they had different ideas about what a valid C code is. For Solaris, the easiest way to build anything was to get gcc instead of sun studio compiler. I think that Sun's GNU packages were built using gcc too.
Nowadays, I don't know about all other unices, because I'm using only Linux and OSX too. From the commonly used OSes, the most annoying to build something on is Windows.
Basically it is mostly POSIX development platform for Window based on cygwin, using the pacman package manager. When I was working on a Windows box oh-so-many-years-ago, I would have loved this. It's too bad that this title is here because I assumed this was running Arch in a container on Windows or some such thing. This is much more useful.
Same build system and everything, it just requires some finagling and copying of the PKGBUILDs.
We hope that other software projects will use us as their library provider and build enviornment on Windows, as that way it should be easier for them to also keep up to date.
Unlike the old MSYS, we are also 64-bit and that allows using --enable-auto-image-base which means no more needing to rebase your DLLs. Also, Cygwin have made great strides over the many years since MSYS forked from them and it is now both much faster and more standards compliant.
Git-4-Windows is also based on MSYS2 now and some day (hopefully soon) we will fully merge their work (a lot has been merged already) and use their native git executable.
If Msys2 is Cygwin-derived, does that mean all packages in the Msys2-Pacman are all Cygwin packages? I.e., making this "Cygwin with Pacman as a package manager?"
Does this also mean Msys2 packages can never be more current than Cygwin packages (not that this is a bad thing, but as far as I know Python3 in Cygwin is still < 3.5).
Is the benefit of Msys2 that some packages could be native windows builds instead of Cygwin-dll reliant packages?
No, the msys2 repo doesn't follow the Cygwin packaging versions or schedules at all. They are built from the recipes at: https://github.com/Alexpux/MSYS2-packages
.. they link to msys-2.0.dll which is Cygwin derived and are thus GPLed and exist to provide a POSIX-y shell and build tools to allow building:
https://github.com/Alexpux/MINGW-packages
.. which are full of native Windows software under various licenses.
As far as I can tell, msys2 is still posix/GNU for windows, with alternate shell (bash, not powershell etc) and alternate filesystem (posix/gnu-like)? While mingw is/was gnu/posix binaries for windows.
Both have their place -- but I'd be really happy if something akin to msys2 was available more like mingw -- so I could use the (now finally quite usable) standard windows command line along with (hopefully) MS compilers and have some hope of compiling stuff that uses automake etc.
As far as I can figure out, msys2 doesn't (try to) help with that? It's more like a GNU chroot that runs under windows? In which case most/many use-cases might be better served with a vm under VirtualBox/hyper-v[2] and/or a true "native" windows port?
Not meant as criticism of the project, I'm just trying to figure out if I've understood the focus of msys2 correctly.
Finally, for those wishing for "more GNU" on windows, one of the most pleasant discoveries I've made, is the scoop package manager:
[1] http://www.darktable.org/install/
[2] Especially if/when Vagrant works out of the box with hyper-v -- I'm not quite clear on the current status, but looks like it's been enabled and is included in default Vagrant now: https://www.vagrantup.com/docs/hyperv/index.html
The M in MSYS2 stands for Minimal. Meaning the Cygwin-y part is as small as is necessary to run a shell, bash and autotools to configure and build native Windows software that links to msvcrt.dll with no dependency on the Cygwin derived msys-2.0.dll what-so-ever.
The software built from the recipes here:
https://github.com/Alexpux/MINGW-packages
.. does not link to the GPLed msys-2.0.dll. It is built with mingw-w64 GCC. The focus of the project is to expand that collection of software.
Eg. for darktable, one needs libgtk+ and intltool -- as well as a xslt library -- perhaps there's a quick way to leverage what's already done for Arch[1] when attempting to build on windows?
Note, I'm not necessarily expecting such a build to work perfectly -- but if one could at least get to the point where any build-failures where in the darktable source, not from missing essential dependencies -- that'd be a start :-)
> The software built from the recipes here: (...)
So, if I understand correctly, one installs msys2 from the exe-installer, then updates the system with pacman -- and installs packages one wants from the ones in mingw-packages... and then those packages/binaries should/can be added to the normal windows path?
And that last bit one has to do manually? (Lets say I just want grep.exe and sed.exe in my path for example)?
[1] http://redmine.darktable.org/projects/darktable/wiki/Buildin...
makepkg-mingw -s (*)
.. the complication is in the recipes and patches (which hopefully end up being upstreamed, but not often enough).Darktable won't work without a good amount of effort as AFAICT, no one's built it for Windows. We have most of the dependencies though.
You'd install MSYS2, then pacman -S base-devel and mingw-w64-{x86_64,i686}-toolchain, then not install anything else (*). The -s here means "sync dependencies", sync is pacman speak for install, more or less. By setting up the dependencies in the PKGBUILD correctly, and adding -s to the makepkg-mingw command, you guarantee that you've got it correct for when people install the binaries when it gets to that stage, and that the CI systems fetch the right dependencies when they try to build darktable.
Don't add anything to your Windows PATH. Ever. Why would you break other software by having it run MSYS2 software by mistake? I've never understood software that presumes to add itself to the Windows PATH. Particularly without asking. Really, just what? We don't even ask. When you run our shells we add you to the PATH during that session.
Anyway, that's my global Windows PATH rant over.
I guess that's the bit you didn't get. To use MSYS2, don't use cmd.exe or powershell. Instead, run msys2_shell.bat (to use makepkg_mingw or makepkg), mingw64_shell.bat or mingw32_shell.bat, accept the ways of and use the mintty terminal.
Next you would convert the following PKGBUILD from Arch Linux:
https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=darkt...
.. to MSYS2/mingw-w64 form (look at the roughly 1000 references in MINGW-packages), use pacman -Ss to find the equivalent package name, use makepkg-mingw -s to test.
Good luck. Join #msys2 on OFTC to ask for help.
Thank you for the detailed reply.
Both types of executable can also be run from from PowerShell or cmd.exe too of course. In very few cases you may need to add the correct "bin" folder to your PATH before you do so.
If you want to build MSYS2 packages using the build system we've adopted, then you need to use our shell. That is all (and a reasonable requirement).
You can't link (at the C/C++ level) msvcrt.dll linked-executables with Visual Studio > 6.0 executables / libraries and expect things to work properly for anything but the most trivial cases.
These aren't things that MSYS2 imposed though, these are just the realities of inter-operation between GCC and MSVC as they stand at present.
Maybe I did :-)
Thank you for all your detailed answers.
No. Cygwin depends on a DLL. Mingw does not.
Msys is just a user-land set of tools.
For software build systems - my use case - this type of security lapse is a deal breaker.
The initial download that includes the master public keys ought to be done over https, of coures.
"cygheap base mismatch detected - 0x612F7400/0x612FB400 This problem is probably due to using incompatible versions of the cygwin DLL. Search for cygwin1.dll using the Windows Start->Find/Search facility and delete all but the most recent version. The most recent version should reside in x:\cygwin\bin, where 'x' is the drive on which you have installed the cygwin distribution. Rebooting is also suggested if you are unable to find another cygwin DLL. Segmentation fault"
It's almost like they're coupled to some particular shell and some particularly configured terminal emulator (Mintty). Cygwin is the historical leader of Linux on Windows but that limitation is never desirable.
MSYS and MSYS2 don't seem to have such a limitation and that's nice.
This is unfortunately just something you need to manage for yourself.
Cygwin got itself into trouble like this because it never had a good enough package management story (because Windows can't overwrite a currently loaded dll, they didn't attempt a good system package manager, as best I understand it) so developers would end up bundling cygwin1.dll alongside their own executables instead, so there would end up being lots of cygwin1.dlls on a users machine.
The same thing could happen on MSYS2 if you had two MSYS2 environments, however, to get them both in-sync again, you'd just have to do "update-core" in each and that would be that.
As has been mentioned, its a DLL management issue, not inherent to the design of CygWin.
Some of the software is Cygwin-like, but not much, only enough to distribute (and build) the native (mingw-w64) software and provide a shell.
There was LINE which was superceded by CoLinux. However the 64 bit port of CoLinux was never completed.
Maintainers are really helpful when I sent patches and on IRC, and the software has been running stable for over a year now.
Rock on folks!
maybe that's the same thing, idk
There's you out of the box. You might have tried pacman -S cmake which will have gotten you the msys/cmake package which is used for building the packages in the msys repo (i.e. those linked to msys-2.0.dll). Having to add the mingw-w64-.. prefix is kind of strange, but you get used to it.
I'll try your suggestion and report back.
The difference will the down to path translation. We don't stick to the old MSYS rules precisely as that code was re-written for MSYS2. MSYS2_ARG_CONV_EXCL can be used to prevent specific translations from happening too.
A bug report would be appreciated anyway.
https://github.com/alexpux/mingw-packages
BTW thing that I was able to compile with old MSYS + cmake from cmake.org, but can't with MSYS2 and mingw-w64-i686-cmake is https://github.com/Absolight/pkcs11-proxy
Generally get used using, asking for and contributing to MSYS2 packages if you can. That way everyone benefits. It's not the norm on Windows, but it very much should be.
Git for Windows uses this project.
This is a more complete suit of tools that intends to be sufficient for building any Gnu/Linux/Posix software on a Windows platform. A much more ambitious goal.
https://www.cygwin.com/faq.html#faq.api.fork
The packages in the msys2 repository (which, if they link to msys-2.0.dll are all GPL licensed) only really exist to aid building the packages in the mingw32 and mingw64 repositories which don't implement fork() at all and are the real purpose of MSYS2, i.e. providing normal Windows software.
If you want a POSIX-y system, please use Cygwin.
We made a choice that we'd rather forge ahead with modern LLVM and Clang (for Rust, Clang-GCC and Julia) than hang back to support Blender. Given our very limited resources, this was still probably the right decision, however, we might have made a choice to create a special llvm35 package at that point, but we had no idea that OpenShadingLanguage would take so long to adapt to MCJIT, which the still haven't done to this day, as far as I know.
The important point here is that MSYS2 is a very open-source, limited resources project. We aren't experts in all packages, so if you are knowledgeable about llvm, OpenShadingLanguage and Blender and are interested to do so, then please help out.
For Blender, getting CUDA support into MSYS2 is probably high up on the priority list too.
It is a pitty it cannot build latest Abiword/Gnumeric yet. But we have Cygwin for this.
In any case it is a very serious addition to Windows ecosystem and people should start rethinking using Visual Studio which provides no pre-compiled libraries or package management.
However, one ca get a descent alternative development environment by using
msys2/tdm-x64 and fedora precompiled packages.
Just use MSYS2. It is all that you need. Can you explain why you would do this?
I have msys2/mingw64 but for my Golang cgo I have an alternative install
tdm-x86_64 + fedora mingw64 x86_64 packages
I used msys2 to hand compile pkg-config with these packages.
In this respect I have libvirt available to my golang.
pacman -Ss base-devel mingw-w64-x86_64-toolchain
git clone https://github.com/Alexpux/MINGW-packages.git
cd MINGW-packages/mingw-w64-libvirt
MINGW_INSTALLS=mingw64 makepkg-mingw -s
pacman -U *pkg*xz
.. and if it doesn't work file a ticket on:
https://github.com/Alexpux/MINGW-packages/issues.. otherwise you should have an MSYS2 libvirt.
even if it does work, file a bug asking that mingw-w64-libvert gets packaged.
You really should not mix binaries from different ecosystems and compilers. I do not know how TDM configure their compilers or how Fedora configures theirs (exception models, C++ ABIs) but it is a risky business and I caution strongly against it. All your packages should be compiled with the same toolchain.
Why would you use TDM compilers anyway? MSYS2 comes with its own: pacman -S mingw-w64-x86_64-toolchain gets you the 64-bit version.