RPM Packages Explained
fedoramagazine.org
fedoramagazine.org
RPM files are a compressed CPIO archive with some magic flags and an embedded key-value store.
When installed or uninstalled, rpms execute various arbitrary stages to manage changes to an operating system before, during, and after install or uninstall. So they introduce not only file changes (including changes to the system's RPM database and its index/lock), but also arbitrary system state changes. Fun!
RPM uses a global package database/index/lock to track what is installed and the dependencies. This can sometimes get corrupt, and then you may have to remove the lock and rebuild it.
(edit: this partly applies to yum) Dependency resolution is crap, because dependency resolution is not dependent on a merkle tree or a transaction log. It's more of a lame recursive DAG that cares more about "what does this system currently have and what can I find in the package repo", versus "what was this thing actually built with/for and does this make sense at all to install". Packages are pretty much never built with a "base operating system" kind of dependency resolution, so it's possible to install a package from a completely different distro/version that was based on the one you are using. If you're lucky you won't be able to install the wrong package because of the recursive dependencies, but not everyone is lucky, and not all packages are built properly. It's also possible to create recursive dependencies so an installed package cannot be uninstalled.
A .spec file defines what and how to build packages. A .srpm contains the source code to the application and the .spec file (handy!). A .rpm file is just one of potentially several packages that can result from building a .spec file, and the source can also be spread along multiple files. It's common for patches to be included in the .srpm and used at build time.
A package repository typically contains both compiled binary packages organized by architecture (.rpm) and source packages (.srpm). If you add or remove a package from a repository, you need to re-generate the metadata files that the repo uses to communicate changes to tools like yum, or yum will have no idea about what you added/removed.
RPM is actually fairly portable. A single .srpm file can build packages for Solaris, Windows, HP-UX, Linux, FreeBSD, etc. It can be a very good compliment to whatever the native packaging is, as long as you keep all packaged files in a unique file tree (like /opt/my_pkgs/).
Where they really differ is the ecosystem. Debian has better tools for managing packages, and their community actually builds them better. But I've also been through dependency hell with both; if you want a bleeding edge release, it's way easier to skip packaging and manually build+install updated software to a unique directory.
no wonder rpm no longer get's much love. RPM is too old, but sadly not old enough. My hope is in 20 years time @Foone[1] will warm our hearts with a thread on how he used RPM's to install the gnu toolchain on a fedora based IoT toaster from 2000-late.
[1] For example, IIRC Lua extensions in RPM were merely a convoluted way to extend RPMs macro system. You're still limited to simply defining macro variables in RPMs rigid templating and translation stages. Most of the complexity of RPM is in the places you least need it, and all the pain points are in places RPM doesn't permit you much, if any, freedom to tweak or automate.
But there's nothing concrete proposed by anyone for the rpm developers (or indeed anyone else) to consider for implementation.
We're probably at the point now where it's a good idea to start looking at it, but what should be done?
Best command ever...
From your comment it seems SuSE must have improved since 2005. I didn't follow them for long time now. It's Arch or Debian for me these days.
The startup costs to Debian packaging are greater. In the most basic case not that much greater--a single control file with intuitive fields, and a rules file (i.e. Makefile) with intuitive targets. However, to do things according to best practice (i.e. official Debian policy) often requires using additional tools and configuration files. So, yeah, it can become byzantine. But if you realize that it all comes back to generating that control file and rolling up a target directory, same as RPM, it can be easier to make sense of everything.
The important distinction is that all the complexity, required and optional, in Debian packaging can be hidden within the confines of the packaging framework. That is, whatever your build rules and workarounds, anyone trying to build your package can just invoke a standard dpkg build without having to understand any of it. Working around deficiencies in RPM often requires adding and exposing additional layers of complexity, like generating spec files in ad hoc pipelines before a user can invoke rpmbuild.
For example, do you autogenerate your RPM package version from the source? How do you embed it within the spec file? And are you able to also generate a source tarball that can generate the same build? It's one thing to cobble together an RPM that explodes to some binaries; it's another to do so in an automated fashion that's also transparent--i.e. generates SRPMS that can be built within clean environments. I know, I know, CI/CD systems can obviate the need to build binary RPMs from SRPMs, but that's exactly my point. RPM alone is seriously deficient. There are lots of little problems like this that pop up when your needs become more complex; you end up building taller towers of complexity than what would be needed using Debian.
That said, rpmbuild has grown features over the years to make it more compatible with that model.
My experience with RPMs, Debian packages, and various other formats has led me to prefer some of the rigidity and explicitness of rpmspec over the sprawl of automagic that is a debian/ tree, and other similar things. But if you want to do weird things to build packages, nothing stops you from doing that and using a trivial spec file as the intermediate to pass instructions to rpmbuild to wrap it up.
The format is not perfect, but I haven't seen anyone make any proposals recently about how to improve it on the rpm-ecosystem mailing list, the rpm-maint mailing list, the rpm project on GitHub, or at any of the developer conferences where rpm developers show up.
You seem to have a ton of knowledge and opinions, why not engage with the project?
I don't have much experience with them but I was under the impression than just writing a specfile and calling rpmbuild was a step up compared to debian, which seems a mess to me with the makefiles and other stuff all over the place and all those dpkg commands which are wrappers for each other.
- yum/dnf take extraordinary amounts of time compared to apt
- debs are all in one source. Upstream sources are clean, predictable, and easy to troubleshoot.
rpm hell was the Dark Ages of FOSS, and I'm so glad that the community has resolved that problem. However, the bitter taste still lingers and for this reason I will never take rpm files seriously again.
You can also revert multiple transactions at once, e.g. restore the package versions to what they were three months ago.
There's nothing remotely like this in apt -- if you install a lot of packages on your system, and then want to remove them, the common solution is something like "parse the (text) apt logs, learn manually what's changed, and revert it yourself".
Some other niceties:
* package groups: `dnf install @'c development'` instead of `apt install binutils gcc make bison yacc whatever-else`. Somewhat mitigated by metapackages, but they rarely cover the whole groups of functionality, i.e. they are often more low-level.
* package streams: `dnf install node-8`, or `node-9`, or one of the other supported versions;
* dnf downloads many packages in parallel, and the level of parallelism is configurable. This greatly helps with upgrades from slow mirrors. I believe it can also utilize multiple mirrors at once, though I haven't tried it.
* dnf supports plugins and you can install them with a simple command. For example, there's a plugin to automatically create btrfs snapshots on every package upgrade.
* COPR saved my skin a couple of times, though it's basically the same thing as Ubuntu's PPAs. This is more of a problem for Debian.
The /debian/ directory contains all distro specific patches, as well as build rules (a Make file), and package scripts.
It's possible to build a 'debian native' package where the /debian/ directory is bundled up with the orig source, but those are few and far between -- primarily packages consisting of debian-specific scripts/tools.
However, if as a developer you want to include packaging support inside the project (for your own or your users' convenience), this is infinitely easier to accomplish with Debian than RPM packages. Various aspects of RPM--e.g. spec file template semantics, source and build directory locations--not only assume that packaging is done separately, but make violating those assumptions costly. Whereas Debian package build tools rarely make such assumptions.
If I want to add "deb" and "rpm" targets to my Makefile, the deb target is a one-liner that directly invokes a dpkg tool using standard options. Whereas the rpm target may require generating a specfile (because some specfile variables aren't trivial to dynamically evaluate inline, if at all) and source tarball, or to go through additional contrivances to avoid either of those two things.
This is similar to what have been able to do to make debs with dpkg-deb.
this is a indepth user guide, at most.
I know rpm is a cpio archive, with the files and some scripts, but the capabilities, when the scripts are run, etc I'm not sure of (I don't encounter rpms much - I'm a deb person, and I'm happy with my "ar -x" to explode my deb and look at the dependencies, conffiles, install/remove scripts, etc to install our custom packages on any rare centos servers we may need)
To look at scripts/dependencies/etc you don't need to explode it at all, rpm has switches for querying that.
this discussion comes up time and time again (in rpm, apt et al). the consensus is: if you need that extra feature, manually download sensitive packages via ssl or something. everyone else (with nothing to hide, heh) keeps benefiting from a global cache of unencrypted transport of (mostly) open source data.
If you want to really hide what you are installing, make a local mirror of the entire repo and then pick and choose from that.
I don't think that the RPMs that I have created in my internal repository and deploy to my field systems are a 'known information' to anyone outside of my organization. If they are, I'm in serious trouble.
I think a more realistic use case for package-level encryption is deploying RPMs that have secrets in them (either keys/creds in configuration or trade secrets in application logic). Ideally of course we should encapsulate these such that they aren't deployed to field/embedded devices but in embedded there certainly may be some use-cases and requirements that those of us used to working in data center and cloud computing aren't immediately thinking of.
Yum with CentOS 6 and above does support SSL for mirror sites and a handful of global mirrors also support it (HEG being one).
I suppose there's a slight race condition (eg how do I update the CA-Certificates bundle when I need the new CA-Certificates bundle to connect to the mirror site to download the update), however I tend to agree there should be some privacy as default.
The real way to protect against this, if it's genuinely part of your threat model, is to maintain a complete local mirror: you can't tell what is installed and at what versions if you simply download everything.
And if it's actually part of your threat model, then you likely have a large enough install base that you need a local mirror for performance/non-security reasons anyway. So it's really a non-issue.
You could accomplish this having a keypair on your field/embedded devices, and then having the RPM distribution system pull each devices public key from the keyserver, build the RPM with encryption specifically for this device, and then push it out. Or maybe you choose to have a generic keypair for a class of devices.
This could be used in cases where you have internal secrets in the RPMs you are building, or in the case of things like proprietary software and software licensing. I don't see how this applies to open source OS updates which is what I think the other sub-thread seems to be fixated on for some reason.
Whether this belongs in the RPM system itself or in a wrapper format, I'm not so sure of.
HTTP supports inline username and passwords, use HTTPS to keep the password encrypted on the wire: http://username:pass@server.tld/...