Hacker News Reader

Comment by tdicola | Hacker News Reader

I've been looking at hosting some apt packages in a custom repo and am truly impressed/horrified by the layers of complexity necessary & suggested to create one. It boggles my mind why an apt host has to be more than a static list of files that are hosted on S3, raw github, etc. Running and maintaining a full blown server just to host some files seems crazy.

View on HN

You can host a dep/apt repo as a static list of files in S3. Did that myself before using reprepro to generate it, then pushed it up to S3. Nothing fancy, but it worked fine. Not sure where you got the idea is was more complicated.

I've also done it using apt-ftparchive using a simple script. Very bare bones, but easy enough.

Was looking at trying aptly next time I needed one. It looked interesting.

https://packages.debian.org/jessie/reprepro

https://packages.debian.org/jessie/apt-utils

https://packages.debian.org/jessie/aptly

[edit to include all links]

Yeah the problem is getting to that point. Check out the docs on setting up an apt repo: https://wiki.debian.org/HowToSetupADebianRepository Good luck figuring out the easy way of using reprepro and S3 from that page.

Point taken.

The sad part is it's even easier than that, if your goal is hosting like two packages. Stick a bunch of debs in a directory, make sure your default GPG key is reasonable, then run `apt-ftparchive packages . > Packages`, `apt-ftparchive release . > Release`, and `gpg -ab < Release > Release.gpg`. Then toss the whole thing on a web server that supports plain HTTP. Honestly I bet GitHub Pages is good enough.

The syntax for sources.list is then "deb http://path/to/your/directory ./". Users can pick up your key by piping it into "sudo apt-key add -", or they can "sudo apt-key adv --keyserver... --recv-key..." with the usual GPG options (and check the fingerprint the same way).

The problem is that most serious users will quickly want tooling to do things like keep track of which versions of which packages are in the archive, not have two versions of the same package, support multiple distros/releases, support separate "test" and "production" areas, etc. And that's where the complexity shows up. (Also why you never see "./" sources.list lines in real life.)

But for "Hey, I built these three packages", it's kind of perfect.

Let me just remind everyone that wiki.debian.org is, well, a wiki. I'm on a mobile right now, so I trying to consolidate the apt/repo-pages, trimming the obviously (too) old information - and adding a more visible "this is how simple this stuff can be, even with proper gpg-signing, and http support" -- is a little too cumbersome. But I've for a while now been intending to help out a bit -- the wiki is somewhat neglected (compared to, say the excellent arch linux wiki). Mostly I think the Debian wiki is in need of some pretty mundane wiki clean-up: consolidation, simplification and modernization (consentrate on documenting current stable and testing).

But more impirtant than me actually doing that, is convincing all of you to update the wiki whenever you encounter errors or missing information! :-)

(No affilation with the Debian project other than being a long time user of Debian)

I feel appropriately chastened now. :-) But that page is huge, and apt-ftparchive is in fact listed halfway down, but it's marked as deprecated two places. And the thought of doing general cleanup on that page (which it can use) is a bit more difficult than just adding some docs, as is the thought of arguing about just how deprecated is deprecated.

But I will try to make some changes to that page.

Tried to create an account on the wiki, got error 919 and a message to contact the debian-www mailing list. Just hoping the list isn't members only now.

If anyone wants to collaborate on fixing that apt page then my email is in my profile. It should only take about an hour to clean up.

I just signed up without any problems. Did you try again?

Yeah, got a reply that indicated I'd been caught in a spam filter (at $EMPLOYER, behind a proxy). All sorted now

What do you mean by the GPG key being reasonable?

Making sure "gpg" with no options does something sane and references the key you want it to reference.

It should exist, and it should be the one you want to use to sign the archive. If not, make another keypair and ... figure out how the GPG command line works. :)

Aptly is a dream to use compared to the others, generally makes sense, and is well documented. Big thumbs up for aptly from me.

I started out with a reprepro archive, which is an abandoned project (or at least, had been for a couple of years early last year (edit: looks like I am in error, there are commits dating to last august... I must have been looking at the wrong repo)). The biggest problem with reprepro is that you can have only one package version per package - so no rollbacks.

But yeah, though you need software to calculate the index files, once they're done, an apt archive is just a static website, easily pushable to s3. Of course, if you want a private repo, it gets a bit more difficult...

You should check out reprepro [1]. The configuration is really easy compared to other solutions [2], and once you've set it up, it's just one command to add or remove packages. The hardest part is generating a GPG key.

One note: it's not quite true that you can't throw it on S3 -- they're just static files. This is how the mirroring infrastructure works, after all.

If you have questions about setting up an apt repo, feel free to reach out!

[1] http://mirrorer.alioth.debian.org/

[2] https://github.com/ocf/puppet/tree/master/modules/ocf_apt is an example

You can also run a gpg-agent to avoid typing the passphrase or your GPG key multiple times during the archive creation.

I can totally relate, that's why we built https://packagecloud.io to provide a service for securely and easily hosting debian and rpm files. We also provide chef and puppet cookbooks to make integration as easy as possible. Give it at try!

For apt repositories, you might be interested in http://www.aptly.info/, especially if you want to host it on S3 as it integrates very well. As others have mentioned reprepo isn't that tough to use either, but moving it to S3 instead of somewhere else basically amounted to using an S3 URI. It also has some other features that might be handy, e.g. versioning/snapshots and serving your repos locally for testing.

In any case, I find the process of actually creating the packages far more arduous than setting up the repo...

Because legacy. Apt predates both of those things by a very, very, long time.

I forgot to also mention -- because apt was really made to service big repositories and big installations (like the main debian repo). What works for a large, stable, heavily-trafficked site isn't necessarily the same as a small site, so small repos are second-class citizens.

Reply on news.ycombinator.com