Just-in-time packager for GitHub repositories | Hacker News Reader

Just-in-time packager for GitHub repositories

jitpack.io

55 pointsjitpack31 comments

Using source repositories as packages sounds like an awful idea. Not only do you have to deal with runtime dependencies, but also compile, test, and other dependencies. What happens if the tags are not numbers? What if a tag you are using is deleted?

Go ask the Erlang community about the issues they have with using GitHub repos as "packages". Here is a discussion about packaging: http://mostlyerlang.com/2015/01/27/054-packages/

I think you might be misunderstanding how this service works.

The Github repositories don't serve the packages.

Jitpack checkouts the repository code, builds it, and serves it like a normal maven repository. Here is a sample repository https://github.com/jitpack/maven-simple and the maven repository it is served from: https://jitpack.io/com/github/jitpack/maven-simple/0.1/maven...

That's correct

I got that it is building a binary and basically placing it in a local repo. I also get that it is looking for a binary from a GitHub Release first, which is mildly better.

I still think solutions like this are putting your build process at great risk. You will have build issues that are no fault of your own and completely out of your control. Unless you clone the github repository and then use that as your source. And cloning will have issues and risks of its own.

EDIT: See pron's reply for describing the risks: https://news.ycombinator.com/item?id=9029870

Thanks for the feedback and its good to hear your concerns. Yes, you can use strings as tags but we recommend using Semantic Versioning just as GitHub recommends for Releases [1]. At some point we may choose to only build tags that match semantic versioning, to be decided. Unlike using just GitHub as source repositories we have some over what we serve and that's the difference.

If you make a conscious decision to release, add release notes and a tag why would you delete it? Especially if you want others to use your project. Its just not part of usual release workflow. Right now you can still get the binary for a tag that has been deleted because we cache it. We will probably add restrictions to prevent building another binary with the same tag.

[1] https://github.com/blog/1547-release-your-software

See @vathpela

https://twitter.com/vathpela/status/563808697151803392 "Apparently Intel's Galileo build scripts that they distribute to users just clone my damn grub-0.97 repo off github."

https://twitter.com/vathpela/status/563808755213557761 "Which isn't a thing I knew when I deleted it Tuesday." "

A very nice project that reduces the barrier-to-entry of Maven artifact publishing, and saves an open-source project author a couple of hours of annoying work, but I have two concerns:

1. It might cause clashes for repositories that are hosted on a public Maven repository. Maven artifacts have one name, and one name only. Now, they may have two (with versions that are not necessarily aligned, to boot). This can wreak havoc on transitive dependencies.

2. Because git tags can be deleted and re-created, this repository breaks one of the strongest guarantees made by all (public) Maven repos, namely, that once a release artifact is published, it can never, ever change. This makes artifact versions unreliable.

In short, release artifacts must be unique and must be immutable; this project breaks both. That will cause actual runtime bugs that are virtually impossible to catch (and are bound to cost much more than the one-time setup -- per organization! -- of Maven hosting).

The sort of capabilities and guarantees required for source code management (and are provided by SCMs like git) are not the same as those required of binaries.

3) This creates an insane MITM opportunity.

Not only are they spitting back opaque binaries, but they're doing so by running arbitrary and untrusted user code.

There are already single-command tools for releasing a project to Maven, including tagging the release, bumping the version number in the build file, building and signing the jars, and uploading the results to a Maven repository.

Given that, why would you SaaS trusted builds!?!

Well, to be fair, any public Maven repo is a MITM opportunity.

Maven artifacts can be GPG signed; GPG signing is required for Maven central.

It would be irresponsible to use a service like this to build binary JARs that you then signed and uploaded with your own signature guaranteeing their providence.

Releases have to be PGP signed, snapshot's don't.

How many people do you know that verify PGP signatures of their artifacts? Do you?

Yes, we verify signatures at our middleware repository cache.

Really? Impressive! Where do you get the public keys? Most projects hosted on Maven Central don't publish them on their website.

http://blog.sonatype.com/2009/04/nexus-133-introduces-automa...

But unless the signers have a public certificate, or publish their public keys on their website (which you need to obtain manually), the signatures on Maven Central can be just as fake as the artifacts.

Thank you for the compliment, Ron. Glad you see an upside to JitPack:) Regarding your concerns:

1. If you add JitPack as a respository then Maven will still check central first. If the artifact is in central it won't bother going to jitpack.io. Also JitPack enforces your group name to point to your GitHub repo to avoid name clashes.

2. Yes, git tags can be deleted but you really shouldn't do that on a public repo anyway. And most of the time people releasing with tags don't change them.

JitPack is built for the common case and not for the edge cases. And we believe that just because there are rare ways to break the system doesn't mean we should keep a high barrier-to-entry. Sharing your work should be easy and it should be easy for others to try your work.

> Glad you see an upside to JitPack

Of course! It's very cool! But I don't think it can be used in production (or even in development of important projects).

1. Say I am the consumer and I use a library author's GH repo with JitPack. Then, when the author releases her library to Central/jcenter/Clojars/whatever, she's free to choose any group name she wants (JitPack can't force her to do anything), and once she does that -- and another of my dependencies transitively depends on that artifact -- I am very likely to get a bug that's very hard to track down. Alternatively, once she publishes to central under a different group name, you might choose to simply break my build, which is also undesirable.

2. Maybe people shouldn't, but they do it all the time. I've done it on occasion, and I'm pretty sure I'll do it again. When that happens -- there's another impossible-to-find bug.

> JitPack is built for the common case and not for the edge cases.

Which means it's a binary repository that is undependable. And remember, those edge cases are out of the library consumer's control.

> just because there are rare ways to break the system

Unfortunately, they are not rare. First, most (actually, virtually all) production-quality Java/JVM libraries are already hosted on some public Maven repo, and any library that becomes production quality is likely to publish the artifacts, so breakage 1 is almost certain. As to 2, you should really get the data from GitHub. I think you'll find that deleting and re-creating tags is a lot more common than you think.

But even if those cases were rare, they can be very, very costly. We're not talking about a breaking build, but actual runtime bugs that might take days to track down.

> doesn't mean we should keep a high barrier-to-entry

True, but the barrier-to-entry for the author is already quite low; jcenter makes it almost nonexistent.

What you've done is something else: you've let the library consumer control the artifact, so you've lowered the barrier for them, which is great! However, by doing so you've ensured that the author will almost certainly inadvertently introduce really, really tricky bugs into the consumer's project, bugs that they can neither control nor track down.

Still, JitPack is a great way to try someones work-in-progress code, to see if it might be worth it to help them make it production-quality. I am sure to give a try some time.

It's an interesting concept. Instead of you uploading binaries the server builds and serves them. Any plans to support other build systems?

There are! Probably Sbt first and then Leiningen.

This is brilliant! Another vote for SBT!

Amazing, this was sorely needed!

Nice! This should make it a lot easier for developers to publish their projects.

I thought the same. Publishing small JS libs is easy, just drop a Bower file and register it. For maven I need to build it, sign it, get access to a central repo (can take days) and then finally upload it.

Of course, this means that the quality is often much higher, but I think the barrier is a bit too high as it is right now.

> For maven I need to build it, sign it, get access to a central repo (can take days) and then finally upload it.

Those are the steps required to host your artifacts on Maven Central. Hosting on other public Maven repositories requires less work, and if you host your own, you set the bar (you can even turn a GitHub repo into a Maven repo, and upload with a git push).

Having said that, if you've built something worth using, the extra steps required to publish to Maven Central are both negligible with respect to your overall effort, and are more than worth it (getting the initial access usually takes hours, or one day at most, and only has to be done once for the entire organization/author).

So far, the only projects I've seen that are not already hosted on Maven Central or any other public Maven repository (jcenter, Clojars, java.net and others), are usually short programming exercises or works-in-progress.

Love this project, almost surprising that nothing like this exists today for the maven world.

It does. There are gradle/Maven tasks to upload your artifacts to public Maven repos, and they're easy to set up so that Travis publishes them whenever you create a tag.

But this allows anybody to depend on something that somebody else wrote even if he forgot to publish (for whatever reason). It reuses the upstream github namespace instead of creating many artifact group ids when random people publish somebody's else package

Sure, which is why it is sometimes useful. But people who make Java libraries that are production quality, usually don't forget to publish, and if they do, they are quickly reminded to do so. I am not aware of any production quality Java library from the past five years that isn't on Central/jcenter etc.. Older ones aren't on GitHub, either. In fact, supporting sourceforge might be more useful than GitHub, for that reason.

Does jitpack currently support repositories in Github Enterprise?

Currently not. Actually you are the first to ask and it sounds like a great feature:) most likely that would mean you host jitpack on premises. It's something we don't provide yet but it is on the road map.

Wonder how high the usage would be. If you have github enterprise setting up Nexus and having build process publish to it seems easier than this build server.

Reply on news.ycombinator.com