Jenkins 2.0 Beta
jenkins.io
jenkins.io
Under the hood, it's implemented by taking a script written in a "DSL" (it's actually Groovy code), transforming it into continuation-passing style, and serializing its state. This is pretty cool from a theoretical CS perspective, but having played with it a little bit, the implementation seems very fragile. There are a number of long-standing unfixed bugs that cause the transformed code to misbehave in ways that you wouldn't predict just from looking at it, even if you're experienced with Groovy. I ran into a couple of them just during a brief period of experimentation. For instance:
https://issues.jenkins-ci.org/browse/JENKINS-27893 (varargs are mishandled)
https://issues.jenkins-ci.org/browse/JENKINS-28277 (currying doesn't work)
https://issues.jenkins-ci.org/browse/JENKINS-26481 (calling a closure in a loop doesn't work right)
https://issues.jenkins-ci.org/browse/JENKINS-28183 (killing a pipeline job doesn't clean up its state properly)
And there are inherent limitations to the approach; for instance, you can't store any non-serializable values in local variables, which means simple things like foreach loops don't work (because you can't serialize an iterator).
I really like the idea behind the Pipelines feature, in principle. But I think building it around a general-purpose programming language, and then failing to support all of that language's features, is a recipe for headaches. If you never try to do anything more complicated than what's shown in the examples then you should be fine, but the borders between what works and what doesn't are very ill-defined.
EDIT: Oh yeah, and there's a surprising amount of functionality that isn't documented anywhere except for blog posts and JIRA tickets.
It's Groovy code? The announcement page doesn't mention "Groovy" anywhere, let alone "Apache Groovy" which is what the backers of Groovy are meant to refer to it as since last November (2015) when it was promoted from the Apache incubator by the ASF.
> A pipeline is a Groovy script that tells Jenkins what to do when your Pipeline is run.
I think the future belongs to statically typed languages that can offer DSL's as terse as Groovy but with actually types in them, so that the IDE can actually do the job it's being paid for.
Not as easy as it sounds. If the DSL is rich enough, and if the types are handled by the host language, type errors can become just as cryptic and unhelpful as with an untyped DSLs, and it's just as hard for the IDE to do something meaningful with them.
With dynamically typed languages, you get cryptic (and vague) errors and auto completion just can't work.
That said, this appears to be achieved by promoting the plugin into the default installation.
It also misses some of additional the advantage Concourse holds over Jenkins and GoCD: build configuration is purely declarative and can be checked in with the project. You know what version of your pipeline built the software at any point in its history. And you have a reasonable shot at recreating that build, because every task in every job is run in a fresh container.
These are killer features, in my view. Jenkins can be extended with plugins to try to sorta-kinda do either or both, but it's not part of the central organising concept of how it works. Windows can run some POSIX apps, but it's not a nix.
Further out, the Jenkins pipelines are tricky to do fan-out/fan-in with; in Concourse it's trivial. You have to lay out your pipeline in Jenkins, Concourse will lay it out automatically based on declarative information about each job. Rather than a very rich API for plugins, Concourse boils the unit of extension down to "resources", which with three actions (check, get, put) can model points in time, git commits, S3 files, version numbers, interacting with a complex product repository and so on.
I used to tolerate CI/CD, as a necessary and worthy PITA. Now I find myself actively looking for regular tasks, resources, sanity checks and so on I can put into Concourse, so that I don't have to remember them or write them up in a wiki.
Disclaimer: I work for Pivotal, which sponsors Concourse development. But I wouldn't rave about it if I wasn't such a convert.
(in the beginning, there was light...)
* Create a small, tight, single-purpose Jenkins job
* Add a small tweak to it
(repeat adding tweaks)
(realize the Jenkins job now contains MANY different configurations options and the job itself is now a shell script in its own right)
* Sweep the "job" into a shell script. Check in said shell script
* Back up the Jenkins config, and hope no one asks why something's happened.
I now have a plugin that automatically checks in the Jenkins config to source control, but it again doesn't solve the problem of matching up a particular jenkins artifact to exactly what built it, and why.
Will look more closely...
https://zwischenzugs.wordpress.com/2016/01/24/ci-as-code-sta... https://zwischenzugs.wordpress.com/2016/01/30/ci-as-code-par... https://zwischenzugs.wordpress.com/2016/02/25/922/
Works really great. Jobs can run on any slave, no snowflakes, full CI config is versioned in the repo along with the code. Jenkins job only point to a single script and that's it.
Agreed that the format isn't ideal but there are non-trivial problems in breaking it down due to the advanced way GoCD manages to tackle the diamond dependency problem (aka the Fan-In functionality).
Totally agree with you re pipeline as a first class citizen: I wrote this back in 2014 so it's kinda old now and surely Jenkins got better but I think it still applies by and large: https://highops.com/insights/continuous-delivery-pipelines-g...
That is, you define the inputs and outputs of a job, then Concourse derives how to carry out your jobs in the correct order under the correct conditions.
A Concourse yaml file is pretty flat, as a consequence.
That's a great writeup. I think you'd find Concourse attractive.
I couldn't find any docs on intra pipeline dependency management though, any links?
Thanks!
As for my own project, we have an increasing group of downstream consumers of our API. There are a few ways I could signal upgrades. They could just watch my repo and narrow it to the directories they're interested in. I can also publish a file on S3 or push into a git repo that they watch.
Seems like you are missing the point of pipeline and the Jenkinsfile concept, namely that the pipeline is part of the SCM.
Yes you can have the pipeline defined in the old-style text box in the job config, but that is intended to be used only while you develop the pipeline. Once you get it developed it gets checked in to source control.
Of course there are down sides to having the build instructions in source control, e.g. drive-by hack via Pull Request... granted this is nothing new... You can always do the drive by hack in a unit test... but it does take a little more work. In that regard having the Jenkinsfile in a text box (or better yet in a separate SCM) can be a useful protection (as can mandating that PRs use the target branch Jenkinsfile rather than the Jenkinsfile in the PR... again a feature in Jenkins pipelines)
For me the real advantage in pipeline is the organization folders support. You can tell Jenkins to scan all the repositories in your GitHub org and automatically create Jenkins jobs for any branches of any repositories that have a Jenkinsfile... PRs will automatically (subject to basic drive-by hack protection) be built and the commit will be tagged with the result.
So as with all things Jenkins, you have choices... we are providing some opinionated defaults (which is a change from the 1.x series)
> That said, this appears to be achieved by promoting the plugin into the default installation.
So the thing to remember is that the core of Jenkins is really better viewed as a platform. The plugins are really where functionality for Jenkins lives. I would expect to see more of the "current core" functionality get shipped out of core and into plugins. There is no reason why the Freestyle job type needs to remain in core. The advantage of having these functionalities outside of core is that we can be more reactive with regards to developing features.
> Further out, the Jenkins pipelines are tricky to do fan-out/fan-in with
I might have a different view on that claim, but hey I'm significantly biased.
OTOH my personal view is that for the 99% of jobs pipeline is overkill and literate is actually a better fit... but sadly most people don't seem to like the idea of actually making your source control have a README.md file with a - shock horror - "build" section that actually has the verbatim command required to build the software in source control (perhaps with an "environments" section that describes the build / test toolchains and environments)... I guess there's too many people who signed up to the mortgage-driven development manifesto [1] to actually want to leave a README file in source control explaining how to build and release the software!
Disclaimer: I created the weather report column and I am an active Jenkins developer
[1] https://refuctoring.wordpress.com/2011/01/13/the-mortgage-dr...
I guess I missed that Jenkins is heading that way. It's what Concourse does and I'm a fan of having CI/CD live in the repo.
> For me the real advantage in pipeline is the organization folders support. You can tell Jenkins to scan all the repositories in your GitHub org and automatically create Jenkins jobs for any branches of any repositories that have a Jenkinsfile... PRs will automatically (subject to basic drive-by hack protection) be built and the commit will be tagged with the result.
For PR-building on Concourse, the resource I'd recommend is: https://github.com/jtarchie/pullrequest-resource
> we are providing some opinionated defaults (which is a change from the 1.x series)
I see this more and more in the Java ecosystem and I think it's a good thing.
> I might have a different view on that claim, but hey I'm significantly biased.
Me too! :)
> OTOH my personal view is that for the 99% of jobs pipeline is overkill
I am starting to head in the other direction. We've historically fallen into creating "big ball of mud" build systems because it was just too hard to easily decompose and manage them as smaller units that could be rearranged quickly and safely.
Concourse makes it so trivial that the gradient for what is easy points in the other direction. It is less painful to lay out a pipeline (a graph, really) of builds that are composed of small pieces, than to have one gigantic Build To Rule Them All.
In Pivotal the practices around Concourse are evolving extremely quickly, because teams are discovering that it's really easy to delegate more and more to it. You start with a simple git->unit->feature->deploy pipeline, but soon you realise it's easy to assemble all sorts of things. The best is yet to come.
Source control can tell you when changes to the pipeline were checked in, but they actually take effect when they're applied with 'fly set-pipeline'. That may be before or after they were checked in. Perhaps a sensible team would set up a pipeline to watch the pipeline repository for changes and apply them automatically. Mine hasn't.
> Concourse boils the unit of extension down to "resources"
Which you can only actually use by forking the Concourse BOSH release to add them to the workers [1]. I'm not sure i'd honestly call it a point of extension.
[1] https://github.com/concourse/concourse/blob/master/jobs/grou...
We've recently started building standalone binaries which should lower the barrier to entry. Concourse itself has never been too tightly coupled to BOSH, it's just been the quickest feedback loop for us during development, so it ended up being the first thing we documented, primarily for internal use as we haven't really "launched" yet.
Binaries are available for download in the GitHub releases[1]. Check the README.md[2] for details. We'll be launching 1.0 very soon and part of this will include a major docs/website revamp which promotes the binaries to the "main stage". It also switches to BOSH 2.0, which drastically simplifies the deployment configuration necessary, but it still takes a backseat to the lower-upfront-cost distribution formats in the new site.
Glad you liked Concourse otherwise, and hopefully this helps. :)
[1]: https://github.com/concourse/concourse/releases [2]: https://github.com/concourse/bin/blob/master/README.md
I think the next step from us may be to start building Vagrant boxes that just spin up a worker, parameterized with credentials to register with a Concourse `web` instance. That way you can run Concourse on OS X for iOS testing/etc. and still have all your resources and Linux containerization when you need it via a tiny local VM.
For example, [Guardian](https://github.com/cloudfoundry-incubator/guardian-release) is in the works to replace the Linux backend with a thinner runC-based backend.
The main value we get from it is having a nice Go API and not having to overhaul everything using Garden every time some shiny new container tech comes out.
GoCD and Concourse add... what?
GoCD and Concourse make the connections as important as the individual jobs. Concourse goes further and makes checked-in configuration and disposable containers the basis of the build environment.
This sounds like no big deal, but it's critical.
The point is not whether Jenkins can "do" these things. With the right stew of plugins, it can. The point is that Jenkins does not really think this way. It starts with the Single Ball of Build as the unit of work and retrofits other possibilities.
The UI is atrocious, job state is spread out among tons of crappy xml files, and the plugin system causes tons of headaches. If you're going to have a system that forces you to use the UI for the most part, rather than scripting up config files that I can load with some automation, at least make that UI nice to use.
Hopefully 2.0 is fixed up, but personally I'd never reach for Jenkins as a CI system if it wasn't part of a client's requirements.
I'd still never use Jenkins on my own though, I'd just use Drone or pay CircleCI.
https://zwischenzugs.wordpress.com/2016/01/24/ci-as-code-sta...
https://zwischenzugs.wordpress.com/2016/01/30/ci-as-code-par...
https://zwischenzugs.wordpress.com/2016/02/25/922/
Does anyone know if any effort has been put in to make Jenkins more 'programmable' in 2.0? I had a quick look at the bumph recently but couldn't see anything pertinent.
> "Jenkins 2.0 is a drop-in replacement of the Jenkins 1.x series of releases and fully backward compatible. There is practically no reason not to upgrade once 2.0 is released."
Skeptical, but optimistic about this claim.
I hope that the Jenkins configuration format is something that can be manipulated more easily in the new version.
I also found that downgrade may not fix the problem, as the upgrade may change your configuration files to use newer features, causing Jenkins to choke on them once downgraded.
Jenkins is a spiderweb of plug-ins held together by shoestring. It's a very useful tool, but extremely fragile.
You can find it's sources here: https://github.com/drone/drone
I just hope it would be able to work without docker, like jenkins.
What is interesting is that Jenkins has a DSL but that this is not part of the repository. This means that it is hard to extend when you introduce new functionality and when you push an old branch it might not work anymore. I think that the model that Travis CI pioneered clearly wins.
I'm torn on the plugins. One one hand it is great to be able to plug so much in. But the plugins have access deep into the Jenkins internals, preventing a rewrite of core functionality. Our idea is: "Because GitLab is open source the enhancements can become part of the codebase instead of being external. This ensures the automated tests for all functionality are continually run, ensuring that plugins always work. It also ensures GitLab can continue to evolve with it’s plugins instead of being bound to a plugin API that is hard to change and that resists refactoring. This ensures we have many years of quality code and great monthly releases ahead of us." https://about.gitlab.com/direction/#vision
>What is interesting is that Jenkins has a DSL but that this is not part of the repository.
Netflix job dsl plugin, Jenkinsfile pipleline, dotci all of these let you put a .travis.yml like file in your repo.
>But the plugins have access deep into the Jenkins internals, preventing a rewrite of core functionality.
Plugins don't have deep access to jenkins internals they use clearly defined "extension points" .
I know you are trying to plug gitlab but 'our stuff is better because we don't allow plugins' is not exactly a plus. Jenkins is so popular because it can customized via plugins to wide range of use cases.
1. Cool that there are different plugins that let you check the configuration in a dotfile.
2. I was not aware they used extension points instead of something more intrusive.
3. Obviously I like GitLab and try to promote it. But I post this too to see what others think and to learn. Turns out I had a lot to learn in this case. I still think that some items in Jenkins should be part of the core, such as the pipeline. But I agree that plugins are needed for some cases. For deploys we're thinking about plugins for GitLab CI too in https://gitlab.com/gitlab-org/gitlab-ce/issues/3286#note_402...
From what I could gather it means that I have to create a specific runner whose login credentials are in its config, which aren't accessible via the WebUi, only as a file on the server where the runner is running and give it an unique tag. Then in my repofile i have to select that specific runner so it then "runs the build" on my deployment target.
Hopefully i'm completely wrong about this, but this seems horribly complicated and very inflexible for deployment purposes. Why would I specify my deployment target in my repo?
Maybe simple deployment jobs like I'd build them in Jenkins aren't in the scope for GitLab CI?
If you want to securely store login credentials consider using secure variables http://doc.gitlab.com/ce/ci/variables/README.html
Jenkins is total Overkill for that but GitLab CI doesnt leave me with enough GUI to skip it either.
The flexibility Jenkins provides as build system is enormous. The plugin community is also a huge benefit -- chances are, if you have a need, there's a plugin for it.
It essentially should be running shell scripts and/or language-specific build scripts/tools. All logic should be self-contained in your project(s) repos, so that it doesn't really matter what build system you use, and/or one could build the project without the build server if required.
With that said, I don't see any reason why release management, containerizing, versioning of assets, etc... could not all be done with Jenkins (or any build system).
I'm a broken record in this thread: Concourse. Stateless, containerised builds. Declarative config checked into repos. Pipelines as the central concept.
All of these as the core paradigm, not from plugins.
Could developers of these tools please stop dictating where people have to put the "Blahfile" dsl config files and finally allow people to configure these kind of details.
Also I would like to propose a common standard directory "projectroot/config" for these kind of files, NOT the project root directory itself.
Thanks for your attention!
The amount of top-level files in these projects has started to become quite large, and it is unclear at first glance to see what it is important when exploring a new ecosystem.
I'd like to see these products would move in a direction where these were would do any/some of the following:
1) use a file extension rather than "important" describing filename (.docker for Dockerfiles and .groovy/.jenkins for Jenkinsfile)
2) moved into a product specific folder (travisci/ci.yml, github/ for the issue and pull request templates)
3) use yet another top level file for indirection that is agreed upon (my initial thought is something named integrations.yml)
Concourse lets you put the file anywhere; the act of applying configuration is distinct from committing it.
Initially people put `pipeline.yml` or `concourse.yml` files in the top of their repo.
It's rapidly evolved so that typically you'll find either `ci/pipelines/{some,number,of,pipelines}.yml` in the main repo, or a separate repo for the CI/CD system. I prefer the former, most teams I've seen so far prefer the latter.
Background: I have "deploy frontend" and "deploy backend" job, both of which need to invalidate some layers of caching, and a job parameter that specifies the environment (it's the docroot).
For now I have a metric shitload of shared code between the jobs as there is no way to move the cache clearing code to its own job and pass the environment parameter through in pre/post build actions.
So given that's the foundation they're layering even more complicated stuff on top of it. Thanks but I'll pass.
Its simply too difficult to use by a laymen developer or sysadmin when accommodating complex buildchains, which only grow more complex over time in most software businesses.
If you're going to program your CI with a web administrative interface, and provide an easy to navigate interface for users, expand with build metrics and external systems, I suggest TeamCity over any others.
Though jenkins is free, my experience is maintenance teams of 2 or more people with tribal knowledge develop around the tool that far outweigh the license cost of TeamCity, which solve many of these homegrown engineered problems as part of the product cost.
If you have a strong developer team, I would suggest using something like python buildbot and programming your CI end-end in a single unified language, much better than shoe-horning plugins and groovy code inside your jenkins textarea fields!
That said, all CI systems are just software, and any requirement can be accommodated with sufficient amount of additional software, forks, or changes. Any will do.
However, the new Jenkins 2.0 fixes many of the issues you've described. Build pipelines are a major new feature, as well as the integration of a "Jenkinsfile" which allows configuration of jobs via a proprietary DSL.
This makes no sense to me. Only how my project is built should be checked into the project's repo, not the whole CI/CD deployment pipeline! If I start deploying to Azure instead of AWS I need to commit that change to my project? It's nonsense.
Come on now, we don't need to go to these contortions just to avoid saying that Jenkins is doing it wrong.
You could indeed just use the Jenkinsfile for building a project. You can also specify the DSL config directly in a job, e.g. in a separate deployment job.
What would be the right place to define the deployment config?
This Jenkinsfile is their solution to the problem. I'm saying it's not a very well thought-out solution, because now they're forcing us to put this thing that's not part of the project in the project, if we want it version-controlled.
There is something amiss in the tech world these days. Too much esoterica and too many specialized tools, too many points of failure and dependencies, and too much duct tape. But, mostly, too much reinvention for frequently incremental "gains".
It seems normal, but it's really a mess. Environments are overly complex and it's a wonder anything works half the time.
At some point we will completely jump the shark. From there, my bet is there'll be a powerful movement towards simplicity.
/rant
Script your build so that you can run it with a single command on any build server or any developer machine. Then have the build server invoke said scripts (with arguments for any build-server specific paths etc).
Having spent a week fixing problems with inter-plugin dependencies I do agree with your point - Jenkins just isn't very good, like a lot of the free software from that generation (FitNess etc).
Glad to see jenkins is moving towards a travis-ci ".travis.yml" kind of format, but I'm not hopeful to see it catch up to anywhere near TeamCity, even Bamboo in user or developer UX.
For multijob pipelines checkout https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Plugin
Jenkins 2.0 will provide support for build chains via the pipeline plugin which will be included as a core plugin.
Access control can be configured quite nicely with the Role Strategy Plugin: https://wiki.jenkins-ci.org/display/JENKINS/Role+Strategy+Pl... + AD/LDAP integration.
Reporting automation is covered by the pipeline plugin. Custom reporting is quite easy to implement on top of Jenkins, its APIs cover almost everything you need.
Now our build and test suite is just a script each and it can pretty much on on any CI server.
What about cascade builds? The pipeline can't be handled in your build script.
What about reporting? (list of compiler warnings in the build? List of tests that failed? In general extracting information from the log)
What about efficient email alert? (with blame list, list of changes, etc.)
What about IRC reporting?
Jenkins is not very good (and written in Java :() but the other available free alternatives are just not better.
Too often I see engineers bias heavily toward inexpensive, free/cheap, or home-built solutions simply because they personally don't think things should cost money, when the understated anchoring costs of building and maintaining, plus the opportunity cost of valuable people time, might shift the toward other alternatives if a wider business context is taken into consideration.
If we're talking about hosted solutions, I agree it's mostly silly to refuse to pay... unless we're talking about open source projects, where many CI vendors do have a free tier...
Anyway, I also see an opposite bias toward costly hosted proprietary solutions and a refusal to even think about how one could accomplish one's goal with, say, a couple of shell scripts and cron jobs—or a self-hosted open source system.
That bias can be very sensible, but it can also be a hindrance, especially when it turns out that the proprietary solutions aren't quite what you want anyway, but since you're opposed to writing your own scripts, or running a more flexible system on your own servers, you settle for a suboptimal setup...
This bias I'm talking about is also a way of undervaluing something. It's saying "I'll pay $99/month for some hosted thing, but I'll never spend a week of engineer effort on really getting it right."
You can do a heck of a lot more in scripts; you've got a full language with all of the libraries it comes with in order to do stuff.
We've build our current build pipeline with Python scripts:
0. Bootstrap the build environment (installs everything you need to build). 1. Build projects order, order is derived from the intermodule dependency graph. 2. Execute unit test sets based on build parameter + save results in output. 3. Execute integration tests based on build parameter + save results in output. 4. Build distribution packages + save in output. 5. Deploy distribution packages to server based on parameters. 6. Run UI tests against the deployed packages + save results in output. 7. Publish the bits if they're OK.
Most of these steps are in different scripts, which can be called either from orchestration scripts or one-for-one in Jenkins.
All we have Jenkins do is invoke the script(s) then process and report the results.
We've basically got it wrong with the design of our CI tools; they do too much. They should just provide scheduling, invocation, reporting and dashboards over the build process. Expose that as an API and let scripts integrate with it.
In any project, if the build fails, you want the latest commit automatically reverted.
Now, do you just copy and paste to each script, or do you have a chain of build processes and logic in the build system?
There is a separation of concern: the CI server logic is a refactoring of things that are common to any project. If your project have scripts that sends email on build failure, I think you're doing something wrong.
You mention at some point "orchestration scripts", this is exactly what Jenkins is supposed to provide help with: orchestration.
The important thing IMO is that the boundaries of responsibilities are clearly defined between what a project does and what the orchestration does.
Also, arbitrary shell scripts, (as usual) provide duct tape for everything else as needed.
This is obscenely bad advice. We run thousands of builds/day and none of the configuration is maintained inside jenkins. All of our job configurations resides in repository alongside all the code.
Works just fine.
As much as Jenkins sucks sometimes, it's what many big shops use for many things. There is tons of community around it. It works, it's battle-tested, and it's been that way for years. Anything else will have a hard time catching up for as many use-cases as Jenkins currently solves for people.
If you hate a part of it, fix it and open a PR :)
We have a custom wrapper around the jenkins-job-dsl that encodes our conventions and templates to generate several hundred jobs, and it works very well for our medium-scale instance.
$12K/month for a much (I mean, it's like going from cvs to git) better solution in that situation should be seriously evaluated; perhaps you could give every team in that org their own dashboards and eliminate/distribute the whole release engeering org. $12K/year, which is the actual TeamCity pricing, is a no-brainer, which unfortunately most teams are way too cheap to pay for, headcount looks so much better. Fortunately at least cloud based CIs are starting to seriously take off, hopefully I never have to deal with a home-grown Jenkins deployment again.
So in my obviously biased opinion, if your team is big enough, either get an actual build/release engineer/devops guy or appoint one of the devs as an owner for this, otherwise you won't get the best results.
If you're looking at Jenkins purely as a CI then absolutely, it's overkill. I've seen Jenkins used as a distributed cron replacement, CI server, pingdom replacement, deployer and even as a means to give non-programmers permission controlled access to execute certain jobs via parameterized builds. Just having granular control to decide how to report on cron failures from a central location is a huge help.
It's ability to combine job execution, detailed result tracking, scheduling, scripting, SSH and failure reporting from a central location with flexible permissions makes it an extremely useful tool.
The extensions that the Cloudbees folks have put together are pretty sharp too.
since everbody's at it, i'll recommend github.com/drone/drone. drone really gives docker a pretty use case.
Looks like a massive facelift, and lots of work done to make things more intuitive, faster, and more pleasing to the eye.
And the problem with that is...?
- High startup time. Anything bigger than a hello world takes seconds, more typically tens of seconds, to start. My Jenkins setup takes 3 or 4 minutes before it is fully initialized. Even starting the JVM in client mode doesn't help much. I know, it's a server, and startup time doesn't matter once it's running, but it feels annoying.
- High memory usage. Heaps in the 1-2 GB range are not uncommon. This can probably be tweaked but as a user who is not a Java expert, figuring out how to do this (and figuring out what value is safe) is frustrating.
- XML configuration files. XML was hip in 2004 but these days it is frowned upon and users generally don't like it. This is technically not Java's fault (and I'm sure newer projects use something else) but it is something traditionally heavily associated with Java projects.
- Non-Unixy feel. Lots of things just behave like they're not really native.
1. High startup time is amortised over long run times. Jenkins is a general CI server, not an interactive build tool.
2. High memory usage is true, though again tolerable given the role and importance Jenkins will assume in any sane project.
This is really not a problem. If you're starting and stopping your Jenkins often, you're really doing it wrong. Jenkins is a server, and really should only get restarted when there's been an update. Not to mention, other large servers that are not Java still take time to startup... there's a lot going on during initialization.
> - High memory usage.
Perhaps more of an issue for some, but again, Jenkins is a build server, and should be on it's own hardware/vm, with it's own dedicated resources. If you're trying to squeeze by with 1GB of ram, you can, but you should expect the obvious results. The more complicated your builds, the more resources they're consume. Compiling GCC usually takes 1-2GB ram by itself. I don't consider this a problem.
> - XML configuration files.
If you're using Jenkins properly, you should not have to interact with any config files. Sure, if you're running it behind Tomcat or something, you'll need to dip into the configs, but not liking Jenkins because it has XML config files is shallow and superficial, in my opinion.
> - Non-Unixy feel. Lots of things just behave like they're not really native.
I'm not sure what to expect from a deliberately cross-platform system. Of course it's not "native Unix-like". But neither is any software written to support OS's that are not Unix-like. And again, you manage and use Jenkins from a web interface, so if you're on the command line, you're doing it wrong. (as an aside, Jenkins has native support for running shell scripts, which I make heavy use of... can't get any more unix-like than that).
Trying to automate jenkins job configurations requires using the terrible job config xml format and a poorly designed rest API which breaks all the time when you add in a plugin or plugins get upgraded.
Jenkins has a ton of issues but with a bit of careful plugin selection it provides a ton of bang for your buck.
Is this really still a complaint in 2016? Your phone has that much RAM.
Honestly if you're worried about Java heap size, spin up a t2.medium instead of a t2.small for your build pipeline. Running Jenkins on a Raspberry Pi is probably not a great idea if you're writing any serious application.
The JVM adds something like 400MB to every layer of a docker container resulting in multi gigabyte containers. Additionally the memory requirements for running a JVM put you immediately into the t2.medium range which will cost you 4X cost over a t2.nano for simply choosing to run Java instead of something with a smaller runtime footprint.
Don't run production code on a nano.
Don't run production code on a nano.
I don't care what language you're using, if you find yourself saying "well, I could run on a nano" just stop.
Perhaps this is the "writing on the wall" that containers aren't what you should be using to deploy your app.
Just because it's the "latest craze" doesn't mean it's the best solution for the task at hand.
Edit: configuration-as-code feature is available in jenkins 1.x with job dsl plugin [1], I'm using it at work, because managing 50+ jobs by hand was just insane. Nowadays we just have few dsl files :)
[1] https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
Oh, no! Seconds? Like five? Seven? For a process that you start less than once a day? Unacceptable.
> High memory usage. Heaps in the 1-2 GB range are not uncommon.
So... what? Spend an additional $30 on your build server and add 8Gb to it. Problem solved.
> - XML configuration files.
XML is fine. As opposed to Groovy, IDE's do a great job at offering auto-completion on it. But if Groovy is your bag, you can use that too.
> - Non-Unixy feel.
I have no idea what that means and I've been using UNIX for about thirty years now.
Using XML is always wrong. If you want to automate Jenkins setup with sth like Docker or Ansible, you have to touch the configuration, and that means dealing with XML, which makes me cry. There are many ways to do configuration reasonably. Not so related to Java but hand in hand, as somebody else put it: "Java is a DSL to convert large XML files to stack traces".
It's impossible to debug, and its plugins are impossible to debug. If it would be in a scripting language, you could see what's going on easily, maybe print debug messages, and wouldn't have to clone-edit-compile-(deploy)-run in order to debug.
When some Jenkins plugin doesn't work out-of-the box, I straight up give up on it, because I know there's no simple way to interactively dive into the code. Based on my experience, I really tried. YMMV.