Literate builds? | Hacker News Reader

Literate builds?

jenkins-ci.org

32 pointskohsuke19 comments

This sounds pointlessly complicated compared to just a shell script with comments that does the same thing.

If your build commands are so complex that you need a shell script, go write the shell script and then in the README you tell the users what shell script to invoke and any required parameters. The point is that you always will have a README where you say "run this command to build". At its most basic this formatting is exactly as you would format such a README. It also supports more a complex series of commands, but for the 90% there will just be one command in the "How to build" section

It looks like these guys are picking up ideas from the travis-ci.org project, which uses a ".travis.yml" file for configuration-

https://github.com/tedivm/Stash/blob/master/.travis.yml

This has huge advantages, especially in conjunction with things like git or github. With travis ci it's trivial for testing to be setup on new branches or even forks. We use Jenkins at work, and I'm excited to extend that level of functionality there.

Yea, this seems overcomplicated for something that can easily be smacked into a .travis.yml file with simple syntax.

Dream result would be what happened with RVM and RBenv merging their simple "select ruby version" files in that Travis, CircleCi, Jenkins etc. all agreed on a generic .build.yml file syntax and we could just use that.

While a standard .build.yml file sounds nice, I'm not sure we understand the problem well enough to figure out what the optimal syntax would look like. This matters because the syntax has large implications for 1) how simple your build process is 2) how much overhead your CI system can take care of, for you.

For example, at Circle, we're still experimenting with yml syntax for user-specified parallelization commands. Each different syntax we've looked at has different implications for what kinds of problems are simple, and what kinds of problems are expressible, using the syntax.

So yes, standards, but not yet :-)

Supporting a .travis.yml config file (or similar) should be relatively easy - as you mention - the multibranch stuff is probably the bigger "improvement" for Jenkins.

Ahem! The initial commit includes: https://github.com/jenkinsci/literate-api/blob/master/src/ma... however it is not completed yet ( see the comment in https://github.com/jenkinsci/literate-api/blob/master/src/ma... ) We had to prioritise which would get released first and I found the Markdown format more fun personally, so it got more love... plus you didn't give the YAML one any love either

If you don't like the sound of

"The literate plugin adds an Action to all Free-style projects that allows exporting these XML configuration snippets in a .zip file for unpacking into your project's source control."

then you might enjoy CircleCI (https://circleci.com).

We've always believed in understandable configuration checked into source control, with sane defaults and configuration over convention. The configuration (https://circleci.com/docs/configuration) is sane and human readable in yaml, yet incredibly powerful, and allows you set up a simple config in 5 minutes, and a complex config in an hour.

We tried Circle but even though their FAQ states to contact them if your tests require more than 1GB RAM, they can't actually do anything to give you more RAM, even if you're willing to pay.

Back to Jenkins we went...

That's changing soon! We're switching hardware to allow this. I expect that will be available in the next month. Let me know if you're interested in beta testing.

Right, exactly!!

When you build something to work for everybody, you make something that's not really great for anybody. That's Jenkins.

Instead, we built Circle to be amazing for a small set of people (web devs deploying to Linux). If you don't fit into that group, use Jenkins.

As time goes by, we're expanding our delightfulness to other groups of people (we only supported Rails at the start, for example).

In that vein, we're rolling out the feature to give you more than 2GB of RAM at the moment, so soon you'll be in the group that can have the delightful experience too :)

This is based on Don Knuth's Literate Programming: http://en.wikipedia.org/wiki/Literate_programming - but applied to build scripts.

(Readme is docs + executable)

I think one useful feature might be the ability to export existing Jenkins build jobs to the README files. Some ppl might be more comfortable with Jenkins interface rather than writing the README files from scratch.

Congratulations! You have invented the Makefile.

Make files aren't usually literate.

"Oh yeah? And who made you the literati?" :)

Totally correct, actually. Makefiles are the cruft of ages distilled through a minimum of ten little-understood tools of the ancients, classically employing a minimum of four distinct languages. Some replacements have been popping up of late... scons seems nice.

I'd love more details on your comment. What ten little understood tools and what four distinct languages?

Haha. It was more in jest as a caricature. You know... sed and awk are usually in there somewhere. bc. Some formal grammar from a parser generator or other. Maybe some document build system. Plus the core language(s) of the source code. Some obscure shell indirection or trickery. And perl, for good measure.

but not because they can't be. They aren't usually literate because few people spend the time to actually master make and explore how other projects have built up their make files to learn what makes an excellent makefile.

I would say that the terseness of the syntax and use of pipes makes it very literate once you know enough of the low level syntax of tools like sed, grep and awk to read them likes sentence fragments joined by pipes. It's parses very linearly as each line usually reads like a complete compound "do this, then this, then this" sentence describing a bunch of steps sequentially.

Reply on news.ycombinator.com