Comment by teraflop | Hacker News Reader

Comment by teraflop | Hacker News Reader

Parent Full thread

It's cool that they're promoting the "pipeline" plugin to a built-in feature, but the devil is in the details.

Under the hood, it's implemented by taking a script written in a "DSL" (it's actually Groovy code), transforming it into continuation-passing style, and serializing its state. This is pretty cool from a theoretical CS perspective, but having played with it a little bit, the implementation seems very fragile. There are a number of long-standing unfixed bugs that cause the transformed code to misbehave in ways that you wouldn't predict just from looking at it, even if you're experienced with Groovy. I ran into a couple of them just during a brief period of experimentation. For instance:

https://issues.jenkins-ci.org/browse/JENKINS-27893 (varargs are mishandled)

https://issues.jenkins-ci.org/browse/JENKINS-28277 (currying doesn't work)

https://issues.jenkins-ci.org/browse/JENKINS-26481 (calling a closure in a loop doesn't work right)

https://issues.jenkins-ci.org/browse/JENKINS-28183 (killing a pipeline job doesn't clean up its state properly)

And there are inherent limitations to the approach; for instance, you can't store any non-serializable values in local variables, which means simple things like foreach loops don't work (because you can't serialize an iterator).

I really like the idea behind the Pipelines feature, in principle. But I think building it around a general-purpose programming language, and then failing to support all of that language's features, is a recipe for headaches. If you never try to do anything more complicated than what's shown in the examples then you should be fine, but the borders between what works and what doesn't are very ill-defined.

EDIT: Oh yeah, and there's a surprising amount of functionality that isn't documented anywhere except for blog posts and JIRA tickets.

the broken for loop functionally wasted an entire day for me because I didn't know if the problem was me or groovy or Jenkins, being unfamiliar with groovy.

> Under the hood, it's implemented by taking a script written in a "DSL" (it's actually Groovy code)

It's Groovy code? The announcement page doesn't mention "Groovy" anywhere, let alone "Apache Groovy" which is what the backers of Groovy are meant to refer to it as since last November (2015) when it was promoted from the Apache incubator by the ASF.

Yep. You'd think this kind of thing would be considered significant enough to mention, but apparently not. They do mention it in the "tutorial", which AFAICT constitutes the entirety of the official documentation: https://github.com/jenkinsci/workflow-plugin/blob/master/TUT...

> A pipeline is a Groovy script that tells Jenkins what to do when your Pipeline is run.

DSL are certainly the future of this kind of tools but not with a dynamically typed language like Groovy. The fact that it's dynamically typed makes it pretty much impossible to get decent support from IDE's. Gradle's support in IDEA has been broken forever, and you know that even if the JetBrains guys can't pull it off, it can't be pulled off.

I think the future belongs to statically typed languages that can offer DSL's as terse as Groovy but with actually types in them, so that the IDE can actually do the job it's being paid for.

> I think the future belongs to statically typed languages that can offer DSL's as terse as Groovy but with actually types in them, so that the IDE can actually do the job it's being paid for.

Not as easy as it sounds. If the DSL is rich enough, and if the types are handled by the host language, type errors can become just as cryptic and unhelpful as with an untyped DSLs, and it's just as hard for the IDE to do something meaningful with them.

Sure, but at least, auto completion works automatically.

With dynamically typed languages, you get cryptic (and vague) errors and auto completion just can't work.

I use auto-completion in Python every day with PyDev. It's not as advanced as for other languages, but "just can't work" is not really true either.

http://www.pydev.org/manual_adv_complctx.html

That also depends on the namespaces used by the language. E.g. auto-completion in Clojure works a lot better than in JS, because every symbol has a unique resolution, known statically (or as unique as in many typed languages). True, you get less helpful suggestions, but OTOH, the error messages can be much clearer than in the typed case. The former can leave error reporting to the DSL, while the latter is restricted to the cryptic types of the host language's compiler. I'm not saying this can't be resolved with some clever compiler plugins, but if we're comparing options available now, the advantage is not so clear-cut (I do favor types, but only as long as they don't get too complex).

Reply on news.ycombinator.com