Hacker News Reader

Comment by bsenftner | Hacker News Reader

So, a task/process queue? Is that what this is? How does this vary from the quite mature task/process queues used by media production companies? (Examples include Deadline, and dozens of tested-by-production proprietary queues at every production studio.)

View on HN

Hey bsenftner. Could you provide a link or an example of what you mean? :-) I googled "Deadline" but what I've found looks not correct.

I usually compare Gaia with tools like Jenkins and Spinnaker. For example, many people use Jenkins Pipeline (https://jenkins.io/solutions/pipeline/) which allows you to write CI/CD tasks in Groovy. In my opinion, Gaia fulfills this job way better because it doesn't force you to use a specific language. It's also super fast and provides features like the automatic (re)build of your pipelines.

> Gaia fulfills this job way better because it doesn't force you to use a specific language

CI typically seems much more dependent on bash/unixy sorts of tools to get things done. This seems to not really support that workflow, requiring code to define pipelines instead.

If it's intended to do CI, how do you deal with CI-style tasks, like shuffling around files between pipeline steps? Or the corollary, what does this do that makes it easier to do CI in practice than with typical unix command based workflows? Inherently, it seems like "Create a golang script that can start a subprocess that runs a test suite" is more overhead than "run a test suite".

At first glance, it looks like more of a competitor to, say, AWS step functions, but it doesn't sound like that's what you're targeting.

In the past you simply had to compile your application, package it and push it to a remote server. Nowadays, it's not that simple anymore. You often find yourself writing scripts to create Kubernetes resources, manage remote APIs to create services which are needed by your application, talk to remote services (like HashiCorp Vault) to get credentials or secrets. Gaia does a great job here because you can directly use client SDKs in your preferred language to communicate with those remote APIs.

Have a look at the Kubernetes deployment tutorial, this might clear things up for you: https://medium.com/@michelvocks/automatic-kubernetes-deploym...

Here's Deadline: https://deadline.thinkboxsoftware.com/ I used it in the past, along with the render queues at a few VFX studios. Very feature rich process scheduling and scaling managers are in heavy use by media productions.

Thanks. In the end it is similar to Gaia (a task scheduler) but also comparable with Jenkins/TravisCI/CircleCI and probably hundreds more schedulers. :-) The basic idea is not new.

In my opinion Gaia is perfect for programmers which are "forced" to write automation tasks. It allows you to write automation tasks in your preferred programming language and makes it super easy for you to schedule them because Gaia comes with an automatic build feature (just provide the git url of your source code and Gaia does the rest). Additionally, Gaia is super fast, lightweight and open-source.

Reply on news.ycombinator.com