Orchestra: An open-source system to orchestrate teams of experts and machines
orchestra.unlimitedlabs.com
orchestra.unlimitedlabs.com
Today we’re excited to be making our first contribution back to the tech community by open sourcing Orchestra, a system for coordinating project teams of experts and machines. It can be used to reimagine and implement all sorts of workflows, from professional services (like legal, design, and data analysis) to new product experiences (like M from Facebook). Orchestra is based upon years of research into machine-mediated expert teams. We hope you’ll come up with new and creative ways of using it!
At Unlimited Labs, we’re excited about building a brighter future of work and are currently building our first product on top of Orchestra. We’d love to answer any questions you have on here, or feel free to email the team at hello@unlimitedlabs.com!
If you're curious how it can be used, Daniel Haas put together a wonderful example of how Orchestra could be used in a newsroom:
http://orchestra.readthedocs.org/en/latest/example_use.html
Happy to take questions!
1. I'm going to play with this as a way of documenting processes. Being able to answer "how do you develop features" or "what's best practice for X" with a usable process is great.
1 a) I'm particularly interested in using this for documenting how to perform experiments, and tracking experiments as they go.
2. I think this is amazing for progressively automating things. Split the task up into manageable, clear chunks to be performed by people. Each step can then (if it seems worth it) be replaced by code.
3. I don't think this is present at the moment (and it probably shouldn't be for a first release) but I'd like to see process hierarchies and importing others. I'd like to say 'Use github's "GITFLOW V2" process for managing feature additions and use "MY_COMPANY_QA V3" for reviewing features before full release'.
Anyway, this is great, and I look forward to playing around with it.
edit -
Even without these, I can already see myself using it just for me.
edit 2 - It'd be good to work on the getting started, so that there's just a "run this script" or "clone this repo". There's a lot of "download this file and copy it here" that feels like it should be scripted.
Your comment about embedding external QA workflows is super interesting. Right now Orchestra does support what we call 'Review Policies', which allow you to specify that a given workflow step be reviewed k times, or with probability X, or by specific expert Z. But it would be interesting to think of whole workflows for QA.
Re: getting started, yeah, we're working on a script--turns out cross-platform support is hard!
That's handy, I can think of a bunch of things that fit well with that (particularly review a subset of x%)
> Re: getting started, yeah, we're working on a script--turns out cross-platform support is hard!
Hah, yeah it is!
I've got a version setup now, but am struggling a bit to setup workflows & figure out quite what goes where, but I think I've found the right bit of documentation: http://orchestra.readthedocs.org/en/latest/example_use.html
As a suggestion, if you add a new example, try doing it step by step of what you need to add & do. It's more work, but I find it really helps show what someone will actually experience while creating / updating workflows.
Also feel free to reach out to us on gitter (https://gitter.im/unlimitedlabs/orchestra) if you're having trouble getting your local setup to run the workflow and we can talk through the details.
Re: MTurk, there's nothing stopping you from stitching microtasks together with Orchestra, but I think that undersells Orchestra's strengths (e.g., collaboration among workers) a bit.
It seems high-level (and great) I'm just curious what implementing it "in the field" would look like.
Research from the Flash teams project that I mentioned in another comment (http://stanfordhci.github.io/flash-teams/index.html) has shown that this type of automation can decrease end-to-end workflow time by as much as 2x!
It would be nice to be able to create tasks from stories in Pivotal, assign a reviewer to them, and, when a reviewer thumbs-up, have it automatically built into a staging server. Anything like that in the works?
On the research side of things, the flash teams work by Daniela Retelny at Stanford (http://stanfordhci.github.io/flash-teams/) has done great proof-of-concept things like build an app in a day with distributed teams.