Runnable Sandboxes: Full-stack environments for every GitHub branch | Hacker News Reader

Runnable Sandboxes: Full-stack environments for every GitHub branch

blog.runnable.com

130 pointshiphipjorge54 comments

My company build something pretty identical to this a few years ago. It's INVALUABLE if you have a QA team or non-tech people who need to test things on different branches. Our workflow basically went like this:

1. Dev pushes branch to repo 2. Dev assigns ticket to QA 3. QA builds a sandbox server 4. QA tests everything under the sun 5. Any bugs are pushed to repo and auto-pushed to sandbox server 6. After QA gives green light, sandbox server is taken down.

Average uptime for a sandbox server was a few hours, and we caught a bazillion bugs this way. I'm no longer with that company, so it's really exciting that someone else has built something like this off-the-shelf so that I can continue with that workflow at future jobs. :)

Founder here. Check us out for your future jobs. Maintaining sandbox infrastructure is a pain.

One of our key lessons is being able to dispose and rebuild your infrastructure on a daily or weekly basis so leaks don't destroy your environment.

We recycle customer environments in Runnable, on a regular basis and its completely transparent to users

This looks great, thanks for sharing -- I was planning to write something like this for my build pipeline, but having something available off-the-shelf will save a lot of time.

Eventually I'll be moving our Docker containers to run under Kubernetes, which might necessitate writing my own thing to replace this; I'd ultimately like to be able to deploy my k8s config files into a per-branch/per-commit namespace, so that the full deploy pipeline can be tested.

This is really a great idea, imo. At my last gig we had four projects in GCP using kubernetes to deploy various clusters, and essentially all but one of those projects were for testing branches. Will be keeping an eye on this.

Any plans to include Bitbucket repositories?

Bitbucket offers a more favorable pricing policy for development agencies, which would potentially be heavy users of your service.

Bitbucket is on our roadmap, just collecting feedback (and hiring developers) to see if we should deliver it sooner

This looks very similar to what is offered by https://platform.sh/ . Anybody familiar with both could contrast them?

According to the website it looks like platform.sh has a focus on drupal, symphony, & php while runnable is platform agnostic.

Co-Founder of Platform.sh here: I am not extremely familiar with runnable.com so I won't comment on them.

Anyway, this is true we currently focus on the PHP market. Though you can already deploy nodejs applications (or a mix as we support complex multi-app clusters).

The platform itself is totally abstract so we will probably add in the near future many other run-times.

The main difference I think it would be interesting to point out is that Platform.sh is a production system... not a staging thing.

What we clone into staging is production .. with all the data.. however complex it is. And we do it really fast.

And we run production in a multi-datacenter / scalable / highly-available setup.

BTW we are hiring like crazy so if you want to work on your pet language having our features... contact me :) We are a fully distributed company... so u can be anywhere the world.

To OriPekelman's point, our strategy is to provide a great development and testing experience for containers.

You don't have to run production on Runnable. We are complimentary to your existing prod system. Most of our customers run on AWS.

Sounds a lot like Heroku review apps[1]

1: https://devcenter.heroku.com/articles/github-integration-rev...

Hey appleton - Here are some differences

1) You don't have to re-configure your environment variables and 3rd party APIs for each branch on Runnable. Every branch runs in the same sandboxed environment with the same env variables and 3rd party services. With the help of an HTTP proxy and a dynamic DNS server, we can route traffic to the right set of containers for the branch you want to test.

2) Runnable is built on Docker, which means all your configuration files and the way your containers are run is non-proprietary

3) You can connect arbitrary branches with each other for testing. So if you have a feature branch on a web server, and a feature branch on an api server - both can be connected easily through our web interface. This allows for cross-branch testing. Underneath the hood, we swap the IP address to connect the right containers together.

4) Our environments spin up much quicker than Heroku's. We build and run on the same machines so there are no network transfers and we can utilize build cache better.

5) Databases clone instantaneously with Runable, versus waiting minutes (depending on size). This because our build system applies Copy-on-Write to database containers

6) You can run end-to-end integration tests as your code is being pushed to a branch. Heroku only creates an environment when you're done coding and a pull request is open

I wrote similar service on dokku few days before https://github.com/istarkov/create-your-own-heroku-review-ap...

And it's really cool that we have Heroku alternatives.

Can someone help me understand what's the difference from Travis CI, Circle CI, etc?

Travis CI and Circle CI push code to a staging or production environment. So you still need to setup a full-stack environment to do end-to-end testing.

Runnable let's you have end-to-end (full-stack) environments with every branch. The minute a branch comes up, you have an environment. This includes a clone of your database

How would this look like when multiple branches from different projects work together?

> Developers can pull individual components or services locally and connect to the rest of the stack on Runnable.

This sounds great, I wonder how this is done.

I am currently evaluating docker-cloud for the same purpose since all of our infrastructure is Dockerized. Can you comment on the differences?

Runnable engineer here.

> How would this look like when multiple branches from different projects worm together?

Each repository has it's own configuration, and each branch gets it's own container. We allow you to connect branches together, not dissimilar to DNS[1].

To work on individual components locally we have a couple of tricks up our sleeves, but at the end of the day we have a CLI that will allow you to do file syncing and ssh'ing into your boxes from your own terminal[2].

1: https://runnable.zendesk.com/hc/en-us/articles/209632083-Aft...

2: https://runnable.zendesk.com/hc/en-us/articles/208018696-How...

Hey All - Founder here. Happy to answer any questions or discuss some of the problems we've solved building on Docker.

I think this is going to be the future if you can find product market fit. I'd transition my team over to Runnable, but we just finished writing our own Docker deployment system.

Do the stacks run with Docker Swarm and Docker Networking? If so, does it work well? Any strange issues you had to work around, etc?

Our stack does use swarm under the hood. So far we have not seen any major issues with scheduling across 100 servers. However we have seen issues with the swarm event stream disconnecting. Our workaround was connect to the event stream of the docker engines directly and use the `since` parameter. We currently use weave net to handle docker networking and have not had major issues with it.

Nice to hear!

What does Runnable allow me to do that I can't on CircleCI?

CircleCI doesn't host your containers. They just build and test them. Runnable runs actual environments for your branches

Can you clarify the use case for that? Essentially allowing me to spin up full environments feature branches?

Runnable engineer here, let me see if I can clarify some things.

The core use-case is being able to manage and test feature branches. In particular, some of the tricky things with pull requests related to integration testing.

Each feature branch can have it's own isolated environment for testing, pointing to different branches for other parts of the stack. This makes it so you no longer need to manage your staging servers, deal with outdated components on the server or even have to wait to be able to test your code. It's always there, always up to date and unique per branch.

You could use the feature branch environment for multiple team members to work on at the same time- i.e. Some visual review of UI/UX changes as they're being worked on. Or you could have a QA hammering on that specific feature functionality in that specific environment.

I see it as a very nice way to isolate some ongoing development that you may want some special context setup that allows people to focus on what's going on in that branch only- without having to configure your CI/CD by hand every time you want to do it for a special case.

Right now we have snapshot builds from Jenkins built and deployed to the development environment from all commits to the develop branch. All the work going on in feature branches are ignored by CI until they're merged to develop in order to avoid churn. We have definitely had some feature branches that would've been nice to build and deploy regularly, but orchestrating all that by hand was more than we wanted to spend for the 5-8 days that the feature branch was going to be alive.

Also with integration testing, it is nice to screen out any bugs created when your new config params aren't in the environment yet or to test an upgrade of your DB schema and stuff like that.

Do you guys support large databases and can every branch get its own database?

Yes, every branch can get a clone of your gigabyte database. This works by applying Copy-on-Write to the containers running your stateful containers.

So it becomes very easy to test schema changes and migrations with this approach

Really cool. I actually hand rolled my own solution using a Node CLI and the AWS API.

What features/problems does Runnable solve over https://elasticbox.com?

Also, who did your promo video https://www.youtube.com/watch?v=mBR-_5dXH4w&feature=youtu.be)? They did an amazing job.

Gitlab support in the pipeline?

We're trying to hire developers to help us do that :-)

What's your stack like? For those interested...

Our stack is comprised of 15 stateless micro services all built with Nodejs. MongoDB, Redis and Neo4j are used for persistence, caching, and dependency management. The fundamental piece our system is RabbitMQ which is used as our event queue. The architecture designed around events. When an application dies or a commit is made, an event propagated through the system.

We also use a lot of open source software. Docker Machines + Swarm is used to schedule and run applications. Docker Registry is used to store images. We use WeaveWorkes weave for inter-container communication.

We are starting an engineering blog as well, keep posted for deep dives into our architecture! http://blog.runnable.com/

Interesting. I haven't worked a lot with Nodejs (Haven't really gotten to it), but have with the rest of the stack. I'm into messaging and networking. Maybe drop me a line?

What is your experience with using Docker Swarm in a production environment? I have heard conflicting views regarding it.

Responded in another thread here: https://news.ycombinator.com/item?id=11306045 Will be happy to go into more detail.

Here's a vote for GitLab-support.

I remember you guys were a cloud IDE service right? Did you pivot or is this a separate product offering?

Runnable Sandboxes is a new product we're launching.

But you might be thinking of our CodeSnippets site, code.runnable.com :)

Looks like exactly what we're looking for.

Why is it only available with a Github organization account though?

Support for other code repositories is on our roadmap — we are looking to hire more developers to help build support for them ! :)

We are using github, but just not github organizations...

The reason it's built for Github organizations is we can access-control your private infrastructure based on the permissions set in Github.

We don't have access to the same APIs with just a personal accout

On pricing it says - "We plan to announce pricing and general availability in Summer, 2016. Plans will be priced between $30/mo to $55/mo per active developer."

Are active developers determined from repository - or is there another way?

This sounds neat - are there plans for an on-prem version instead of EC2? We have Mirantis OpenStack running in a bubble at my organization (we are cloud-averse).

It's on our roadmap. Can you submit a support ticket and I can fill you in over email

This is really cool. Can you explain if I would use this instead of my Jenkins server, or in addition to my Jenkins server?

You might be able to replace Jenkins :-). Runnable builds every branch as soon as it's updated with a push, and then deploys it. This allows you to spin off branches for your app and use/test them in a matter of seconds. Builds are exportable as Docker images which are stored in an authenticated registry.

Would love to hear what your use-case is and whether it's a good for or not! Send me an email jorge@runnable.com

I guess for me it would be most useful if it also had a production service and pricing, any plans?

This looks very promising. I wonder if/how well they could handle a 500gb database.

Founder here - We absolutely can! Just give us a heads up so we can make sure it loads up properly on your account

How do you import multi-GB databases? Where can you import them from?

Database backups less than 5GB can be uploaded directly to Runnable (https://runnable.zendesk.com/hc/en-us/articles/208221743-How...)

Reach out to us for bigger backups, and we'll provide direct access instructions to your storage infrastructure.

Reply on news.ycombinator.com