Show HN: Hocus – self-hosted alternative to GitHub Codespaces using Firecracker
github.com
github.com
Anyway, excuse my old man rant, awesome work and good luck making a business out of it! I'll give it a try.
* It has the dreaded "View License" on GitHub
I remain a firm proponent of FLOSS, but I am also realistic about the value of truly great proprietary software!
This mirrors my own experience with JetBrains products, in particular with their Java IDE in large enterprise projects (4000+ source files). NetBeans kept freezing when I tried to use code search, whereas Eclipse seemed to randomly crash. I also recall the data source and MyBatis XML integration (with plugins) being better in IntelliJ IDEA as well - even though the startup indexing is slow and I also needed to increase the max memory for the IDE, same as with others (a few GB per instance). Aside from that I might just like the NetBeans UI a bit better but the language support isn't there.
This was a few years ago, but JetBrains still seems to have great code completion, tools to explore the codebase (dependency graphs, ER diagrams for DBs), as well as support for most of the popular stacks out there, which is why I use their Ultimate tools package with everything. Only Fleet was buggy when compared with VSC, but it was tech preview when I tried it. Oh and the PyCharm experience with Python and venv was a bit off when there were build errors (they weren't displayed prominently when you used the context option to install dependencies), but Python in general seems to have issues on Windows sometimes with compiling drivers or whatever.
Edit: oh wait WebStorm had issues with refactoring in a few thousand line long AngularJS controllers in one legacy project.
To me it feels like software like that should be free, but sadly that's not the world we live in. I personally lucked out with their student offering, graduation discount and recurring discounts. For smaller projects I still do some editing in VSC, not a Vim/Emacs user personally. Cloud based dev tools seem... interesting?
Do you mean that the license is not a commonly recognized open source license?
I have a team of 30 devs, so it would be $1,200/mo. That's on par with GitHub's enterprise plan plus Codespaces instance usage.
There's no way I could consider this for my company at that price.
256 cores would be about $2.50 to $5/hr in AWS depending on reservations and how much you're able to oversubscribe. That's another $1,500 to $3,000/mo on top of your $1,170/mo.
If you're paying for the compute then cool, that's a good price. If this is self-hosted though then I can't really see why I would bother unless I had no choice but to self host and cobbling together a simpler if less ideal solution isn't allowed.
Disclaimer: Know the authors, who are extremely bright engineers.
Based on my discussions with one of them, Hugo, re the architecture a couple weeks ago, I really think you should take a look at this if you want to do auto-provisioned dev environments in a self-hosted fashion, they know what they're doing.
1. Comparing to something like GitPod (which lets you run on your own instances as well), where do you think Hocus shines?
2. Given you're leveraging Firecracker for isolation, and Firecracker doesn't support GPUs, I assume that adding GPU-enabled machines isn't on your near-term roadmap?
1. GitPod doesn't officially support self-hosting anymore - https://news.ycombinator.com/item?id=33907897. When they did, it was extremely hard to set up. In fact we tried to do it at my previous company several times and failed. Hocus is designed with self-hosting in mind, and we want to make deployment and management straightforward. And from an end user's perspective there are 2 main advantages:
- Hocus dev envs are VMs instead of containers, so you can run anything you want inside. For example Kubernetes or nested KVM.
- Workspaces start in <5 seconds, while Gitpod can take a minute or longer.
2. Actually we are exploring moving to Cloud Hypervisor or QEMU, so we may support GPUs sooner than later. If you have a specific use case in mind, feel free to contact me - contact info is in my profile. I'd be happy to hear why you need them.
1. Good to know about GitPod - I haven't looked at it for a while so looks like I was outdated. The rest of what you said is good too.
2. This is mostly for ML development, where GPUs are sadly often required even for dev work.
With Hocus you specify your dev env right alongside your code in your Git repository with a hocus.yml file (https://hocus.dev/docs/hocus-yml). Hocus is deployed on a set of servers you manage (only 1 server during alpha). It has a built-in CI system that watches your repository for new commits and continuously builds the environment - we call this process a prebuild. You can specify that your dependencies should be installed and your code should be compiled during a prebuild. You can also describe tasks that should run when you open a dev env - for example automatically starting the app you are working on. Once a prebuild is ready you can spin up a new dev env - which we call a workspace - and connect to it automatically with your IDE. We also have a VSCode extension that automatically opens terminals connected to those tasks from hocus.yml. All this combined allows each team member to connect to a fresh, ready-to-code dev env anytime they want, whether they are reviewing a PR or writing code themselves.
> watches your repository for new commits and continuously builds the environment
That feels wrong to me. For example, what happens if I want to check out a 1 year old commit and edit it? Will the dev environment that was used at the time be available or will it be rebuilt? If it gets rebuilt, there's no way I get any guarantees it's going to work.
IMHO most of the industry is getting it wrong when it comes to dev environments and build systems. I've always called it butterfly effect builds. You make a commit and it triggers a ton of non-repeatable builds. There's not much value in doing that for every commit IMO. What happens when something like an APT mirror fails and the whole system grinds to a halt? What happens when a runtime or dependency gets updated and breaks at exactly the worst time during the week?
I'd rather see something on a (calendar) schedule with the potential to trigger a manual build if needed. For example, skim this [1] article. That's always made sense to me and, in the context of a dev environment, why not build the base dev environment once a week (or day) and only layer in the resources / code on-demand? IE: Build the things I don't control on a calendar schedule and only build on-demand for changes that are part of my project (using the base env).
The way I've always envisioned it is to have a stable env build once per month or per week. It's fast enough to keep everything rolling in terms of being current, yet slow enough that I'm not spending time wondering if a problem I'm debugging is caused by a change in my dev environment. I try to do the same thing as dedicated build containers for projects, but it's hard because everything is tunnel visioned on per-commit re-builds these days.
Your dev environment is version controlled too because it lives in your repository. You would run a prebuild for that old commit and get a dev env from that time. It would only break if your old dev env can no longer be built, for example if it depended on some external resource which is no longer available on the internet.
> I'd rather see something on a (calendar) schedule with the potential to trigger a manual build if needed.
That's actually on our todo list, but it didn't make it into the 0.1 release. This feature is especially useful for monorepos, where rebuilding everything on every commit is too much.
That's what I was getting at. Version controlling a config doesn't make a build deterministic when it depends on transient resources that aren't in version control.
There's nothing repeatable about a container style build. They can change between runs that are 5 seconds apart. I see a lot of Docker builds where the author claims they're repeatable because they're version controlled and then they start with 'apt-get update && apt-get upgrade'.
At least you don't claim yours repeatable, but I'd much rather build my dev environment on the first Monday of every month, check it in to an artifact repository, and use that for a full month. The re-build would be similar to (using Docker as an example):
FROM mygroup/myproject:2023.04
RUN git checkout ...
From the perspective of being the developer, I'd rather specify a release cadence and have the matching dev environment used. Ex: monthly, weekly, daily, ci. IMO container like systems that are used as part of the dev / build chain should be promoted and reused, not comitted and rebuilt. The only person using a CI based build environment should be the person responsible for maintaining it. Everyone else should be on a slower cadence.
It's seems so weird to me to come to HN and see people livid about Windows auto-updates and then having no problem with their dev and build tools (potentially) change 50 times a day.
To put it another way, would you auto-update VS Code, your runtime, and your dependencies every time you commit to Git? If not, why build the auto-magical dev environment so it works like that? Maybe I'm misunderstanding what it does. Admittedly, I didn't try it out yet.
We're going to be going source-available under ELv2 soon as well. Good to see more companies lately using the Elastic license.
- Ease of deployment and management are central goals of Hocus, with minimal assumptions made about the underlying OS and hardware.
- Hocus should provide a native development experience with performance on par with bare-metal development.
- Hocus should be scalable to accommodate large teams, with support for thousands of users and heavyweight repositories. We're not there yet.
Roadmap
- Add basic single node support
- Optimize single node storage usage, performance, and reliability
- Add permissioning for teams, admins, and regular users
- Add multi-node support
- Add support for more IDEs, particularly JetBrainsYou can really lower IOPS/memory/disk usage with an approach like this.
I think
> Add multi-node support
will be important for non trivial project development where the 2 distinct environments might be needed to talk to each other like say for prototyping or "sharing configs"..
- make dev envs privileged containers, effectively giving any random user root control over the machine where Hocus is hosted
- mount the Docker agent from host into the container, but that also allows the user to gain root control over the host machine
- use Linux user namespaces, and give you a root user with limited capabilities inside your dev env. That's actually what Gitpod - another Github Codespaces alternative - does. That root can't do everything though. It can't run Kubernetes, LocalStack, or the Elastic stack, because it can't modify some system limits.
A dev env backed by Firecracker is a full-fledged virtual machine that has none of the aforementioned drawbacks. It has some performance tradeoffs though, but we are working to minimize them as much as possible.
https://serverfault.com/questions/1043441/how-to-run-kvm-nes...
Have you given any thought to offering pre-baked development templates? For example, here is a standard Django build with poetry + js utilities + Postgres drivers + jq, etc.