Show HN: Feature Flags Backed by Git | Hacker News Reader

Show HN: Feature Flags Backed by Git

flipt.io

93 pointsbullcitydev44 comments

Hey all,

I'm the founder of Flipt, an open-source (https://github.com/flipt-io/flipt) feature flag tool that I (and others) have been working on for the past few years.

About a year or two ago we added support for serving flag state from various non-relational declarative stores like Git, OCI, Object Store, etc.

Many of our users came to love the git-backed storage, as it allows them to keep their feature flags close to their code in there organization's repos. But they had one (rather large) ask, and that was the ability to use the UI to manage flag state changes and still use Git as the backend.

So today we're rolling out our hosted solution that does exactly that (and more soon).

I would love any feedback, questions, etc, or if anyone would give it a spin! Happy to extend the 14-day trial as well for anyone interested in providing product feedback.

Thanks!!

Open articleView on HN

This post comes at a great time! I've been wondering what folks prefer - git or DB backed configs and feature flags. How do you decide to go with one or the other?

What do you guys do when GitHub inevitably has an outage?

Nice thing about git feature flags: the state of your system is all in one place. It changes only when you do a code release, which includes feature flag values, or an infrastructure release. You can easily see, just off git history, when the state of the system changed (and what changed), which makes incident debugging much easier.

With DB feature flags, there's one more source of truth for changes to production infra.

(downside, of course, is that changing feature flag values is much slower using git vs DB)

This! 100% All the same benefits of config as code, infra as code. And with feature flags if something goes wrong its a simple `git revert` to get back into the previous state.

Another benefit is you can easily replicate the current (or previous) state of production/staging/etc flags locally just by doing a `git clone` and then run our self-hosted version locally. Its a single binary, can be installed with curl or homebrew and can read the flag state from your local filesytem.

This allows you to test your code locally or in CI with the same state in production

Maybe I’m not fully understanding what this product is doing. For something like configs/feature flags, it imo should be dynamic.

I was thinking that git (in a separate repo from main code) would be used to store the changes to configs, but then you would still need a system that tails the git changes and distributes them to clients.

That’s the way config serving was done at Facebook - a mercurial repo with all configs and tooling to edit configs which creates mercurial commits. Then, the mercurial repo is continuously tailed and values are saved to ZooKeeper, and then client libraries read config data from zookeeper / subscribe to updates / etc.

This is actually how it works.

Flipt is live tailing the repository and serving this dynamically to the clients.

The repo with flag configuration can be solely for flags, or alongside other infra configuration on in more of a monorepo. You decide how you want it setup.

Obviously if it is alongside code, you may to contend with CI in order to validate a change. But with the rules in CI or other monorepo tooling, what runs and when can adjust this behavior to improve time for configuration to become live.

Once a configuration change is integrated into a target branch in the repo, then it becomes readable for Flipt and servable once fetched.

I tried to answer your first question below, hope I did!

Re: GitHub outage, each org gets their own instance in our cluster and maintains a checkout of the git state, so you can still write/read from your environments, they just wont be synched to GitHub until they recover.

We're also thinking about adding other 'sinks' like S3/object stores and OCI as backup sinks.

I worked with a mesos config app, backed by git. Watching the whole go down when github died resolved none of us to ever trust it again for that sort of role.

Use s3. Honestly.

On this, we support publishing the state to object storage (S3, Azure, GCS) and to OCI as well in Flipt.

Flipt Open-Source can be run to consume from these locations. You can go as far as configuring a workflow to publish on push, so that you can combine our managed UI with any of these distributions methods through Git.

With any of these backends (including Git), we periodically fetch and cache data in-memory. Evaluations work on an in-mem snapshot. So temporary downtime doesn't propagate into your applications being unable to get flag evaluations.

Caching in this scenario isn't something I'd lean on unless you can invalidate or repopulate easily. I've used etcd in a feature flag scenario due to the speed of it's replication and it's ability to queried frequently without the need for caching.

Nothing special, because Git != GitHub.

I noticed that when trying to evaluate flags, the context only allows string values. Is that by design?

edit: it also seems silly to force users to choose an organization name with a big warning that they can't change it later. Why stop me from signing up? I almost didn't sign up because of this, but decided I would just try it out.

Yeah the context only supports a map/JSON object of k/v string values for evaluation at the moment.

We could definitely look into supporting other types and making this simpler. The constraints that are matched can be of several types, which we try to parse at evaluation time https://docs.flipt.io/concepts#constraint-types.

Also thank you for the feedback about not being able to change the organization name. Currently we use the 'slug' of the organization (based on the name) when we setup the routes for your environment, so it was just simpler to make it a one time thing.

But I think we could support renaming your org, it would just make the previous URLs invalid, or we could maybe handle redirects on our end too.

I’ve always wanted this to exist! Very nice execution.

Thats great to hear! Would love to learn more about your usecase or what else you'd like it to do.

Feel free to send me an email at mark <at> flipt.io, or drop into our Discord to say hi at https://flipt.io/discord

Very glad to see percentage-of-users experiments, that's where it falls short to simply deploy/rollback the new version.

Can this probably be done with GitHub Actions and GitHub Pages without any 3rd party platform (just no UI)?

Yes its just sshfs not sure why anyone would pay for this /s

Flipt itself is open source and includes the git backend. So if that works for you, great!

In our experience a lot folks came and said… but the ui is so important for us to be able to use a feature flag tool.

I am a user. What i would like is something very "simple", no server

Use the same SDK and ping github rest api to get the values of the the flag.

I know, this is very weird... But i cant deny it would be helpful for very small projects

Thats a cool idea! Feel free to put something in our community on Discourse (https://community.flipt.io/c/flipt-cloud/6) or Discord (https://www.flipt.io/discord) so we can talk through how this could work.

We've been thinking a lot recently about supporting 'edge' k/v stores like vercel/cloudflare and having the client side evaluation SDKs pull from there, which wouldn't require a server at all. This seems very similar just with using GitHub. We could even use GitHub Container Registry as the store as its OCI compliant.

Thank you for using Flipt btw!!

cool. Looking forward for this.

Also S3 (+ azure and GCP) would be a good 'edge' k/v store.

Someone could start with Github in a simple project, migrate to S3 to handle more requests and then migrate later to a full server solution... All of this keeping the same client codebase

Love this, and definitely where we're going.

We already support S3, Azure and GCS, as well as OCI (any compatible registry) as a source in the open-source server-side evaluator. So if you pop a deploy step to any of these sources from your Git repo, you can use them via the Flipt server process as a source of truth in production. Our server-side and client-side SDKs can source from Flipt in these scenarios.

But, we are keen to both explore skipping the Flipt server middle-man for client-SDKs, as well as make the publish step to these locations a simple configuration process in our UI. To avoid having to write things like GH actions to achieve the end to end result.

One thing to note is that we currently only support GitHub organization accounts.

We're working on personal accounts next, but due to GitHub API/apps permissions, we have to request a different set of creds to get that to work.

I like this design choice!! Wondering, what will be the biggest drawback pop in your head compared to traditional DB config or feature flags solution?

Complexity of initial implementation was certainly one, as we developed it. It’s not the most well trodden path for this kind of problem (well trodden for other kinds of apps). Obviously it lacks things like relations and schema, that we have to build on top of data in flat files.

One thing is that running Flipt open source on your infra, means running replicas all sourcing from the same Git repo. They currently polls for updates and this means eventual consistency comes into play when you scale. We have plans to help mitigate that with cloud though (pushing updates from cloud to your self hosted runners).

You can get around most of the consistency problem by "scheduling" the change. So, if I know it is going to take 2 minutes to make the flag available to my entire infra, I can schedule it for 5 minutes from now (could even make this configurable—a "default feature delay") which moves the consistency problem to infrastructure clock-sync.

Thats a great idea! I hadn't thought of combining it with a schedule for when a change is readable.

Can this do anything spring cloud config can‘t do?

This is my question. If your flags are in your repo, what is functionally the difference from just having them be variables in your code somewhere.

"Feature Flags that live inside your code" - That's just variables, no?

Feature flag state is still served dynamically through Flipt. Your code doesn’t have to redeploy for the changes to “become live”. That’s the main benefit.

Means you can experiment and target different cohorts with variants of your app without restarting processes everywhere.

Thank you. I guess I was imagining that the flags lived in your source code repo, and required a commit and push to update, thus triggering some CD build and redeploying your app anyways.

You're not wrong there at all. That is a very reasonable assumption and I think the default behaviour with most early CD pipelines. Every commit leads to a deploy event.

However, this can be changed, so that not all commits/pushes are treated equally during CD. Either by using rules to ignore changes to certain sub-directories / files or through having reproducible builds and skipping the process restarting parts when the resulting artefacts between two commits haven't changed (e.g. the digest of a docker image not changing from one commit to the next).

This is often an optimisation though, and takes time/effort to put in place.

Love the design!

Thank you!! Very much appreciated

the sound mix on the video demo is unbearable for me. the vocal is lower than the background music.

yeah sorry, I will fix that! It was bothering me too. I put the music to the lowest setting.. I used https://kite.video/

The voice volume seems to get quieter in certain spots, even when its set at a constant level.. I'll reach out to the Kite team

Kite founder here–Sorry about that, we’ll take a look. We have some auto normalization turned on that we obviously need to tweak. Thanks for flagging!

Also happy to manually fix this for you in the meantime if you send us the project file (instructions in the in-app chat).

Hey! thanks for the reply. We are loving Kite, great product!

Will send over the project file now, very much appreciated!

If a user can turn a toggle on and off using a UI, what is even the point of Git?

Auditability and, presumably, the ability to require approvals to commit changes are nice to have. I would assume the editor is just a UI to generate a commit. Maybe even to approve them.

Approvals are coming next btw. With the ability to lock down envs like production and optionally require all state changes to go through a proposal/approval process

Full version control, which can be collocated with other configuration for the rest of your system (thinks terraform or k8s manifests) means it becomes easier to build a picture of how your system was configured at a given point in time. Because you have a single history to walk.

looks awesome

Reply on news.ycombinator.com