Hacker News Reader

Comment by ilikerashers | Hacker News Reader

Feature flags should be used judiciously. They are better suited to frontend-type changes, enabling a button or something visible.

Flags to enable a web service call or the like is clutter most developers can live without.

I find architects perceive them as the perfect fix for release management. To me, they result in too much config bloat with minimal value.

View on HN

I agree that feature flags, like any tool, should only be used where it makes sense. I disagree that they should only be used for visual changes though. Often, backend changes can have performance impacts that are only visible at scale, so being able to roll something out gradually can help there.

Also, a flag can be used as a permanent control to disable a part of the system (like an external call) if that part of the system is having problems. Imagine if you had a github activity feed integration, and you wanted to be able disable that feature when a github outage caused that call to take a very long time. Being able to easily disable that feature without needing to change your code can be immensely useful.

nb- I work for LaunchDarkly.

What are the situations where you did not (or would not) opt to use feature flags?

I usually skip feature flags for things like bug fixes where the risk that my 'fix' is actually worse than the initial bug is very small, or changes where it cannot make sense to have both modes operating at once. An example of this might be a new billing scheme, where the old system billed by the item, and the new system billed by the pound. There might not be a technical reason why both cannot co-exist, but it would cause marketing/support headaches to have some customers billed one way, and some billed another.

One surprising use case that I have had great success with, though, is in database migrations. I detailed my approach here, if you are interested: http://blog.launchdarkly.com/feature-flagging-to-mitigate-ri...

Interesting, thanks for the link!

No problem! If you happen to be in the SF Bay area (and if you will forgive the self-promotion) I'm going to be presenting this at a meetup in a couple weeks: http://www.meetup.com/AWS-EASTBAY/events/232870012/

How exactly does the api work? Is every call to check a flag making a HTTP request to LD or does the ld client sync periodically?

No, every call to check a flag is not making any sort of network connection. The flag ruleset (includes details like specific users that should get each variation, or particular user attributes that should indicate a specific variation, or a percentage rollout) is streamed to the SDK in a long-lived background connection.

The SDK maintains these rulesets in memory, and when you need to evaluate a flag for a given user, the ruleset is evaluated. This way, there is no I/O at the time that a rule is evaluated, so there is no real performance impact. At the same time, because of the way we stream the rulesets to the SDK, any rule changes take effect immediately.

https://launchdarkly.com/performance.html has some diagrams, etc, detailing this (sorry for the marketing fluff :)

Great, we will be looking at applying this in a new project of ours, and replace our existing feature flag implementations in other systems if things work out.

Strongly disagree. Working in operations, most days I don't care what the frontend looks like. But, for example, what if a developer wants to switch to a new recommendation algorithm? That's going to go behind a feature flag so we can ramp it up incrementally and switch back to the old algorithm if there are any problems.

Absolutely. In my experience, most unforeseen issues are related to performance at scale with unexpected data (the kind that seems to only ever crop up in production, and is really hard to design a load test for).

Performance issues in the browser are rare, and have nothing to do with the number of users using the application.

Can you elaborate? It sounds like you've had some bad experiences, would love to hear them. (I've had mostly good experiences.)

Sure. They create friction in a release process.

Flags set via config files require constant updates to said files, not a problem in a small team.

Work in a secure environment where config is controlled? You now need a process for controlling all these new flags.

Have hard dependencies on config (this flag MUST exist)? You tie your application to your config, both now need to be released at the same time.

Code clutter? Yes and it is avoidable. Also flags are not suitable to every type of feature so you're in a half way house anyway. Something which requires a DB change COULD be done via a flag but not worth the effort.

My experiences are mostly around CD pipelines that start breaking because of complexity. Feature flags might suit some application deployments and that's great. In everything I've worked on, they can be replaced by incremental (non-breaking) updates. In that situation anyone can do the release, no deep-feature knowledge required.

It sounds like you are using a bad tool if feature flag creation or changes require editing a config file. We use Gargoyle at Eventbrite and there is a nice API and UI for making changes.

I've seen a few feature flags turn into ways that a company/developer can gloss over fixing a really hard problem.

A personal example: we were developing a new insurance billing workflow. Since it was a huge change, and very risky, and not all health insurance companies make it easy to test, we rolled it out practitioner by practitioner or customer by customer.

We got to about 95% 'enable_improved_billing_flow=True'. The remaining 5% were turning into a huge project to migrate due to the specifics of their practice / their primary insurance companies / multitude of business reasons (those ~5% were an outsized percentage of our revenue). In the end, we ended up having to maintain two billing flows.

I'm curious: How do you think this rollout would have worked without the use of feature flags, and how would that have been better?

Reply on news.ycombinator.com