Hacker News Reader

Comment by xg15 | Hacker News Reader

And? All that is written in the article. The question is whether or not forcing users to partake in field trials is a good idea.

View on HN

But that question doesn't make sense. It contains the assumption that the alternative would be to never ship broken features to users. While the actual alternative would be that they'd ship the same broken feature to all users. Because at some point there's no other testing left.

If they're shipping features that need testing, it seems like the Beta version is the right place to ship them, not the stable Chrome or Chromium build. As in: ship the possibly-broken features to people who have already opted into testing, and don't subject your users that chose the "stable" to random testing.

Getting the bad end of someone's unannounced feature testing is the kind of thing that makes me decide to use another browser for a few months, hoping that the problem disappears if/when I go back to it.

If they're not running new features through their Beta channel, then you have a point. Otherwise, I'd assume the features that make it to field trials have already gone through Beta testing and are thought to be ready to release--which isn't always the case, hence the gradual rollout.

I'd say that if they're doing A/B testing or "field trials", then they aren't sure enough to roll the feature out to Stable. That's not Google's policy, though.

They run experiments in Canary, Dev, and Beta, then conduct a roll-out to stable to get a broader test (per the explanation here): https://textslashplain.com/2017/10/18/chrome-field-trials/

I'm just grumpy that we don't live in the perfect world, where confusion-inducing processes like this wouldn't be necessary. I've never been that comfortable with the "break a few eggs" method of progress.

The piece seems to suggest that a) multiple processes were the performance problem and b) that a field trial was causing that and "mak[ing] my computer unusable".

Multiple processes do take more memory, but there's a good reason for them. While it's possible that it was a field trial that was causing issues here, there's no evidence to support that. Sadly, it was probably just excessive resource usage independent of any trials: Chromium did have a period where memory usage especially was allowed to drift and that was an oversight that Speed team are correcting (https://chromium.googlesource.com/chromium/src/+/lkcr/docs/s...).

On the other hand, field trials produce important feedback. One example just from today: TLS 1.3 draft 18 had significant issues with middlebox bugs to the point where it wasn't viable to deploy it. Chromium has been using field trails to test variants of TLS 1.3 intended to work around these issues and allow TLS 1.3 to be used for real. This has resulted in concrete changes to help TLS 1.3: https://www.ietf.org/mail-archive/web/tls/current/msg24908.h...

These sorts of things require large-scale testing. We brought all the middleboxes that we could find for local testing, but that cannot account for the variety of firmware versions and configurations used in the wild.

The point of the post is that there is no way to opt out. We all understand more datapoints the better. But I also need to not be fucked with when I'm trying to work.

If you know the name of the field trial you should be able to opt-out on the command line provided you use the correct syntax with the --force-fieldtrials option.

However it was not established whether or not field trials (A-B testing) was even the problem merely that they existed on the chromium processes. Even the stable version of the browser will likely have these options on the command line.

> But I also need to not be fucked with when I'm trying to work.

But that's true of shipping any bug and isn't guaranteed to happen with a field trial any more than any new feature.

(and the OP doesn't seem to establish that it has anything to do with the field trial, just that the processes were run with that option? Admittedly I don't know how they actually work :)

I think the point is more about intentional performance degredation versus unintentional; it makes sense that you can't ship a 100% bug proof piece of relatively complex software, and some has to be patched out later, but such performance degredation or workflow interruptions can be planned for. But to intentionally enable an function which knowingly degrades performance or impedes workflows just for the sake of testing without the consent of the user is a bit rude. To repeat the author and other comments, if it were a beta-channel or test build where it was clear this was part of the intended functionality of the build, I don't think we'd be having this discussion.

> But that's true of shipping any bug

The difference is though, that I can choose the time I update Chromium myself.

Not to worry, chromium is open source. Just pop open a text editor and trace the code from the option parser to see where it's used. How hard can it be?

That TLS anecdote is pretty fascinating. Middleboxes are holding so much back on the internet. I wish the IETF didn't use such euphemisms as "middlebox robustness"!

I hope HN'ers push back against middleboxes at workplaces.

Reply on news.ycombinator.com