ankitdce

author here - a few things: 1) code review was always a lossy verification system 2) to eliminate the need for code reviews, we should build systems that can blend predictive execution with deterministic verification 3) if we cannot build bug-free systems, we have to resilient ones 4) a core value of code reviews is knowledge sharing - this we must still preserve through specs reviews

Have you tried https://github.com/aviator-co/av

Yeh, Attention Set is a game changer. We (Aviator) also took inspiration (ahem.. copied) attention set from Gerrit: https://docs.aviator.co/attentionset

If you are looking for an open source stacked PRs CLI, you can look into av CLI (https://github.com/aviator-co/av). Unfortunately this also only works with GitHub, but it should be possible to add support for any Git-based platform.

Disclaimer: I'm the founder of Aviator who supports the av CLI. It's a free tool to manage stacked PRs.

There are a few free open source tools available these days to manage the stacked PRs. Eg: https://github.com/aviator-co/av

Background: I've worked on all side of this spectrum, left Google 10 years ago and have been working in startups since then. And now founder of a startup.

I agree that from a financial outcome perspective, the odds of making big in startups may not be in your favor.

"Interesting work" perspective may need a bit more color. No matter if you work in a big tech or startup - majority of the work is very mundane. Even if you are working on the coolest AI big-tech or startup, you are probably not doing "exciting cutting edge" work most of the time!

This is probably also why folks who join a startup looking for exciting work get bored quickly with the day-today mundaneness.

Eventually what matters from the interesting work perspective is to understand what part of the work truly gives you joy - what makes you excited about getting off the bed and do great work - even the mundane one.

The motivation could be - seeing your product being used by millions of users, solving a hard problem, sharing your research with the world, having a great work-life balance, or getting rich. Understanding that can help answer the question on what is right place for you.

One challenge I've found in big-tech is that even if the company is doing amazing work, not everyone gets an opportunity to work on the most exciting projects.

Curious if investors are considered creditors through chapter 11. My understanding is that they will still lose the money, but the FTX users will recover everything.

Yeh, perhaps the big shift for the reincarnation was their ability to get a foothold and grow their enterprise sales business. Their strategy of building the Microsoft software ecosystem paid off very well. Also the reason why Slack could not continue competing with Microsoft teams.

The more interesting question would be whether they will go through the same downward trend again sometime in next few years.

Surprised this was still supported

Also shameless plug, check out out Aviator.co also provides merge queue support for GHE self-hosted and cloud.

Excited to see this thread. Aviator also offers a high performance MergeQueue with flaky test support: https://docs.aviator.co/mergequeue/managing-flaky-tests-in-m...

Disclaimer: I'm the co-founder of Aviator

On this Star Wars day, another take on monorepos!

But on earth, we have seen now several instances where teams have moved from polyrepo setup to a monorepo. Although "monorepo vs polyrepo" is always a debated topic, and it's hard to scale a monorepo, large companies like Stripe, Canva, Cruise, Doordash have been able to manage monorepos by building strong tooling and automation to handle the scale.

We're excited to announce "The Hangar", a community created specifically for software engineers working in developer experience space to come together, and learn from each other.

The Hangar already boasts a community of dev-productivity teams/ dev-experience teams from leading companies like Figma, Cruise, Square/Block, Slack, and Netflix. The group features open discussions around best practices, useful tools, relevant industry news and career tips.

At The Hangar, we host monthly “off-the-record” zoom hangouts where you'll find detailed insights on topics such as migrating to a self-hosted CI system or setting up Bazel correctly. This is a space where vetted experienced professionals can exchange ideas, share hard-earned wisdom, troubleshoot issues, and ultimately help each other in their projects and careers.

Join our community to talk shop anything DX: https://dx.community/

I've also had terrible experience posting on SO lately. Even though I have been posting on it for several years, they keep flagging my questions. Plus none of the questions these days get answered without a bounty. So, now I have to wait over 2+ days to get an answer.

Dev productivity metrics have come a long way from the time when one use to measure the number of lines of code written or the PRs created. I think Dora metrics is still a step up from that as it focuses on measuring team productivity vs individuals, and provides some indicator of the true bottlenecks for productivity.

For everyone who complains about monorepos, remember some of the most forward thinking engineering companies like Google and FB also use monorepos. All the arguments that people make in favor of polyrepo are making so because of lack of strong tooling for monorepos. That's also why Google and FB would not have scaled if they were using GitHub / GitLab but had to build their own. Also Google's original source control was built on top of perforce!

Google's Changelist are just standard PullRequests

From what I have heard, BuildKite seems to be a more reliable solution as it lets you spin up your own instances.

That's very interesting feedback. We certainly don't have a way to force simulate failure.

A related capability we are working on is to also rerun the identified flaky tests X times so they pass. This depends on the capabilities of the test runner, so it will work with specific ones first (cypress, pytest, etc). That way you still make sure that flaky tests pass instead of supressing.

That is a fair argument. Not all organizations have the bandwidth to measure and manage stability of builds. Some companies build internal tools / dev productivity team for this purpose. There are always right intentions to comment out the flaky test with the mindset of coming back to it, but it is also a very low priority item in most cases when you have to ship new features.

Fixing flaky tests can very commonly take longer than writing new tests.

Curious, how often do you see a flaky test in your system? In my past experience at one of the mid-size startups, we used to get a new flaky test almost on a weekly basis in a monorepo. We started the process of actually flagging them as ignored (we created a separate tag for flaky tests), but later realized that the backlog of fixing flaky test never came down.

In another case observed, devs just got used to rerunning the entire suite (the flakiness here was about 10-20%)

Haha great point. Well from what we have learned from our users is "fixing" test typically end up with "delete most of them". Fixing tests can be time consuming effort.

Another way to think about it is, whether Flaky tests are worth keeping? At some point if the tests fail often, do these really add value. And we think - it does. If you are able to identify flakiness from real failure and reduce noise, you can still avoid real failures.

Yeh flaky tests generally creep up in a big service due to several issues. There are some best practices to avoid the tests that requires some discipline and good oversight! We wrote some stuff around it: https://www.flakybot.com/blog/five-causes-for-flaky-tests

This is by no means an exhaustive list, but our goal with FlakyBot is to get better at identifying root causes as we identify flakiness across the systems.

Hi HN, we are Spriha and Ankit building Flakybot is a tool to automatically identify and suppress test-flakiness so that developers are better able to trust their test results.

Most CI systems leave it up to teams to manually identify and debug test flakiness. Since most CI systems today don’t handle test reruns, teams just end up with manually rerunning tests that are flaky. Ultimately, tribal knowledge gets built over time where certain tests are known to be flaky, but the flakiness isn’t specifically addressed. Our solution, Flakybot, removes one of the hardest parts of the problem: identifying flaky tests in the first place.

We ingest test artifacts from CI systems, and note when builds are healthy (so that we can mark them as “known-good builds” to use while testing for flakiness). This helps automatically identify flakiness, and proactively offer mitigation strategies, both in the short term and long term. You can read more about this here: https://ritzy-angelfish-3de.notion.site/FlakyBot-How-it-work...

We’re in the early stages of development and are opening up Flakybot for private beta to companies that have serious test-flakiness issues. The CI systems we currently support are Jenkins, CircleCI and BuildKite, but if your team uses a different CI and has very serious test-flakiness problems, sign up anyway and we’ll reach out. During the private beta, we’ll work closely with our users to ensure their test flakiness issues are resolved before we open it up more broadly.

That's a great question. FWIW Gitlab does have merge train with similar baseline functionality. There's two part answer to your question: - even though it's trivial from the face of it, since we specialize in merging process, we are able to provide much more complex workflows, such as batch mode. - starting with MQ, our goal is continue focusing in engineering productivity and build as we understand the use cases better. We imagine a world where every team has the toolset that is built internally by developer productivity teams at the likes of Google, FB. We have to still start somewhere right :)

Are you worried about market size of developer productivity tools in general or just about MQ? We intend to expand into other dev productivity capabilities adjacent to MQ. Our exploration into Flaky test management is also a step in that direction.

Wow this is amazing. May be you should add MergeQueue to your list :)

- Yeh we define our own pre-conditions because some developers also use it for non-protected branches. The default picks up the Github build-in preconditions and allows you overwrite. - Can you clarify your fixup commit suggestion? So there's typically an option to squash and merge when merging the PR, but sounds like you want to maintain some of those commits and squash the rest? - Just read up the AzDO's semi-linear rebase.Yeh MQ should be able to do that as well. You can specify different config for rebasing the branch and for merging the PR.

Curious since you have done this much research - have you built or used any of the merge queues in your company? Would love to chat more. ankit[at]mergequeue[dot]com

Interesting, you can potentially use MQ for automatic rebasing as well, but if there is a merge conflict, MQ will not be able to resolve it for you. If a rebase can be automatically resolved, then MQ will update the branch for you.

Wow, that's a great story, maybe we should quote that :) If you are open to sharing more of that experience, I would love to hear that. You can also directly email me at ankit[at]mergequeue[dot]com if you don't want to post publicly.

Yes it's somewhat similar to GitLab merge train. Of course we work with Github. And similar to other comments, there are a few more features including batch mode and flaky test management that make MQ a more interesting offering. We do not support Gitlab today.