AWS BugBust
aws.amazon.com
aws.amazon.com
But as best I can tell, this is the worst of all worlds: I have to already have an AWS account to both create the competition and also to point CodeGuru at the repos, then I have to pay for running CodeGuru on any potential PRs, but then also separately create a BugSnag account seemingly just for this stunt. And in the end I ... get Internet points on my BugSnag account?
I fail to see how anyone but AWS benefits from this, unless a participant happens to be one of the top competitors who accumulated enough Internet points to get a free trip to re:Invent
That said, when you've setup a build/pipeline, it mostly just works as it should.
I also prefer externalizing the CI/CD process with Jenkins or Spinnaker-based setups.
I appreciate this is a bad reason to recommend a service, and am not defending it. Just explaining why we wound up using it.
The service itself is not horrible, but like every other AWS service, I felt like I was definitely not in the target market: some easily implemented feature was left out and the configuration space easily allowed you to specify things that weren't covered in the (very) thin documentation and seemed to behave unexpectedly.
You can get much better results almost anywhere else.
Also, the bug buster benefits if they get up high on the leaderboard, because job recruiters will most likely use the leaderboard for lead gen. If you're looking to make a job move, moving up on the leaderboard will help.
I already get plenty of recruiting inbound without having to donate my time to fix bugs for Internet Points.
What you're saying is that this is a worthy use of time for people because they'll maybe move up in the leaderboards which then might lead recruiters to reach out to them.
And after all that, you have to do a phone screen, technical phone interview, a full-day onsite, plus whatever other hoops you'll be made to jump through to change jobs.
Yeah, no. Hard pass. Seems like a huge waste of developers' time.
You're not the target audience.
And also, you wouldn't be donating your time. The whole point is that a company puts up a contest for their own employees, who are presumably getting paid to participate.
You do realize most developers are asked to fix bugs anyways, regardless of any silly competition?
Their pitch to Organizations is that
1) They may be able to help you save money 2) They may make it more fun / easier to solve bugs.
Pitch to developers - they'll give you some perks for doing your job.
Note that I also think it's lame because I don't think code guru is that good and don't care about re"invent
If you want me to fix your bugs, pay me for it. Don't do this gamification nonsense.
Edit: apparently I did misunderstand how this works. It seems like the "Bugbusters" are just a company's normal developers. It still seems kinda dumb, but not as exploitative as I initially thought.
That said, other providers of this type of tooling might take some ideas from something like this.
You misread pretty much everything I wrote. My original comment wasn't an explanation for why fixing bugs in general is a "waste of time". My opinion is that uploading your code to this service that uses ML to automatically generate bugs for your team to do a bug bash on is a waste of time.
Our bug backlog has hundreds, if not thousands, of bugs already. We don't need to rely on ML to tell us what to fix.
We have real issues that customers are facing that need immediate attention. This is just a total distraction and a waste of time.
1. Stackoverflow to show my expertise
2. Any Open source contribution that shows up in my github profile
> if you’re looking to make a job move,…
Why would companies allow their best developers to be poached? Unless of course there’s a way for companies to keep their developer scores private…
This invalidates your other point that it benefits developers. Why would a dev sink countless hours for a “very low” chance at being on the global leaderboard and hence moving jobs.
You can’t simultaneously posit it’s good for both companies and developers.
Not at all. The company undervalues their employee and sees little risk in them showing up on the board, and the employee overvalues their skill, thinking that showing up on the leaderboard is a shoo-in. Maybe the employer is right, maybe the employee is right, or maybe both or neither is right.
Also, the employee does it because they get paid to hunt bugs regardless. It just helps the company by putting a structure around what they are already doing.
Come on, this is just being coy — you and I both know this is a terrible deal for employees — as you write earlier, there’s a very low chance that a random dev will appear on the global leaderboard. What’s more, the best devs won’t even play this game, so it’s left for juniors and people trying to “prove themselves” to trample over each other for peanuts.
Not to single you out, but I wish influential tech folks like you would speak up more about the cynical exploitation that’s going on here in plain sight.
Let me put it you this way - would an engineer at Netflix ever “play this game” to win ? Would employees at Netflix even put up with something like this? If not, why would you think employees at random tech co should?
I think you're missing the point. The leaderboard is an extra bonus. The main thing is squashing bugs you were already squashing. But yes, there are definitely engineers at Netflix who would play this to "win", just for the love of the game.
But let me relate another similar story -- when Amazon released the Deepracer, they added a leaderboard. All around the world, people got paid by their employers to compete. I think the first winner worked for Samsung and was paid her full time salary just to compete on Deepracer. They saw it as a point of pride that one of their employees was on the leaderboard, with enough value to pay her to work on it full time. I'm pretty sure that is what they are going for here with the global leaderboard. Everyone wins in that situation. The company gets to brag about having the best bug squasher, the employee gets to get paid for squashing bugs, and Amazon gets publicity.
> I'm pretty sure that is what they are going for here with the global leaderboard
Sorry, I’m confused. This seems like a change in your position? Earlier you argued the intent is for employees to be able to showcase their skill to recruiters… you also responded to someone else who said they already have recruiter attention with “you are not their intended audience”
> paid her full time salary just to compete on Deepracer
Wait, are you seriously comparing a global competition in cutting edge tech to squashing bugs … in your own codebase? Come on.. there’s no comparison. If anything, a company would be semi embarrassed that they have so many bugs in their pre existing codebase.
I’m asking you in good faith. Why the contortions to defend such a product? I see from your profile that you are an investor and advisor. I would seriously try pushing this product to your companies and see how they react. If they do adopt it, run an anonymous survey of the devs .. then I would love to see you argue with a straight face that it “benefits developers”
There is a benefit to both employees and employers. Being on the leaderboard benefits the employee because it raises their profile. Having an employee on the leaderboard, as an employer, also raises the company's profile. It's like how companies send people to conferences to talk. It benefits the employee by raising their profile but also benefits the company.
> I’m asking you in good faith. Why the contortions to defend such a product?
I'm not really making contortions. This product is aimed at large enterprises. It's not aimed at startups (so I would never suggest it for any company I invest in) and it's not aimed at companies that are already well known (FAANG). It's aimed at the 1000s of other large enterprises out there who want to incentivize their employees with some prizes to do a job they're hopefully already doing while also possibly raising the company profile a bit by having people show up on the leaderboard.
But it definitely solves a problem for it's target audience, which you don't seem to be a part of. I am not their target audience either. But I see the value to their target audience.
I would politely disagree — in any case the thread is there for all to judge.
If you sincerely believe that this is a product that should exist and will positively impact everyone involved, I can only hope you’re right [0]. In the end it’s all up to your conscience, but I do wish our leaders in tech were a bit more sensitive to seeing the broader picture.
[0]: pour one out for all the naive devs that will no doubt sink 100s of hours to be on the leaderboard for little benefit to them.
Only for cheapo companies. Companies should (and do) _pay_ their devs to attend re:invent. I get your point though. For naive devs, it’s like getting to go to a concert. But it’s quite cynical for AWS to exploit this
There is literally another story on the front page [1] about such "leaderboards" and what happens when people game them to move up.
I would not advise anyone to participate in such a contest, it sounds more like a race to the bottom while providing valuable training data to AWS for free.
-------------------------------------------------
If those findings are imported into a BugBust Event [which is defined by the tuple of (start date, end date, scoring, and repos in scope)], then one can (invite|publish) a link to the BugBust Event and at the end date the participant who fixed the in-scope issues with the highest point values claims the cited prizes. Further (again, AIUI), at the end of the *AWS* event, the BugBust account who has the most *overall* points wins a re:Invent trip
I don't believe AWS gets into the business of monitoring the PRs, only observing the CodeWhatever outcomes -- if CodeWhatever found an issue, time elapses, and the CodeWhatever sees the issue is fixed, then BugBust will look up the category of said issue, multiply it by the assigned score in any Events it is associated to, and award the Internet points to any Event participant it knows about
Looking at https://aws.amazon.com/bugbust/ the prizes are yet more corporate trash that don't need to exist ("AWS BugBust Varsity Jacket" okay ), and they have a pricing page so it's not even free.
All this tells me is that Amazon doesn't value your time
What I can't wrap my head around is that the top of the FAQ says that 1 Point has an estimated Dollar Equivalent of $769, and for 100 Points you get a t-shirt. As a "prize" in exchange for saving $76,900 I'd be expecting a car at the low-end...
Even if Amazon took a $269 cut from each point, I'd expect to get paid $50k for doing 100 points. But instead you yet a $12 t-shirt.
> For information on how points are awarded, see the Official Rules. Actual savings may vary.
Section 5 of the official rules document states approximate retail value of the prizes as; reInvent:$4000, Resin Trophy:$60, Hooded Sweatshirt:$27, tshirt:$12
https://d1.awsstatic.com/product-marketing/bugbust/AWS_BugBu...
$77,000 for a t-shirt. $1,500,000 for a jacket. If you're saving somebody that much money who cares whether Amazon gives you a t-shirt? So either have prizes comensurate with the alleged savings involved or don't insult people with such low value prizes.
I for sure did not get that impression; where did you see they're offering free analysis?
> When you create your first AWS BugBust event, all costs incurred by the underlying usage of Amazon CodeGuru Reviewer and Amazon CodeGuru Profiler are free of charge for 30 days per AWS account.
BugBust itself is free. You pay for usage as per normal for CodeGuru Reviewer and CodeGuru Profiler (after the 30 day free trial of BugBust and 90 day free trial of Profiler).
>(after the 30 day free trial of BugBust and 90 day free trial of Profiler).
So it's not free, dev time is expensive and has a lot more value than their proprietary service usage time—so they're coming out on top whilst charging people for it.
The feedback was indeed similar to what standard linters and static analyzers can provide with more noise. I have no doubt they can improve their recommendations as they get more feedback and analyze more codebases, but we were not interested to pay to contribute.
We have had much more success with https://deepsource.io/ , which found real bugs and we generally agree with their recommendations. We are currently evaluating whether or not we will include this service as part of our process.
Deepsource is rule-based, but my understanding is that they are starting to use ML to rank recommendations.
I fail to understand the value of this. Seems to me like they want you to use and pay for CodeGuru, which will tell you shit about your codebase. Then you can import those issues into some sort of campaign inside this BugBust thing so you can create competitions and as you do it you compete with other teams around the world from other companies(?)
If that's what it is, then this sounds like an idea created in a vacuum, with no customer input at all. I fail to see any value on this.
You wanna solve or avoid millions bugs? Instead of hiring people capable of memorizing how to invert binary trees and similar bs, companies when interviewing and hiring should focus on real hands-on experience, should value developers who empower and demostrate best practices and good code craftsmanship.
Ask candidates to write and review real production code.
It is just unbelievable how low is set the bar when it comes to writing high quality code even at so called FAANG. You can just imagine how bad it gets for any other random company out there.
Or how surprisingly solid developers can be at non FAANG places. Not everyone aspires to work at Amazon.
Presumably you pay candidates for this, right, if we're talking about "real production code" and valuing developers?
Basically testing real day-to-day job. If you were to work in this company and you were given this task, how would you approach it?
You can easily tell which people care, strive for good practices and who have high quality standards vs the ones who tend to just hack things together.
It worked well for me, and never regretted any hiring.
So as opposed to a boilerplate test or asking developers to solve a challenge with a few lines of code and discuss their thoughts on it, they're asked to look at real production code that is in use, opine on it, and add/change functionality to it?
Can I re-ask my question if I've read you properly and understood: presumably you pay people for this right?
I ask because in my mind, solving a coding challenge or talking about something hypothetical/conceptual probably isn't too much different than solving a problem on actual code the company uses from a technical standpoint in as much as they both require effort on my part to think about the problem and bang out a solution...
... but in terms of time spent interviewing and evaluating employers, I'd feel less comfortable contributing anything to production code without the guarantee that the company wont just take the work and implement it, and then later say they hired someone else/went another direction or just decided not to hire at all.
Which has happened.
I mean either that, or you are back at the : implement a linked list, invert binary tree, check palindrome string and similar bs.
> customers can set up private AWS BugBust virtual events
Why should the jacket be sponsored by Amazon? I don’t get it.
A team can easily just dedicate an hour or two a week and do their own bug bash, right?
What's the incentive for people to actually do this?
Doesn't even sound free.
Amazon has been openly adversarial to businesses, including software companies, so what goodwill are they banking on?
I looked into Snyk Code, but it's mad expensive.
They also have their paid service SonarQube which is really good. https://www.sonarqube.org/
I can see this as a future repository challenging stack overflow, and with automation, challenging other CI pipelines, and then as someone else mentioned these magic carrots could be used as hackerrank/linkedin/hiring.
Please god no. We have enough terrible hiring practices as it is, we don't need to make every other tech company a sweatshop like Amazon is.