Rookie coding mistake prior to Gab hack came from site’s CTO
arstechnica.com
arstechnica.com
They're up against the world, and they're spreading themselves very thin, aiming to replace all of the "big tech" use cases, for better or for worse. They hate Silicon Valley but they "move fast and break things" more than anybody, arguably by necessity. I'll cut them some slack if their public platforms have some bugs, but they shouldn't do anything with sensitive user data.
In this case, it's more of a "everyone gets a CxO title because there's fewer than 26 of us" type thing.
I think we see a lot of tech company failures when the CTO is just a manager type that doesn't understand the technology and doesn't have the ability to contribute to it.
If you require that person to also be the most technical person in the room at all times, whew. Either that's a very tall order or recruiting/hiring needs work. More technical than that person who's spent a career getting absolutely ace in their niche? More technical than the data scientist who spends 8 hours a day digging really deep into bleeding edge neural networks?
I'm totally with you that a manager should be able to understand the thing they manage to a certain (not too shallow) depth, but you hire experts for a reason, and in ordinary circumstances, and in an ordinary IT department, a C-level manager should not strive to out-expert the experts under their purview. That kind of micromanagement/smartassery will just drive away the experts, and then you need a new CTO and new experts and maybe go bust.
Exception to the rule: If you're in a startup where the IT departmend consists of a CTO and one guy remoting in from Bulgaria who runs the servers and is really really cheap, you as the CTO actually should be the most technical person in the room. But once you hire the first actual expert, you should take a step back and get more big-picture-y.
Basically I set the team values, priorities, and processes, but I do need a lot of my team to have deeper knowledge of certain things than I do.
For a decent size company I'd hope the same, but with the proviso that over time any specific industry experience will degrade to a more general understanding, which is fine.
And if they don't have the experience they delegate.
So bearing in mind the original story, either the CTO should be at least decently technical enough to avoid composing SQL like that commit does, or the CTO should not be committing code in the first place.
Your generosity naively assumes that given enough time they would resolve your concerns. That is never the case with these anti-social social media companies.
It's taboo to talk about, but the correlation between political conservatism and overwhelming amounts of technical debt leading to non-existent operational security is hard to overlook.
While I realize that all major platforms have had their dark days, those days usually pale in comparison to a dark day at a conservative-based platform.
[1] https://www.theverge.com/2020/12/22/22195115/voat-free-speec...
[2] https://www.washingtonpost.com/news/the-intersect/wp/2014/09...
[3] https://greatawakening.win/p/11SJjocsHm/-one-of-the-moderato...
[4] https://arstechnica.com/gadgets/2021/03/rookie-coding-mistak...
[5] https://www.wired.com/story/parler-hack-data-public-posts-im...
[6] https://www.theverge.com/2017/8/24/16200002/daily-stormer-dr...
I think you’ll find extremists of all stripes perform poorly, mostly due to the much smaller talent pools involved. Can you name a handful of highly successful extreme left tech companies?
Moreover, assigning any random highly successful tech company as ‘leftist’ seems silly. What’s less leftist than a vehicle for minting billionaires and further enriching existing ones?
Honest question, perhaps it's that they're just more socially aware? That they respond better to societal pressures and feedback loops.
Technical ability has helped many startups leapfrog others in a similar space and time. It's the lore which HN is based upon. In a scenario where lots of actors are trying to censor your site, having great tech chops can help with resilience. Technical ability unlocks new revenue streams and optimizes existing ones.
Viability starts to approach zero if the business faces security issues. Its as if the hull of a ship is breached. The safety of everybody on board now is at risk. People literally get murdered when the wrong DMs or photos leak. Exposing your users digital life to grifters and governments a huge risk to them and your business as a result, let alone the hard cost of managing a data breach. Companies recognize these risks and put checks and balances in to ensure that these things get caught. Some even have a fiduciary responsibility to do so. That said, when a CTO decides to push code, they generally have the ability to override any of those safety systems.
I think you're spot-on that extremist companies can't grow very large and struggle for talent.
It's entirely possible to generate lots of wealth while advocating for wealth to be distributed more broadly. One can argue and support reforming how Trusts are taxed, and may make a bigger dent in the problem than they could by holding up a sign on a street corner. I don't see entrepreneurial success as antithetical to progressive causes. Using the position of power that your success affords to change the rules of society to benefit you and yours... that there is conservative, commonplace, and where leftist opposition should be laser-focused.
There seems to be an assumption that the big tech firms are inherently better at picking the better 'engineers'. Having experienced a lot of what they produce (not specifically Facebook, but Apple, Microsoft, Google, etc) I'm inclined to assume the opposite personally.
More like "move fast and break yourself."
It's been a while since I wrote any Rails but the offenses that jump out just from a cursory inspection:
- large raw SQL query which could almost certainly be accomplished in a more idiomatic way with AREL or ActiveRecord
- no user input sanitizing
- using a regular string literal for a large text block instead of a Ruby here document
- leaving a mess of commented-out code at the bottom of the method
Apparently Gab isn't exactly hiring the best and brightest.
Remember: Ruby is a dynamically typed language.
Not that one should rely entirely on them for these sorts of situations.
It's definitely bad code, but if you're going to write an article which so heavily implies "this idiot's bad commit is why a breach occurred", then whether the code is actually exploitable seems worth investigating first imo.
We can't tell from just that commit whether that code is definitely exploitable, but they'd have to be doing non-default things to make it not exploitable.
The general fix is bound parameters, not typed arguments.
I felt much safer doing that knowing the parameters I'm concatenating are integers.
"Former Facebook engineer" means Facebook paid him... No more and no less.
You'd be surprised how often I've found leaked API tokens, passwords in plaintext etc on github for instance.
Universities grant degrees based on the successful completion of course requirements. I don't know if it's fair to conflate that with what you said. As long as you can get C or better in every course required, you have a degree. Does that necessarily mean that you are competent?
It's an even further stretch if you think that at University cares all that much if one of their Computer Science graduates makes a mistake.
It means that the university granted you a degree (and, indeed, the med student who got a D average in every class is still called "Doctor" when they finish the process). The point is: universities have a vested interest in the question "Does that degree indicate you have competence?" and incentives to change if the answer, on average, trends toward "no" for their graduates. Corporations have no particular vested interest in the question "Does the corporation's name on an ex-employee's résumé mean that employee has competence?" That's some competitor's problem, not theirs.
> Universities don't grant degrees until someone has proved they have whatever competency the degree indicates
That is not a rule. I have interviewed countless folks with CS and CE degrees that can't get past basic scalability questions in abstract systems design. I've started plenty of hour long interviews only to learn five minutes in that the candidate has a four year degree and cannot correctly identify a use case for binary search nor implement it in plain code.
> Universities have a reputation to lose if they grant a degree to someone who goes on to mess up badly
They absolutely do not lose reputation. We don't publish on a website where the degrees came from that allowed Cambridge Analytica to exploit the American people. Everyone will easily write off the offending programmers lineage as a fluke and move on with their lives. Nobody really cares where you came from in this world, just what you're up to now.
> Mr. Marotto making a major error when he's no longer employed by Facebook doesn't reflect poorly on Facebook.
I agree that this would be a pretty trash take and a poor precedent to establish long term.
> "Former Facebook engineer" means Facebook paid him... No more and no less.
Simultaneously, this is not true. Working for Facebook carries some prestige, as we can see from all the folks who proudly sport "x-Facebook", "x-Google", "x-Netflix" show us. Obviously some people make use of this market and obviously someone is foolish enough to believe the sheen. The value in that perception rests with the viewer; if they hired him because of his work at Facebook then it was significant, if they would've hired him regardless then it matters very little.
haha
you'd be really shocked if you knew about the great lengths people go to get their degree and learning stuff ain't one of them :-)
I love decentralized software, but don't let Gab fool you. They're just out to grab power right now.
When a large product in Production is having performance issues, it's very possible that you'll write something like that in a time crunch to get the site back up. The code might not be the cleanest possible because you aren't sure how well it will work yet. It might be checked into Git because any remotely sane deployment procedure requires it. And maybe somebody years later will pore through the Git logs and come back and drop snark on you without knowing anything about what was involved in writing and releasing that code and they've never had to deal with something like that.
No, don't sanitize user input. Don't trust it, which is a big difference. Use bound parameters and this problem goes away.
Anything else is madness. Of course, if you're dealing with legacy systems you might have to do something tragic like not allow people to put quotes in passwords.
From the article:
>The change, which in the parlance of software development is known as a “git commit,”
I would rather, as an engineer, discuss core issues we can fundamentally address: compromising on inadequate workflows (including core architecture and paradigms), commitment to over-delivery, and the ever-dooming deadlines. What victories of the CTO went ignored, as part of "the job"?
It's nice to be smart when you have regular 8 hours of sleep. I've had enough stress to remember just how idiotic many of my decisions were, as a "leader". Most of them went ignored just because we were covered by being invisible, by design. Morally, I can't judge this CTO. If you look at your coding history, can you?
The core leadership behind inadequate decisions, often above the CTO, are frequently of the "don't care about the math, just the numbers" type.
We may still have a fragile and unique culture in software, that perhaps contradicts past history such as engineering in construction (look up "corruption construction") or the unique corruption of medicine ("sugar lobby", "food pyramid").
Despite bad decisions and the fumbled cover-up, the attempt to perform in public on their part is valuable to me. We don't have easy access to which of the doctors took money to publish "research" that "calories are the same", pushing for more carbohydrates in diets. This may translate to multiple people, people you might personally know, dying of diabetes.
With open software, we get the names. This should not reward click-bait media witch-hunting.
> "A quick review of Gab’s open source code shows that the critical vulnerability—or at least one very much like it—was introduced by the company’s chief technology officer."
"Independent experts indicate (fee undisclosed), a powerful malevolent actor was involved in the recent malicious attack on our infrastructure. This aligns with the recent series of threats identified by the State Department and other US government agencies as enemy state activity to undermine Democracy! They hate our Freedom!"
However, it looks like the CTO pushed this directly, no PR.
Surely the difficulty in recruiting people to work for their shitty website with shitty politics should illuminate for you and everyone also in denial that politics is also engineering.
This is not correct. The mistake was to use ‚find_by_sql‘ without parametrizing the query. The mentioned reject and filter methods are merely skipping some of the data the query returns.
But you're of course correct that it's not the replacement of an ORM call with SQL that's the problem.
This is completely and utterly untrue.
https://guides.rubyonrails.org/security.html#sql-injection shows examples that are exactly like the code in question in that commit
Bound parameters were a new thing like 15 years ago.
There is no excuse for writing an SQL injection in 2021. Zero. None. And if you're in the position to write code that'll be deployed to production, you darn well better have it reviewed by peers before it's merged.
If the CTO did this and worked around the developers, he's a freaking idiot. If the engineers saw this and signed off because he's the CTO, they're freaking idiots. I wouldn't ordinarily be this harsh about it, but come on. SQL injections in 2021? That's astoundingly bad.
That's when Rails came out :)
I'm fairly sure ActiveRecord has always supported them.
Well, Rails was < 2 years old then, so everything was new in Rails.
When they were new depends on what language and database library you use.
Perl's DBI had them 25 years ago.
Yet I shall prepare this query.
Though, the inflection point, at least in my circles, was in 1998, when rain.forest.puppy talked about "piggy backing" SQL commands.
Even there they mentioned using parameters with stored procedures to protect yourself. It's just clear no one in IIS land did at the time, so there was a wide open exploit playground for everyone that saw that release of Phrack, once they got done playing with directory traversal exploits.
> Specifically, line 23 strips the code of “reject” and “filter,” which are API functions that implement a programming idiom that protects against SQL injection attacks.
That's not true at all. If you are going to do technical analysis that calls out mistakes, the publication should have multiple technical people review the article.
I'm guessing he was not familiar with the codebase and maybe not even ruby/rails in general and just wrote the whole sql query by hand, not bothering to figure out the correct way of parametrization , using string interpolation instead.
What a joke, and then blaming the documentation...just wow.
Lmao, that's not how AGPL works. Although, it would be pretty funny if a license explicitly required you to post all your Ls for public ridicule.
The GPL family, including the AGPL, defines source code that must be provided as “the preferred form for making changes”; if the form the distributor uses is a VCS repository with full history, publishing either static snapshots or a repository with an expurgated history could arguably be a violation of both the letter and the spirit of the licenses. The language of the licenses makes the requirement dependent on actual development practices, not fixed, and that's not an accident.
> The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work.
I don't think that's referring to the method by which I make changes or manage those changes. It's simply trying to make it clear that source-code is something that should be in a form you can actually change, and not something obfuscated. e.g. minified, transpiled, object-code, etc. The second sentence gives the first more context.
The spirit of all GNU licenses is that the end-user should have the freedom and ability to understand or modify software that's *distributed* to them for themselves. It has nothing to do with having a full history of the source-code available. What does full history even mean? I'd like every auto-save to every LGPL licensed file, please.
There are numerous examples of start ups and even mature companies making basic mistakes. This is easily explainable without resorting to conspiracy theories.
"Free Speech platform Gab has announced Fosco Marotto as the company’s new Chief Technical Officer (CTO). According to a blog post from the company, Marotto was formerly a software engineer, production engineer, and developer advocate during a seven-year career at Facebook.
Marotto reportedly brings 23 years of industry experience to the platform along with extensive knowledge in backend infrastructure and insight that will help Gab scale as it becomes increasingly popular."
CTO: https://github.com/gfosco
Parse Server: https://github.com/parse-community/parse-server
Parse Server API doc snippet:
"If your app is compromised, it’s not only you as the developer who suffers, but potentially the users of your app as well. Continue reading for our suggestions for sensible defaults and precautions to take before releasing your app into the wild."
https://docs.parseplatform.org/rest/guide/#security
Per Ars, he removed security code.
Your OP wanted to paint a picture of inexperienced development team. Both items in my post show that both the team (who initially had it right) and the CTO (who inexplicably removed the two lines) are definitely experienced.
That Parse Server looks like a substantial piece of code and there is a community around it, so let's be fair and grant the CTO a measure of competence, shall we?
I said absolutely nothing about a honey pot. Review my posts. Occam's razor strongly suggests to not make un-necessary assumptions. My post challenges your assertion of incompetence. You need to remove that assumption from your analysis.
> What's your theory for how this happened, if neither incompetence nor deliberate?
Honestly no theories, just rubber necking here :) but since you ask, here is what my shaving session with Occam brings up:
- cognitive impairment (alcohol, various fun chemicals)
- emotional agitation (relationship, personal matter, etc)
- rushed for time (silly mistake)
- arrogance ("I can do C++, Scala, JS, and PhP. This Ruby stuff will be cake, and I do not need to ask my team why they had those 'funny' filters on top of the function")
- malicious coding ("Sure, I'll help you scale it. I hate SV too. I hate it so much I spent 7 years at FB.")
So re. that last possibility, I also do not agree with your GP that this indicates that that platform was a honeypot. This specific event can be construed as placing a backdoor in their code.
Even with all that, I would still expect him to not make any commit similar to this CTO.
*A strong engineering team should've had vulnerability scans at some point in their build process. That SQL injection vulnerability would've been easy to spot
*That SQL injection probably should've been picked up in a code review
*They should have been using parameterized queries in the first place. The fact he removed input sanitization methods is besides the point, that they shouldn't have been relying on those in the first place.
*A senior engineer or CTO should've known better. But I've seen very senior people make bad mistakes before. These mistakes are much more likely with immature processes and safeguards.
*Sometimes someone can get to a 'senior' level without necessarily knowing how to do some aspects of software engineering well.
A strong engineering team is about the performance and practices of the whole group. You can have individuals who are experienced at particular skills but have big gaps in the skills and experience needed to build a strong team.
These are all solid points modulo the optics of memory holing commits.
https://git.rip/gab/gab-social/-/blob/develop/LICENSE
This will make things interesting for them going forward.
It takes advantage of a somewhat-not-well-known feature of Javascript template literals in that you can apply functions to them, e.g. sql`SELECT foo FROM bar WHERE id = ${userEnteredValue}`. No need to manually escape userEnteredValue, the sql template literal function does it for you.
You get the same benefit of "template literals" but the data and query are separated at the DB level. The DB knows what is data and what is query therefore any attempt at escaping from SQL will be quashed there.
[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
We shouldn't call out and shame individuals for making a mistake like this. Any company which even does something like this internally is toxic; the blame lies on the company and the process, not the individual. The flimsy excuse "oh, theys a bawd bawd company filled with icky people" doesn't hold water; we're talking about basic decency here.
You want to write a story about the commit? Hold the company responsible? Encourage processes and procedures which would have caught this? That's fine. Blur out the names. Ars isn't a tabloid, but they're acting like one; there are real, serious consequences for shit like this. You didn't set out to create the perfect ammunition for someone within a company arguing against the open sourcing of some project, but here it is.
I'm physically disgusted by this reporting.
I think National Inquirer and The Sun uses the same tact.
Totally agree here: blame the company and the leadership for not investing in the right processes and tools. People will always make mistakes.
He failed at doing his job as a CTO and then failed at being a moderately competent entry level software engineer.
That … doesn't make any sense.
The people at the top might not be at fault, but they sure as hell are responsible.
No, I don't think people at the top get paid to be a voodoo doll of responsibility. They get paid a lot because good executives are hard to find and can produce huge benefits, so the market values them very highly. It can feel good to throw around moral judgement like "that still does not absolve them individually" but if a set of incentives/environments consistently produce bad outcomes, the people involved are not responsible. It would be unproductive to punish the people involved when their replacement would do the same (especially considering that punishment is notoriously less effective at deterring human behavior). I personally think it is also morally wrong to do so, similar to punishing a thief for stealing food in a system that consistently deprives him of the ability to acquire food legitimately.
People's lives were destroyed by that breach. It wasn't a victimless mistake. The public wanted to know how it happened, and hiring the wrong person is part of how it happened.
It's just something that isn't and shouldn't be a concern of most engineers. I don't care about these things in terms of solving them.
You entrust the collective wisdom in each area like security for these types of things.
Which is exactly why it's hard to believe a former Facebook, veteran CTO would commit some code like this.
No legitimate electrical engineer would do the equivalent.
So you get to be the CTO but actually being an awful enough person to put up with the company.
Sure, hold the company responsible, but don't just let the person get away with it so they can run off and do the same at another company. Crooks and arseholes go from company to company causing damage everywhere they go because firing companies won't tell anyone while they let a person go. This seems to be a result of just that culture of silence, because how else can you explain someone becoming CTO of an org like this without know basic stuff about their own area of expertise?
When minions make mistake, let the company take the heat. But when the person screwing up is in leadership and therefore directly responsible, it ought to fall on them.
One important note: Gab is currently running a fork of the Mastodon software, licensed under the AGPL-3.0. It's not their right to decide whether they want to publish their source code.
The key is that humans make errors, errors of commission, errors of omission, errors of misplaced/almost brilliance, and errors of outright apparent stupidity. IMO, you should dive deeply to understand exactly what happened (which includes who did what and why), but you don't use that as ammunition against the individual humans.
Pretending that nameless, faceless committers did this thing isn't as useful as "even our CTO can make a stupid error [and not get fired for it]; here's how we seek to eliminate this entire category of error reaching production: <1> <2> <3>"
For a random line worker, sure. For the CTO, the position is absolutely relevant. You can't have a conversation about what processes or procedures might have caught this without taking that into account; in every small company I've worked for the CTO was either tacitly or explicitly able to bypass any amount of code review etc..
The fact that it's the CTO is informative, educational, and important, and should be reported. Their personal details are irrelevant but hiding them would also be meaningless.
EDIT: that link isn't working, but here is a screenshot: https://imgur.com/4lyElZI
I have seen more shitty code than high quality code in my life. It’s hard for me to empathize with anyone who thinks startup code is going to generally meet some decent quality bar.
People like Brian acton and Jan Korum exist, but they are the exception. If you always expect people to write good code, you will be fucking disappointed.
The only solution that scales is having adequate speed bumps and guard rails that prevent people from pushing code till after it’s been properly tested. And just because it’s in a git log doesn’t mean it gets pushed to production.
This article reminds me that journalists only make money when they squeeze your amygdala. Reading content from ars is the equivalent of an intellectual jump scare. Add 127.0.0.1 arstechnica.com to your resolv.conf file for mental health. Fuck off ars
https://www.mathyvanhoef.com/2012/05/whatsapp-considered-ins...
https://www.mathyvanhoef.com/2012/07/whatsapp-follow-up-unau...
What’s important here is that the CTO is ultimately responsible for the failure either way. That’s the point of the article and it’s why the article is valid (even if you don’t like it for some reason): the engineering buck stops at the CTO, especially so if they personally create a bug that exfiltrates all of the company user data, regardless of how they managed to screw it up.
I don't know what experience you have, but senior engineers constantly make security mistakes. Linux kernel developers are some of the smartest programmers in existence yet they continually introduce and fail to patch security vulnerabilities.
Accidents and SQL injections happen, usually because of non-obvious query-building but this is a different level.
[1] https://threatpost.com/whatsapp-bug-malicious-code-injection...
I'm not familiar with that vuln, but I don't see how it could be the "same mistake". My guess is there simply wasn't support for parameterization or there was a non-obvious concatenation problem when building the query string—but please correct me if I'm wrong.
This instance is novel because the parameterization protection was removed in favor of concatenation and that the vuln is so obvious a first-year CS student wouldn't struggle to identify it.
Seriously, if this is a mistake a senior engineer could do at your company, why would you work there? I'm not one for gatekeeping but this is learned within a couple of code reviews in your first years.
Unless you're claiming that those SQL injection exploits were introduced AND review approved by junior engineers, a practical impossibility according to policy.
Your claim that senior engineers don't make whopping security blunders is absolutely false according to all available evidence. The smartest programmers working on kernels make them constantly.
Raw untrusted string substitution into a SQL query is just unacceptable. It's on the level of storing unhashed passwords, or 1000-line god functions: the kind of mistake you'd expect from a self-taught developer writing PHP in 2002. Literally every popular SQL client and ORM provides quoted substitution; please use it. And, dear reader, if this was news to you, and you write SQL, please go get educated, because it's not the last nor the sneakiest pitfall in this space!
* Senior enough to be writing raw SQL code at Facebook
WhatsApp had a raw JavaScript RCE literally last year.
Also, blaming colleagues for making a mistake is extremely toxic behavior. I've fixed dozens of >9.0 CVSS vulnerabilities at multiple billion-dollar organizations. I'd say senior engineers were involved in their introduction at least half of the time.
I'm with you in the sense that it always is the process, rarely the dev. It's complicated that this mistake was made by the CTO, who is responsible for being sure such processes are implemented. The lesson for me is that we all must be continually vigilant against both shitty code and evil, both without and within. Anyone can write terrible code, irrespective of political stance. Likewise anyone at all can do evil.
Companies are very porous, people take shortcuts, temporary workarounds get forgotten and left in place, etc.
You can tell who actually has real experience by who is saying this and getting downvoted for it, ironically. All of the downvoted comments in this thread are correct.
I can imagine myself doing this, having not slept in the last 20 hours, after 7 martinis and one especially long island. Maybe he did that too, but he should have the common sense as CTO of the company not to!
That’s like going 120mph and telling the cop you were just “distracted”.
You’re correct that we often make mistakes when we’re busy, but this isn’t that. This is OWASP #1 and the most plain example of it.
This is not a “mistake” that engineers make, this is unethical to allow this to go into production. He even has a stackoverflow answer where he explains this exact problem: https://stackoverflow.com/questions/11554015/syntax-sql-wher...
If you think I’m wrong, then what could an engineer be responsible for? Literally nothing ever?
If the CTO pushed this code, and the processes the CTO themself is responsible for designing did not catch this, it is safe to say the CTO is incompetent.
Dumb mistakes happen, true. This one should never happen to security experts, ever, unless their expertise is fake/bluff.
Stripping out safe parameterised code and replacing it with injected user input is only something that "happen(s) to everyone" at a very early stage in their career.
- For an intern or a junior dev, it's just a learning experience and no names.
- For a dev with a few years experience, it's a cause for concern but still no names.
- For a senior dev, I'd wonder if they lied on their CV but still no names.
- At CTO level this is not acceptable. Maybe no names, I'm on the fence, but they are a semi-public figure by nature of their role and their name was on the commit so they are the ones that made it public.
Either way, the CTO should have lost all credibility with their colleagues and be out the door. They represent the norms, processes, and safeguards the company employs and should not be in that role.
PS. I understand that there is no proof this commit caused/allowed the data exfiltration. It doesn't matter; the commit is heinous enough on it's own merits.
I could certainly understand someone working for Gab for a paycheck, and then feeling guilty and "making a mistake".
To be clear, I'm not saying that one should feel that way, just acknowledging the possibility.
Imagine trying to explain this to a future employer.
I can guarantee that any system that you’ve worked on has numerous OWASP security bugs. You’ve probably looked at the bugs countless times and never noticed it.
Every software engineer of all levels has overlooked obvious sql injection bugs in their code base. Most likely you’ve added to the bug list.
Software bugs are simple part of any development effort. All major companies, Microsoft, Google, Facebook, etc. has very simple bugs like this in their systems.
That’s why they pay out bug bounties, it’s cheaper for them to add the bug and have some random security researcher find the bugs for them.
On another note, how do we know that the new CTO of Gab didn’t just do this on purpose so that there would be a breach?
That applies to anyone any employee, including the CTO, especially so. They're just people.
Publishing an entire article to shame them is just disgusting. Like, why? The reporter if anyone deserves being shamed for apparently making a career of flinging shit at people.
What about when the individual is responsible for establishing the processes? And when you blame "the company," are you not ultimately referring to those in charge, i.e., the C-level executives? After all, don't these same top executives claim a large share of the reward by arguing that their positions carry more responsibility and risk? So it seems entirely fair that they should expect to be held personally responsible when there are failures under their watch.
It can make you a developer, aka code monkey, but proper engineering requires more comprehensive training, that focuses a lot not only on basic coding skills, but teaches you about design, managing lifetime of products, security/safety/ethical considerations, etc.
A lot of people who call themselves engineers in IT fields are much closer to code monkeys than engineers.
[1] https://stackoverflow.com/questions/11554015/syntax-sql-wher...
Reddit investors: "Reddit raised $50 million in a funding round led by Sam Altman and including investors Marc Andreessen, Peter Thiel, Ron Conway, Snoop Dogg, and Jared Leto."[1]
- Sam Altman is an open democrat and has donated 250k USD in support of the democratic party.[2]
- Peter Thiel is pro-LGBT and pro-Trump.[3]
- Snoop Dogg is known for liking (and singing about) weed.
I don't know about you, but this explains Reddit's front-page and their overall content policy quite well. There is not as much information about ArsTechnica as there is about Reddit, but if we do some research I think the realities of ArsTechnica's potential biases will reveal themselves as well.
[1] https://en.wikipedia.org/wiki/Reddit
[2] https://en.wikipedia.org/wiki/Sam_Altman
[3] https://www.nbcnews.com/feature/nbc-out/openly-gay-libertari...
The first response of the GAB exec team to a bug which has exposed all the private information to being leaked was an expletive and slur-laden attack.
I am quite comfortable with the CTO being treated the same way he is happy to treat others.
Technically, the CTO is an officer of the company. In a very real sense, he is (a piece of) the company. Furthermore, how would one "hold the company responsible?" -- review their security culture? Where do you think that culture comes from?
Yes we should. This individual is the company. He's the CTO. "Chief" Technology Officer. The buck has to stop somewhere. How can you possibly suggest with a straight face that he should not be held professionally responsible for the code that:
A) He personally wrote B) He is responsible for professionally as the CTO
wow is that what I sound like
> Besides the commit raising questions about Gab’s process for developing secure code, the social media site is also facing criticism for removing the commits from its website. Critics say the move violates terms of the Affero General Public License, which governs Gab’s reuse of Mastodon, an open source software package for hosting social networking platforms.
Gab is likely breaking license agreements by performing this sort of cover up. So yes, this is newsworthy.
That's the problem with reporting like this; it jumps to conclusions to justify outrage, while clearly being written by someone who is overworked, underpaid, underexperienced, and only has a cursory understanding of the systems at play.
Of course, most people are underexperienced in domains like this, so they rely on traditionally high-quality reporting sites like Ars to provide accurate, precise, and fair reporting. The wording Ars used is accurate; your interpretation of their wording isn't. That's the responsibility journalists have; what they say matters, a lot, because people interpret what they say in different ways.
Ars said that they're "being accused" of breaking license agreements. You said they're "likely" breaking license agreements. That's a world of difference; if for no other reason than because one is correct, but useless, and the other is (probably) wrong.
If a civil engineer makes an egregious mistake and a bridge collapses, it's not toxic to say an egregious mistake was made.
And, in an organization, if you are unable to recognize and call out mistakes (in a respectful, constructive manner that is), you're a failed organization.
Rookie mistake is certainly not a constructive way to report on this problem, but it is true.
You used it here a couple times in here so far.
I've been on the wrong end of a SQL injection once and I was called out publicly multiple times for it to my endless shame from my customers who bought the software and were hacked.
Was that toxic for them to call me out?
Heck no. In the 15 years since that time, I made triple-sure to never code something that could lead to a SQL injection again.
There's this tweet floating on the web which this made me think of. Something to think about I guess.
> If you have suffered in life and want other people to suffer as you did because you "turned out fine", you did not in fact turn out fine.
I can talk about the commit, and I can talk about the article.
We were getting crushed with scaling from 3M to 30M visits a month, and every day was a 16 hour day. The site was falling over on a daily basis, and the performance of the home feed was atrocious. The APM traces were full of loops. The muting and blocking features were incurring tons of additional queries. As I remember it, one night I decided to write a SQL query to replace it. I believe I thought the values were already being sanitized, but I have no problem agreeing that I should've looked into that further and made sure. I'm not a rails dev, and I'm generally negative towards rails and ActiveRecord. I wrote SQL for many years in the past, and as the folks who went through my stackoverflow saw, I am not unaware of the importance of sanitizing user input. I've written much code that properly sanitized user input, in various languages.
It's really easy to imagine that Gab is a large company, considering the breadth of their platform and products, but you'd be shocked and impressed. It's still really early days, like this is only Chapter 2 for us.. we're growing and will keep getting better.
This article is interesting to me for other reasons though: 1. there's been no confirmation this commit was related to anything that has happened. and 2. the person who reached out to Ars with the story, a link to the commit and a quote, is a former co-worker with a personal grudge. I can only imagine the glee with which he wrote that email.
For the last 2 days we were being extorted for nearly half a million dollars. I received a death threat today, first one ever. I think we're doing something important. Go ahead with your code critiques.
Have a great day,
Fosco
It's interesting that the technical language in the article is so clearly mangled. Either Ars Technica has no editors that know how to edit (unlikely), or that know the basics of git (again, it seems unlikely). It reads like a journalist taking notes from a dev ("uh-huh, a 'git', right. Oh, no a 'git commit'. really?").
That explains a lot, and shame on ArsTechnica for enabling that person's petty revenge.
Anyone, including a CTO, can make mistakes and introduce accidental vulnerabilities. It's not productive nor healthy as a culture to blame someone for being imperfect.
The best thing to do, as a company, is to learn from such a mistake, and improve processes to catch and fix it earlier, before it gets released.
It's disappointing to see Ars participate in a low-ball personal attack, calling out a "rookie coding mistake". They're encouraging a toxic mindset.
Meanwhile, Ars still has a few decent articles/departments, but it seems like 50% of their content is tabloid-grade cheap and mindless bashing of anyone who disagrees with the elite mainstream opinion. Probably like 80% of the mainstream media right now is even worse.
I wonder how Ars would fare if somebody was to tally up all of the boneheadedly stupid mistakes they've made in journalism, grammar, spelling, site administration, etc, and publish articles trashing the specific employee who did it each time.
And don’t give too much credit to Dmitry - he didn’t think of you differently than any other fascist white supremacist. You earned this coverage through the sheer force of your own incompetence.
Best of luck to you all heading into Chapter 3! Can you give us any exclusive hints as to what’s coming? I think it’s even odds between a wave of mass shootings and being overwhelmed by child porn.
You’ve changed your tune since your bootlicker “we should all be Team Facebook” comments on every Zucc post were a mainstay on Workplace.
You predicted Trump would win; he lost. You said Qanon was harmless; they stormed the capital. You thought you could do this job (and you started while you were still pocketing $ from Zaddy); you ate it just a few months in with a cock-up so basic that it would be laughable if it hadn’t seriously hurt your users, company, and reputation forever. Honestly, you should get therapy.
ugh.
Hopefully this eliminates any doubt on Gab's real mission. It isn't free speech.
Naturally, most of these majority belief systems and viewpoints aren't functionally being suppressed, because many employees at these companies hold the same views. It's usually just the extremist stuff, and some of the people who bail from Twitter to head to Gab or Parler end up causing trouble there too by being so extreme they run afoul of those sites' incredibly lax and generous policies.
I could understand this complaint if the government was applying force or other punishments (financial penalties, etc) to suppress a majority viewpoint, but in this case it's private corporations exercising their rights to exclude majority speech from their platform. This is an important right for them to have regardless of how popular a view is.
It wasn't that long ago that polling numbers showed the average American supported the Kent State massacre, and it wasn't that long ago that interracial marriage was illegal in parts of the US. The nature of "majority views" and the impact of reducing their visibility is difficult to grapple with.
It may seem juvenile, but it is the kind of thing people do in an adversarial situation when they have no better hand to play, AKA they "got nothin'".
I have no idea why they did it in this case, but I've seen it done to stop overly aggressive antivirus or antispam software from looking inside when the archive contains stuff likely to trigger false positives.
It's because they are juvenile, far-right goons.
SQL injection bugs are not rookie mistakes, it’s prevalent in many current and future applications. Look into Vtech sql injection hack, a large company with lots of resources had similar bug.
Look at previous hacks, solar winds hack, Sony hack, were all preventable common hacks.
Large companies are even more likely to run into this sort of issue.
It's still a rookie mistake. Any given well established company will contain a large number of 'rookies'. It's up to the company and everyone involved to make sure these are caught before going into production.
The reason it’s so prevalent is because it’s not a rookie issue and very difficult to fix properly, without impacting significant changes.
That’s why OWASP has it as huge part of security analysis and resolution.
This class of sql injection issues can be eliminated by simply enforcing that all queries are string literals.
Neither of these claims is true. Placeholders have been the recommended way to do this since the 1990s (I remember having this same talk with Perl & PHP 3 newbies) and one of the points of using a framework like Rails is that these are much easier to avoid if you use an ORM. The problem in this case is that they found a problem they (probably incorrectly) believed couldn’t be expressed in the ORM _and_ ignored the placeholder support _and_ didn’t validate their inputs. None of those require advanced experience to fix and at least the latter two are trivial to implement.
SQL injection bugs of this fairly trivial type are. This is literally what web tutorials were pleading with PHP developers not to do 20 years ago, and they weren't new then.
The whole situation is similar to having a construction scaffolding without safety railings, and calling someone falling off it a rookie mistake.
I like to think I'm pretty good, but I would never claim that and certainly wouldn't put money on it. I'm sure I've got boneheaded mistakes scattered through my code. Sometimes you're having a bad day. Sometimes the reviewers are a little pressed for time. It doesn't take much for really stupid code to slide through as long as it appears to work.
Glass houses and all that ...
I still judge the technical decision on how it's impacting the system today (specifically around whether or not it's sustainable), but trying to ascribe intent or brand something as "dumb" is almost always done without the relevant information that went into the decision being made.
I got this knocked into me by a very senior engineer when I complained about some really silly circuit on an older version of a microprocessor. His comment: "Okay, let's design that adder better. Oh, by the way, you only have two layers of metal in that technology. Let's walk through the options." My response: "Uh, no. We don't need to. There are exactly 3 options and only one that works." His response: "And now you know why the engineers prior to you made that decision. Don't assume your predecessors were untouchable geniuses, but don't assume they were moronic idiots, either. Just like you, they had a job to do, did it, and, most importantly, shipped it."
Remember Bobby Tables? That was 2007.
When it came to user security I _never_ fucked around. I have about 6 years of experience now and consider a trivial and catastrophic mistake like this unconscienably reckless. Never mind 20 years and experience working at a FAANG company.
However, I believe in blameless postmortems, even for CTOs. I'd feel awful to see this in the press.