“It's open source! We’ll let our customers fix it.” | Hacker News Reader

“It's open source! We’ll let our customers fix it.”

miles.land

145 pointsepoch_10049 comments

Contrast the work I'm doing this week with Microsoft's Graph API.

The C# library Microsoft provides for this is on GitHub, and their library docs it turns out are auto-generated ... But they aren't automatically tested! So it's possible for the docs to simply not work, and since they're auto-generated they're automatically kept not working. So GitHub issues about non-working docs get closed because even the maintainers can't make small doc fixes. Brilliant /s

I wanted to call this "Continuous Disintegration" but somebody already coined that phrase.

Your experience reminds me of this one phone platform with poor third party app support but a pretty vibrant indie developer community. They'd make unofficial clients for all sorts of things, only to eventually get lawyered out of existence. It was both sad and hilarious:

"Hi, to protect the blah blah blah of our service you must take down your unofficial application"

"Okay, but is there an official client coming for the platform?"

"We have nothing to share at this time."

"But if this issue goes unresolved for a long period of time, my employer might pay me to fix the issue myself and contribute the change upstream. That doesn’t sit well with me. We pay Google a lot of money to use their products, and having to fix bugs in those products ourselves isn’t what we signed up for."

I hear this, but I'm not sure that this is a case of Google expecting customers to fix bugs themselves or "outsourcing" to volunteers. I can think of a lot of instances where people would be thrilled if they even had the ability to fix a persistent issue with software that the company won't fix itself.

If you pay a lot of money to Google and expect customer service, it seems to me that you should be filing a bug with Google around BigQuery if you expect Googlers to fix it vs. the Apache Project. (If Google has committers in that upstream then they can be assigned the work if it's important enough.)

I guess what I'm saying is that filing a bug with the upstream feels like an indirect route if you expect customer service. I'd be filing the bug or discussing the flaw with my account rep / salesperson as close to the vendor as possible.

> I can think of a lot of instances where people would be thrilled if they even had the ability to fix a persistent issue with software that the company won't fix itself.

And that's Stallman's start in free software - he wanted to fix a printer driver but they wouldn't give him the source code.

What fraction of customers is affected by the issue?

That's always the first question I ask myself when I get a bug report. I can't fix every bug / annoyance. My software is already way more complex than I can handle, so I have to prioritize which bugs I fix. If the bug only occurs in a special setting and only happens for one customer that isn't important, then I probably won't fix it. I'll just say, sorry that my product doesn't work for you.

Now that customer will be annoyed, because my product was a good fit except for this one issue.

But if the product was open source, then they can fix the blocking issue themselves! I see this as a win/win situation.

> What fraction of customers is affected by the issue?

Also, is there a workaround and if so how easy/tedious is it?

Take the number of users in the field, A, multiply by the probable rate encountering this issue, B, multiply by the average time wasted working around the issue, C. A times B times C equals X. If X is less than the time to make a fix, we don't do one*.

(not speaking officially for any oss I may contribute to, just found the Fight Club resemblance fun)

I agree with this to a point.

I don't think this really has to do with open source vs. closed source. But let's look at what would happen if you were paying for a product, and were given a binary blob in order to interact with it. You find a bug. You report the bug to the company. They triage it, and decide it's an edge case, fixing it isn't a high priority for them, and they toss is somewhere in the backlog and expect to get to it in 6 or 12 months or whatever.

As a customer, you're stuck. Either you figure out a workaround on your end, live with the bug, or find a new product to use. The last option might be difficult, since switching costs are often not small.

If the client was open source, you could end up with the same reaction from the company, but in this case you have the ability to try to fix the problem yourself. Should you have to, in an ideal world? No, of course not. But we often don't live in an ideal world.

I agree that a company shouldn't have the attitude that open sourcing their client absolves them of all responsibility for it. I mean, sure, if they want to do that, that's fine. I would certainly guess that would make their product less attractive to many potential customers, but if they're ok with that, that's fine. It's up to the company to decide where to spend their finite resources, and a model where they let their users fix their bugs might actually work out best for them, even if it doesn't for everybody.

A self-respecting company doesn't let their customers just be stuck like that. While fixing the issue may not be feasible, there's usually a workaround that the company can help their customers find or implement. But then again, not every company lives up to these expectations.

> A self-respecting company

Citizen's United notwithstanding, companies are not people and don't have (or lack) self-respect. People who control policy for a company have all kinds of reasons why they do things, and while they might see and value one benefit of a particular action (lower support costs), they might not realize that there are negatives to that action (worse reputation leading to lower sales and poor customer retention). "Self-respect" has nothing to do with it. Companies make these sorts of errors all the time.

I don't know how we affect major change on this sort of issue, besides a vote with the wallet. I tend to encounter it most in areas which are already underserved - therefore you can choose not to pay but you're likely punishing yourself with a worse competing product.

The temptation will always be present for (some) organizations to use Open Source as a crutch to underfund development and support. I think it is a phenomenal way to grow your product in ways that you initially didn't expect from your customer base, but it cannot be the way in which your main support is bolstered.

I think the real promise of open-source is “If we die, and it’s important to you, you can keep it working.”

Also, security bug bounties natively.

The former is fairly pointless for the case discussed here, API bindings to a proprietary product.

That assumes the cloud service is irreplaceable. Having the source to the API means if Google shuts down their service you have the possibility of migrating it over to a new service with only a minimal API change. This assumes you have some way to get your data off before the service goes kaput though, which is not always the case.

It at least assumes that a competing service is different enough to give the API wrapper low value, yes.

If you're paying Google for their product, and you have support, the solution is to speak to their support team, and mention this. Failing that, you should contact your TAM. Even if Google's engineers looked at the bug, for all they know you're some random person who hasn't paid for support or whatever else.

I've reported multiple bugs with different vendors, and any time it doesn't get triaged within a week, I contact their support and mention it. It usually gets triaged pretty quick. This entirely depends on the vendor but more often than not it works.

Reminds me a lot of Rich Hickey's "Open Source is not about you" gist:

https://gist.github.com/richhickey/1563cddea1002958f96e7ba95...

The first few paragraphs really stuck with me in how I think about the OSS that I use.

The tone is blunt, but the content is spot-on.

For more context around what specifically is going there: https://www.reddit.com/r/Clojure/comments/a0pjq9/rich_hickey...

I think there's partly a generational divide there. I remember when GNU software was released as tarballs on a schedule of roughly "whenever the maintainer decides to cut a release" and CVS access was limited to the core maintainer(s).

> If you have expectations (of others) that aren't being met, those expectations are your own responsibility. You are responsible for your own needs. If you want things, make them.

Great advice for life generally.

> In general, open source maintainers owe you nothing.

My open source users donate money to me. So I feel like I owe them a lot and would never want to give them anything short of the best software in the world. It's amazing how a few bucks a month can totally change how you feel. Next time you use a piece of FOSS and find it cool and useful, consider tipping the person who made it possible!

I enjoyed this article, and I’m sorry for focusing on such a small thing, but can anyone explain the need to signal having attended Recurse Centre at the end?

> Thank you to everyone who provided valuable feedback on earlier drafts of this post, many of whom are Recursers.

I’ve noticed a few instances of people really drawing attention to having attended, and it always seems a bit awkward to me. Like, if I attended Cambridge and put “thanks to those who reviewed this most, many of whom are fellow Cambridge grads”, it would certainly come across as fairly pompous. I really don’t mean to put down the author, I just don’t understand the motivation for this kind of thing if not to boast, or signal some kind of social status.

Are alumni encouraged to advertise RC wherever possible? Or is it really that life-changing that everyone who’s been wants to share it with the world?

Author here. It's to encourage people to apply, and to bring attention to what I think is a really awesome place. I don't want it to come across as pompous, though I get that to some readers it'd come off that way. :/

I also feel a bit obligated to include this, because I got a _lot_ of feedback on this piece from other Recursers — if it wasn't for them, this post probably wouldn't exist!

Fair enough, I'm sorry for the harsh words, I think I'm just overly cynical, but I'm really glad RC was such a positive experience for you.

Comparing Apache Beam with Stripe's API is unfair in my opinion. Apache Beam can be operated outside Google Cloud, but the Stripe API is only useful for Stripe customers (as mentioned by the author). I don't think it's weird that a company provides more support to paying customers. The author should have raised the issue with Google Cloud's support team, instead of creating a Jira ticket for Apache Beam.

The article is specifically about the Google-maintained BigQuery component _of_ Apache Beam. But I agree that raising a support ticket is a good idea.

Apache Beam _can be_ but this person is talking about using it inside of Google (e.g. as the Dataflow API) and with a Google service. I think the comparison is on the money. That being said I think the best way forward would be both google cloud support & a JIRA ticket.

Although, to a degree, I think the authors complaint about Google's mentality towards open source is on the money, a lot of the OSS work is understaffed with the theory that the community will pick up the slack, even when the primary users of the OSS project would be Google customers.

(Full disclosure: I work on Apache Spark for my day job, have previously worked on Apache Beam for my day job, and have friends who work on Bigquery so my world view is maybe skewed).

> Apache Beam _can be_ but this person is talking about using it inside of Google (e.g. as the Dataflow API) and with a Google service.

Right, but it sounds like they filed the bug with Apache Beam, not with Google Cloud. Working through the Beam bug tracker means the team behind it doesn't have any context as to what kind of user they are. And even if they said "I'm using this with Google Cloud", the people they're interacting with there aren't necessarily paid to work on Google-related issues. Sure, a better answer (instead of "it's open source; you can fix it") might have been "you should contact your Google Cloud account rep to report this to them", but still, I think OP reported this bug in the wrong place given their expectations of how it should be handled.

Part of the problem is, if you open-source something, you will become overwhelmed with pull requests and critiques of every code change you make. People will expect you to do things in a transparent and community-driven way. They'll yell at you if you develop updates inside your company offices and then "throe it over the wall" to the open-source side.

open source still means we can fix their shit. when say, a gadget has closed source, we are stuck with it. the firmware is usually shit anyway.

I don't know if this would make the author more or less happy, but in my experience this isn't just an open source thing. I read the author's bug report, and just off the top of my head, I can think of multiple similar bugs that have been sitting around in my company's Jira for longer. The data analytics space doesn't have a zero-bug culture, so "weird error when using this specific intersection of features" tends to be seen as more of a feature request.

> But if this issue goes unresolved for a long period of time, my employer might pay me to fix the issue myself and contribute the change upstream. That doesn’t sit well with me. We pay Google a lot of money to use their products, and having to fix bugs in those products ourselves isn’t what we signed up for.

If they ignored the issue, it's likely they'll ignore the pull request to fix it as well.

A lot of companies seem to end up maintaining forks of various projects with fixes.

>If they ignored the issue, it's likely they'll ignore the pull request to fix it as well.

There is in fact a way to automate the ignoring of issues and pull requests : just add stalebot.

If an open-source software doesn't meet your needs, no big deal, you're free to use or create an alternative.

If a big company's open-source software doesn't meet your needs, no big deal, you're free to use or create an alternative to that big company.

If a government, ISP, or otherwise no-alternative's organization's open-source software doesn't meet your needs - then it's a big deal. But I'm not sure the examples in this article are of that case. Surely there are alternatives to Apache Beam and BigQuery.

> But if this issue goes unresolved for a long period of time, my employer might pay me to fix the issue myself and contribute the change upstream

And why do you care? That's the employer's money and their waste. You're getting paid to code what your employer wants, why do you care if they want you to code for Google? You have the option to quit, and your employer has the option to switch to another service.

> And why do you care? That's the employer's money and their waste. You're getting paid to code what your employer wants, why do you care if they want you to code for Google? You have the option to quit, and your employer has the option to switch to another service.

Exactly. We live in a world of limited options. "Perfect software that never fails" isn't a thing, and even "really good software that has the best support" isn't that common.

If you have a closed-source client and the company won't fix their bug, and you can't find a workaround, your only option is to search for an alternative that meets your needs (which may or may not exist), throw away a bunch of integration work, and start from scratch, possibly needing to heavily refactor or rearchitect your application. At least if the client is open source, you have one more option to consider before taking that big step.

I still get paid either way. If the project ends up being late because of problems with a dependency, that's life.

I want to see more examples besides these two. My suspicion: open source is a red herring here, and all we're seeing is Google's longstanding culture of being terrible at support.

This blog post is basically comparing two companies in different parts of the "standard" company life cycle: first make them love you, then make them use you, then ignore them[1]. Support in large companies is a high-leverage activity. To receive support you need leverage in terms of dollars spent per month.

[1] In the past I've characterized this from the customer's perspective as https://merveilles.town/@akkartik/106742502781631361

Ok, here you can see more examples: https://killedbygoogle.com

There are similar pages for other big companies too.

The author would have a point in case of a bug, but this is actually a feature request. Open source or not, customers are not entitled to demand new features.

> But last month, I filed a Beam bug report for an issue in Beam’s BigQuery integration (which, as far as I can tell, is officially maintained by Google). The gist of it is that when you’re using the native Python Beam implementation, you can’t upload data to BigQuery in large batches — you can only stream it, which is significantly slower than batch uploading. While it’s still mostly usable (streaming the data into BigQuery instead of uploading it in one big batch works well enough), the issue makes uploading some large datasets prohibitively slow.

I guess that’s cool.

The issue here seems to be that Google doesn’t allocate enough resources for that particular integration.

Meaning you shouldn’t use a Google technology if you don’t have hard proof that it's prioritized by Google on just the same level as it is by you. Which is not unlike the usual “Google tech tax”.

The funny corollary is that Google gets to hold a lot of hot potatoes. They wanted a browser monopoly, now they have to support the most widely used browser core. Everyone else may contribute. Or not.

This is very well reasoned and I mostly agree, but the reality is that if it was closed source it would be the same except you just couldn’t fix it yourself reasonably and legally. Google may be falling on its maintenance, but stuff like this crops up in proprietary software constantly. Presumably their calculus for putting resources on fixing your bug is just tilted against you. I think the cost of fixing this bug should be something that you consider part of the cost of using the service; no less than the cost spent maintaining or operating your own services should count when comparing the cost of self-hosting FLOSS stacks.

Tl;dr: I think open source is still just a straight improvement here. I think it’s fine as long as the cost of fixing it is considered the cost of using the stack for decision making purposes.

Completely agree. Having to fix a bug yourself because the vendor doesn't care is annoying. Being stuck with a bug because the vendor still doesn't care is a real problem.

This is especially infuriating when the vendor is charging you six figures a year for a "support contract", but doesn't consider fixing bugs you find to be a form of support. In a just world you would get a discount on your support contract for bug fixes you make to their product.

I think it is a great strategy to push maintenance costs on customers as long as they swallow them :) Now seriously it is fine, it makes some room for competition to jump in. Sure as a customer I would like to have a great support for little cost. So I will look for such solutions on the market. I could also whine, but it is not too productive.

This is a very sheltered experience. There have been countless times where I had to pull out the disassembler because the 3rd party company support has been incompetent and ineffective. At least with open source, I'm a lot faster at fixing a blocking issue than when I literally need to cook up a binary patch for it.

It's absolutely, unequivocally better that these tools be open source (and therefore fixable!) than closed source. My (admittedly small) gripe is just that a large, well-resourced corporation is shifting the maintenance burdens of its product to customers. This situation is certainly better than having to pull out the debugger or beg with customer support.

Each time this is true is an opportunity for a competitor to fill a gap.

Sure, except that there's a race to the bottom; genuinely Free Software will always be cheaper than corporate offerings. The author is not asking for people to compete with e.g. Google and Stripe, but asking for Google and Stripe to deliberately eat the loss when it comes to maintaining free libraries.

Those companies are not willing to eat the loss though for whatever reason. If the company cannot be invested in it’s product offering there’s no reason a competitor to that product couldn’t successfully rise up. These are not mutually exclusive ideas as a competitor may inspire the company to pay due diligence to the product.

I think these opportunities in giant corporations are exacerbated where your competitor may very well be within the company itself.

With what? A competing python library for Google BigQuery (referencing the article)?

I get what you're saying here, but I don't think this "gap for a competitor to fill" is accurate for all times that customer support has let you down.

An alternative implementation of BigQuery? If the integration libraries are that bad, perhaps the market would go for a BigQuery alternative with better integration support.

I expect that the integration libraries probably work pretty well for most users, and implementing a database like BigQuery would be a massive amount of work. So this pretty much explains why Google isn't sprinting to fix this issue.

I'm not sure that's just open source products. Par for course with the largest online services too... looking at you Google, Facebook, and Amazon.

At least with an open source product if I want to pay I can have an issue addressed ultimately. Good luck with that deleted YouTube account or owned Facebook business page.

Thinking on it I'm sure if you're paying enough IBM will be happy to throw developers at any problem you can come up with.

Reply on news.ycombinator.com