Hacker News Reader

Comment by mort96 | Hacker News Reader

[flagged]

You're free to march on with your weird bias. The rest of us can benefit.

Can't "the rest of us" make other good compression formats?

Because large companies are willing to pay people market salaries to work on this stuff, and then give it away? Isn't that what we want, a model where free software development is properly compensated?

FOSS funding is a major problem. "Get a job at Facebook and make products for them" isn't a great solution to the FOSS funding problem.

It sounds like a perfect solution if you ask me. Free software gets made, developer gets paid.

Free software gets made, to the degree that it benefits Facebook, in the ways which benefit Facebook. Software development in areas which don't benefit Facebook suffers. Facebook gets people to use their software, and therefore gains influence in places where they don't belong. Yes, the developer gets paid, but it's not a good solution that's healthy for an independent free software movement.

TBH, I worry a lot more about this kind of thing in the case of Google, where they can afford to give away things that other companies would have made an entire business out of, which devalues those markets, then they lose interest and sunset it all— and only in a few cases (like Wave) does any of it end up open source.

But FB isn't really competing with other companies here. They're giving away frameworks, libraries, tools— this is building block stuff, and the fact that it's perhaps particularly optimized for FB's workloads is just their privilege, having been the ones to fund it.

It should be no surprise that Facebook works on projects that benefit Facebook. By making it free software, it also happens to benefit the wider world at the same time. Win-win.

Is your problem with the idea that developers are getting paid at all, or just that this instance is at Facebook?

My main issue is that I would rather not use Facebook products, and I would rather we don't create standards which rely on Facebook products.

The FOSS funding issue I more of a side-point which I didn't even bring up, but I don't see "get hired by Facebook if you want to work on FOSS" to be a healthy solution to the FOSS funding problem, we need a way to ensure funding for FOSS developers who don't work for Facebook. But that's not strictly related to the zstd thing specifically.

Calling a GPL-licensed piece of software with nontrivial public contribution "a Facebook product" is begging the question.

How is it begging the question? Is it the word "product" you're unhappy with? I can call it a Facebook project instead if you prefer, it doesn't change the substance of anything I've written. The problem I have is with the "Facebook" part.

I think the problem is that you have a hard time living in a community with some actors you may not like. That's just life.

I guess, but when there is already a great format which is GPL licenced what would be the point?

Make one that matches zstd performance then, and get back to us.

Doing this kind of thing requires a shitload of time and money.

Indeed--at this point zstd is the product of something like 25 man-years of improvements and optimizations.

A lot of consumer surplus is derived from companies trying to undermine their competitors by commoditizing non-core competency. To not take advantage of that is to have to bad parts of capitalism without the good parts.

Standardising on Facebook products is giving Facebook an undue influence in areas they don't belong. That's not one of the good parts of capitalism.

Reducing their own network traffic is an area they don't belong?

I didn't say Facebook shouldn't do what they need to reduce their own network traffic. They don't belong in our file systems and our gene sequencing standards and our non-Facebook web services.

Zstd isn’t a product. Facebook isn’t getting anything out of anybody using it. So it isn’t a product.

It's completely open-source, general purpose software that is essentially static (lower chance of drama or ideological/management concerns ala golang).

It would be like avoiding C because it was developed by ATT.

If AT&T still controlled C compilers and the C standard and and contributions to the C compiler everyone uses was gated behind AT&T I might strongly have considered avoiding C.

If a good non-Facebook implementation of ZSTD shows up, I might strongly consider using (that implementation of) ZSTD.

1. There are several implementations of Zstandard other than the "reference" libzstd that we maintain, tagged as "Ports" in the table here [0]. You are of course welcome to create or fork your own. We will happily take a PR adding yours to the table.

2. The format spec is an RFC [1]. It's frozen. We're not going to pull the rug out from under you somehow (and why would we??).

3. I think we do a pretty good job accepting contributions. But again, if we aren't, you can always fork it.

All in all, we've worked pretty hard to make zstd universal--portable, stable, flexible, robust, etc. I guess I don't really have a point... I guess I hope you reconsider your rejection of zstd? It's pretty cool.

[0] https://facebook.github.io/zstd/

[1] https://www.rfc-editor.org/rfc/rfc8878.txt

This is going to sound like I'm moving the goal posts, because, well, I am. I apologize. But what I notice with all those ports are that they're all in languages other than C (or C++), which means they're pretty much confined to their language's ecosystem; there is no port that's prepared to take the place of the Facebook zstd library. If I want to write, say, Common Lisp bindings, I'm not going to write them against the Java port. The one which comes the closest is probably the Rust port, since Rust is establishing itself as a core low-level language which people write bindings to, but the Rust port is only a decompressor.

I'm also concerned with the term "port". I don't know exactly what you put in the term, but to me, it sounds like someone basically translated the C code directly to other languages. What I would be looking for is more like a "clean re-implementation" by an independent group of people who properly understand all the algorithms which are used and where all the code is written from scratch without the Facebook code as a "crutch". This would ensure that whatever Facebook does, there will be a group of people that is capable of fixing tricky bugs in the algorithms or implementing further optimizations. Basically, I would want the alternative implementation to be as unrelated to Facebook's libzstd as Clang is to GCC.

All of that said though, from a purely technical perspective, zstd seems like a truly great piece of engineering. Kudos. Though I will never understand what drives great developers to dedicate their life to producing value for Facebook.

money and probably the fact that they are near to other great developers...

I simply don't think it matters. This isn't golang/react. Features, bugs and upgrades don't move with nearly the velocity of something like a large, opinionated library or language. But I rest my case.

Reply on news.ycombinator.com