Google Doorman: Global Distributed Client Side Rate Limiting | Hacker News Reader

Google Doorman: Global Distributed Client Side Rate Limiting

github.com

73 pointsryancox25 comments

Hopefully there is a Googler reading who can answer this: I'm curious why the README.md says

> Note: This is not an official Google product.

However the copyright is Google, and you must sign their CLA. That seems pretty official to me? Or is there some other implication to "official" beyond ownership?

It works like this. Google has an "open source office". When you open source code at Google, it's either a product, or it isn't. Even if it's not a product (which has its own extra set of launch procedures), it's possible to open source it. It's still a Google-owned thing, but not a "product". Google ownership does not necessarily mean it's a product, but it still requires the copyright stamp and the CLA.

If it's easier, just drop the word "official". It's not a Google product- IE, a saleable or free service that google provides to customers- but rather just a thing Google put out in the world, subject to the rules that Google applies.

This is a project whose author is a Googler thus Google has the copyright and all the CLA stuff. However Google is not directly involved in the project, in the sense of having employees solely work on the project and have an OKR on this project, or roadmap whatsoever.

Think about this: someone work for Google wrote a software and Google is kind enough to open source it.

Disclaimer: I work for Google but my interpretation could be wrong.

Does Google also own code that you write in your spare time at home?

Usually (close to) universal answer is something like is: "If you've used any employing entity resources, even incidentally, your employing entity owns it, especially if the project is directly related to the business"

Some places will attempt to assert more ownership and some will not.

They pretend to, as would many companies if asked. I don't know how that would hold in court though. They also, for instance, pretend it's illegal to discuss your salary while the opposite seams to be true, so...

That is because Google claims copyright on everything you create while you work there, even if you create it in your garage in your free time. You baked some cookies for your daughters birthday party? Sorry, can't distribute them for free, consumers of those must sign a CLA (Consumer License Agreement) and Google retains copyright on your recipe.

This is false. Source: I am a Googler who has released open source code, written in my own time, using my own hardware/software.

Here's how it works (in California, anyway):

Google reserves the right to review open source projects to determine whether there is overlap with a current or potential Google business. This is done by submitting a form that describes one's project and then waiting a while for a decision. If the decision is "yes", then you are free to work on your project with your own time/resources, and publish it under your own copyright.

Google being Google, almost anything could be construed as overlapping with Google business interests. However, my anecdata suggests that Google is quite liberal in this regard: I had no problems myself, and know a few others who also had no problems.

In this case (i.e. this Hacker News story), I'm guessing that we're talking about something different. I suspect that the project in question was written using Google time and resources, and it is IMO appropriate for Google to own the copyright.

Question: Does this also apply to Google employees at Google Hamburg, and, if yes, how far? Because, as far as I know, German law directly prohibits such contracts.

I can't answer your question, because I don't know the answer.

However, I would recommend against relying on wereHamster's apparent mis-information... unless Google's policies are different in the jurisdiction that wereHamster is familiar with, in which case I apologize.

I think wereHamster’s text was satire, but the idea was repeated by many others: Anything related to your job you do is owned by Google.

You can’t contribute to ejabberd when you work on Hangouts.

> You can’t contribute to ejabberd when you work on Hangouts.

As a Googler I suspect you could, you'd just need permission (possibly per patch).

Googlers contribute to lots of open source projects. Google itself is fairly liberal in terms of what it's willing to share outside. Exceptions tend to be product specific code (you probably couldn't contribute the Hangouts UI and front-end code), particular technologies that are distinct competitive advantages (search ranking, etc.), or things that are Google specific (I work on Google Compute Engine -- there's a fair amount of code that is specific to Google's infrastructure).

What does it tell us if people don't recognise that as satire? Is it so close to reality to think that Google could claim rights on our cookie recipes?

And if you really have to ask if such a clause applies to you, you haven't properly read your contract and/or you don't know your rights. People, please educate yourselves, don't let large corporations, even if they claim not to be evil, violate your rights.

> And if you really have to ask if such a clause applies to you, you haven't properly read your contract and/or you don't know your rights.

I’m not working at Google, but I’m a compsci student in Germany, and obviously interested in the situation in the job market.

> What does it tell us if people don't recognise that as satire? Is it so close to reality to think that Google could claim rights on our cookie recipes?

It’s certainly possible that some might expect that – from the standpoint of someone who only heard of German law, for example, a large part of US employment laws would sound just as crazy as being able to have the copyright of your employee’s cookie recipes.

Rate limiting looks outdated. Suppose you can spawn new servers (and turn down unused ones) fast enough to react to demand, and have a good LB that sends the traffic where you have capacity, then you do not need to limit anything until you reach full capacity, in which case you probably want to degrade (by query class and/or client class) rather than limit anyway.

Rate limiting is an inefficient way to distribute a service, that makes sense only if you preallocate your resources and have queries with predictable cost. Let's use this technique only as a bug-prevention tool not for resource economy, as organised scarcity is likely as inefficient for the data center as it is for the distribution of goods :)

If you degrade by client or query class then in the case of just one class of query or one class of client you would have an unfair distribution, because one client could consume many resources and others would be limited to whatever's available on the margins.

I would like to see an adaptive system that, when resources are scarce, pushes towards a more equitable distribution.

I know it expects cooperative behavior. However, I wonder if the protocol design could be used in a setup where an embedded firewall did the rate limiting and mediated traffic from the possibly-malicious host. Boeing already has rate-limiting in their embedded firewall but cheap or OSS project could use a OSS rate limiter prebuilt to save time.

As in, is this protocol inherently cooperative or could an implementation have checks/controls added?

Offtopic but related: I want to like GRPC but I think Google is not working hard to make it quickly usable. Just try to perform a pip install in Visual Studio and obviously... it will not work. This is not the first time it happens, try to compile V8 (NodeJS is easy) or Chrome in Windows and it will be difficult even following the step by step instructions.

I have the opposite experience: The Go version of gRPC works like a charm, following the official instructions.

So do I understand this right: Doorman could be used to rate-limit access to a website by integrating it with the sites proxy server, the result being individual clients would only get a certain number of requests per minute based on available server side resources?

I think it's more meant for internal RPC within a cluster, to e.g. avoid overloading a service.

This is the correct answer, since this exact problem is a major issue when dealing with distributed system failovers. Here are a few examples of AWS service disruptions due to exactly this problem, where they were unable to recover the system because nodes kept failing over under load:

https://aws.amazon.com/message/5467D2/ https://aws.amazon.com/message/2329B7/ http://aws.amazon.com/message/65648/

An example quote:

"When this network connectivity issue occurred, a large number of EBS nodes in a single EBS cluster lost connection to their replicas. When the incorrect traffic shift was rolled back and network connectivity was restored, these nodes rapidly began searching the EBS cluster for available server space where they could re-mirror data. Once again, in a normally functioning cluster, this occurs in milliseconds. In this case, because the issue affected such a large number of volumes concurrently, the free capacity of the EBS cluster was quickly exhausted, leaving many of the nodes “stuck” in a loop, continuously searching the cluster for free space. This quickly led to a “re-mirroring storm,” where a large number of volumes were effectively “stuck” while the nodes searched the cluster for the storage space it needed for its new replica. At this point, about 13% of the volumes in the affected Availability Zone were in this “stuck” state."

So these things are very hard, can occur in totally unexpected situations, and I'm not at all surprised that a company like Google comes out with something like this.

It should have been called "Velvet Rope" :-)

Care to you explain the joke for the culturally unaware?

When waiting to get into a club or party, you wait behind such a rope until the bouncer allows you to enter (or doesn't).

Reply on news.ycombinator.com