GitLab is now running on GCP | Hacker News Reader

GitLab is now running on GCP

twitter.com

253 pointssahin152 comments

I wish they hadn't done this. Google has a unique and strict interpretation of US sanctions on countries like Iran and Cuba, much more strict than that of Microsoft or Amazon. If your site is hosted on GCP, Google blocks access to it from any sanctioned countries.

Exactly right. Much more attention needs to be brought to this subject. Imagining Google Cloud having as much market share as AWS is downright nightmarish. It would break large parts of the internet in those countries. I regularly travel to Iran, and can attest to a large number of random breakages (for example: the documentation of the date-fns library, because it loads data from Firebase).

Google as an infrastructure provider simply has no business being involved in where my content is available. If they sell me a server that is not accessible from certain locations on the internet, I consider them to be selling a broken product.

Google is very pro-government, pro-censorship, pro-regulations, pro-military. It's better to just accept that it will never be a decent infrastructure provider because of that.

I know Amazon or Microsoft aren't much better, but it's sad to see Gitlab use Google. I though Gitlab are the good guys.

Last year we couldn't access some aws server at work because it was on US gov blacklist. An aws server that hosted maybe hundreds of other unrelated websites. Amazon is just as bad. But I guess they have better PR.

You couldn't access _some_ AWS servers for a _period of time_, and you think it's equivalent to people in Iran, Cuba, etc. being completely blocked out of GCP? And the only difference is marketing?

Your post makes it sounds like it is US Gov that is behaving like Google (i.e. running blacklists) not AWS?

They voluntarily withdrew from China to avoid censorship even though they are now backtracking that decision. Amazon was the one providing facial recognition to law enforcement. I don't see any of them as clear "good guys" or "bad guys".

They withdrew from China due to the scorn of their security team having their asses handed to them by the people's liberation army ( https://en.wikipedia.org/wiki/Operation_Aurora ). 3 years later the exact same thing happened with the NSA ( Snowden leaks, NYTimes front page quotes of Google engineers swearing at internal network diagrams appearing in the leaks ), only ragequitting your home country doesn't quite work so easily

Snark aside, the asymmetry in their dealings with foreign governments compared to their own is quite illuminating, and the above is by far not the only such incident

More context on the "backtracking" remark: https://theintercept.com/2018/08/08/google-censorship-china-...

They bought a Chinese directory with a "search engine" that just redirects to Baidu, but kept all the juicy search terms to seed the future product.

I work at Google. Google can be very conservative and risk-averse in following regulations, but ask far as I can tell Google is one of the least pro-government, pro-censorship, pro-regulations, pro-military corporations in the world.

There are hundreds of instances of Google censoring websites, YouTube channels and plus accounts that promoted political philosophies that did not agree with the company culture. I think by that metric Google is very pro censorship when it comes to their own social media platforms

Name 5

sometimes they do a "soft" censor and just push them down the search result list so that any casual user would never see them... but they also do some hard censoring.

Is your point that Google haven't censored more than 5 times, or that Google mostly censors people who aren't famous/memorable and therefore it's hard to name 5?

https://www.reddit.com/r/conspiracy/comments/816mzt/big_alph...

Perhaps you're saying that, if provided with a list of 5 examples, you would defend why Google chose to censor in those specific cases. Unfortunately I don't know enough about the details of all these cases to be able to pick 5 particularly controversial examples, but I still feel uneasy about the amount of power Google has to silence opinions it disagrees with.

Linking /r/conspiracy doesn't help your case.

Is the information correct or incorrect? I’m not a massive fan boy if such kind of topics either. Often the crazy ones tell us research things that we often bypass.

This is one of those times. I don’t agree with their messaging, but Alex Jones should be allowed. Again I completely whole heartily disagree with him, but we can’t be selective in this area.

We’ve kind of validated him to some degree and made him much much bigger than before.

There is a class of people who feel like they aren’t being heard or lied too. Yet we often use tactics to discredit them.

I think this only embolden them. We can even discuss things, without doing the mental laziness of saying. I don’t wanna look at your source, because it’s x.

Within universities, you can’t use one source, why would you think it’s ok now?

You need supporting claims, if you disagree, than support it.. stop being mentally lazy.

How exactly do you find a list of YouTube channels "correct" or "incorrect"?

YouTube bans thousands of channels a day. It's their right to ban whomever they please and they likely didn't even give the channel owner a reason much less me. For all I know each one of those violated very reasonable terms.

Where did this list even come from? Why are these channels special enough to make it to this list? My guess is the poster agreed with their content, or found news about their banning through various conspiracy-oriented websites.

There's no reason whatsoever to think this list is interesting. There is, however, plenty of reason to think where it was posted is not worth my time.

The discussion has gone from "There are hundreds of instances of Google censoring ... YouTube channels ..." to "Name 5" to a list of dozens, to "YouTube bans thousands of channels a day", with different commenters making subtly different points at each stage.

I think the original point was that Google has a policy of censoring people based on the (perfectly legal) political views that those people have. As you say, Google has the absolute legal right to censor people because of their political views, and presumably you think it is not a problem if they do so.

So I don't think the interesting question is how many of the thousands of channels a day were banned for their unpopular political speech (nor is it interesting to know how special the channels on the r/conspiracy list are), but rather we should ask whether it is good for society if an entity as powerful as Google has this much editorial control over who is exposed to which ideas.

They have no such policy, and you have no evidence whatsoever that people are being banned for political views alone.

If I could show that YouTube channels are being banned from expressing certain political views, would you agree that Google has too much power and censors too much? Consider this recent example then:

https://www.bloomberg.com/news/articles/2018-03-21/youtube-b...

Perhaps you will say that "instructions on how to assemble firearms" is not an expression of a political view, however I would argue that preventing people from imparting legal factual information to help other people exercise a constitutional right seems like something that, if the government were to do it, would be a breach of their First Amendment rights.

So as not to be seen to be taking a specific view on "gun rights", let me give a different (hypothetical, and non-equivalent) situation to clarify what we mean by "banned for political views alone" here.

Suppose that Google had different politics and decided to ban channels that provided assistance to women seeking an abortion. Perhaps Google would allow channels to advocate, in the abstract, for "abortion rights", but not to give any practical advice on how to exercise them. I think that most people would agree that this would count as unconstitutional "viewpoint discrimination" if the government were to do it, so it is effectively a "ban for political views/expression" if Google do it as a private corporation.

They ban instructional videos on items that are federally or state prohibited. Shit they don't want to have to deal with. This hardly is a ban on political views.

The information itself isn't federally or state prohibited, though, right? If they didn't want to "deal with" it, then they shouldn't have a specific policy about it at all. Unless the government (or the courts) were pressuring Google on this issue, then it is simpler (and arguably fairer) to not try to be the arbiter of what is and is not allowable speech.

I can't speak for the people that create these channels or want to make these expressions, but I have to assume that at least some of them think that they are making people safer by teaching them how to legally arm themselves. That seems to be a view that Google disagrees with, so they have implemented an editorial policy on YouTube which prevents people from expressing that view in a meaningful way.

Perhaps you think that Google would only be banning people for their political views if the company openly said to them: "We are banning your channel because of a political view you have, even if you never express that view in any of your videos." That seems like an unreasonably high bar to require though. If Google said "You can support any party you want, but we won't allow Democrat political candidates to put their campaign ads on their channel." then that would, at least to me, be a ban on political views.

Bans Alex Jones from youtube (tbh fuck that clown)

Fires the memo guy

Complies to DMCA

My favourite is Google officially comparing EU’s right to be forgotten law with russia’s anti gay propaganda law... cannot get lower than that.

So 2 then.

Google has well-documented ties to the US state dept at the highest levels of both organizations. For example:

https://wikileaks.org/google-is-not-what-it-seems/

There are many allegations in there, but the most salacious one is that Schmidt’s girlfriend was illegally working for Hillary on State dept matters. (I believe they made audio recordings of the phone calls that prove this, but can’t find the shorter write up anywhere).

Since the sibling posts are mostly right wing, and I only mentioned Hillary Clinton, here is a more bipartisan example:

https://en.m.wikipedia.org/wiki/Jared_Cohen

The CEO of Jigsaw (formerly Google Ideas) advised both Condoleezza Rice and Hillary Clinton.

What's the actual legal argument here? What's the text of the regulation? Why does Google interpret it to mean this whereas Amazon and Microsoft don't?

Are you an official spokesperson for Google?

Just literally so wrong lol. Not even any citations or sources

Because they are targets of everybody, liberals and conservatives alike, even their own employees. Everybody want to have a piece of Google in the fuzz they intended to create just to advance their own agenda. I won't blame Google for playing safe

Talk about hyperbole. "Nightmarish" smh

Imagine the internet with all AWS data centers inaccessible to you.

Google as an infrastructure provider simply has no business being involved in where my content is available.

No, Google has no business in it, but export control does.

Microsoft and AWS don't seem to have any problems complying with the law

Wouldn't every US based company be under the same laws in regards to sanctions?

Laws don't dictate how companies behave.

Actually they do most of the time, but when there is room for interpretation, different companies with different incentives choose to interpret them in different ways.

The legal requirements are lower. They are not allowed to sell hosting to persons from sanctioned countries for example, and all companies follow that rule. But interpreting the law to mean you cannot show hosted sites to visitors from those countries (like Google) or that you cannot allow downloads of free software from your website (like Oracle JRE) is an over-reaction only some companies do. The real reason is that since they cannot extract money from these countries because of the sanctions, they don't care about their existence, or how much trouble people from those countries would face due to their decisions.

I'm not sure if that is the reason. It is rather puzzling. A lot of Google Products work just fine, say Google Mail. On the other hand, the whole Google developers site is inaccessible (including everything from Android, to their Chrome/HTML5 information site etc). US law clearly requires no such action on their part.

The don't earn money by letting packets from Iran route to their network; The people who could be losing money are their customers who's content is restricted.

> cannot allow downloads of free software from your website (like Oracle JRE)

The Oracle JRE contains strong cryptography which is illegal to export to certain sanctioned countries. It being free doesn't change anything about that. That one is actually legitimate.

It's not just JRE. You cannot download any software from Oracle, period.

Plus, is it really illegal to export cryptography software? If that is the case, why are Java downloads the only ones restricted? Shouldn't, for example, Visual Studio and .Net also be blocked? I don't think it is for any valid legal reason. Just that they cannot make money from these countries, so they can't be bothered to let them access their download servers.

Yes[1], export is controlled. Oracle Java contains an implementation of cryptographic primitives, similar in scope and purpose to OpenSSL.

Microsoft does not sell products to any sanctioned countries[2].

Note that [1] has a giant loophole for open source software, which requires notice but not permission - so builds of .NET Core, OpenJDK, OpenSSL itself, and so on, can be exported no problem. Also, programs built WITH the primitives are "mass market", so are exemptable that way.

[1]: https://www.bis.doc.gov/index.php/policy-guidance/encryption [2]: https://www.microsoft.com/en-us/exporting/faq.aspx

Last time I checked, you couldn't even use strong cryptography in Java without tiny additional download. So forbidding JRE downloading seems a bit excessive.

As of 8, they integrated that portion into main tree because it was ridiculous and a bit of a pain for the 98% of the world who wasn't under US sanction to have to do that. It's "legitimate" again - so all the Iranians that want Java have to download it from the source tree.

> The real reason is that since they cannot extract money from these countries because of the sanctions, they don't care about their existence, or how much trouble people from those countries would face due to their decisions.

That doesn't make sense to me, letting people from those countries access resources on GCP does make Google money, since presumably Google will be paid by the website owner (which doesn't live in any of these countries) for the extra bandwidth/resources used by their website.

Ironically you might have inadvertently provided the answer to your own question. If Google make money from a website owner (in the EU, for example), and that website owner makes money (using their website) from someone in a restricted country, then an imaginative prosecutor could argue that Google are guilty of knowingly facilitating some sort of "money laundering", from the restricted country to Google themselves.

This is not much more far-fetched than the idea that a site like The Pirate Bay is guilty of "conspiring" with its users to infringe copyright.

This is not much more far-fetched than the idea that a site like The Pirate Bay is guilty of "conspiring" with its users to infringe copyright.

"We're a piracy site, run by pirates, and we love piracy and pirates because of what piratical pirates we are, so we built a piracy site specially to cater to pirates and piracy! Also, we do not condone it in the slightest and built it for totally legitimate purposes, we promise!"

Yeah, gonna go with "no" on that one. It does not take an "imaginative" prosecutor to find the incredibly unbelievably subtle and deeply-hidden links between the Pirate Bay and its intended purpose.

I see what you're saying, and perhaps The Pirate Bay wasn't the best example I could have picked. Remember that the site IsoHunt was taken down as a result of a lawsuit, despite the name of the site literally suggesting it was a search engine for finding disk images of Linux distros. They also effectively complied with the DMCA, despite being based in Canada where the DMCA didn't apply:

https://en.wikipedia.org/wiki/IsoHunt#DMCA_takedown_notices

Of course the "intent" of the accused is something that a legal process can consider, but I've always been unimpressed with the argument of "It's right in the name!" when considering whether to censor The Pirate Bay. If they had changed their name to the "I Rate Bay", and branded themselves as a site for people to discuss and rate the quality of certain hex strings (magnet links), then, by the "It's right in the name!" argument, the site should be perfectly legal.

There were a number of factors which lead to the legal judgements against The Pirate Bay (across the multiple jurisdictions where the site is censored), but if the legality of a site were to come down to a single letter in its domain name, and not based at all on the actual functionality of the site, then I think that would put us in a very odd place in terms of freedom of speech and the enforceability of the law.

We're back to the "wine bricks" of the Prohibition era, with their "warning" not to leave the contents dissolved in a gallon of water in the cool cupboard for 21 days, to prevent "accidentally" creating wine:

https://www.smithsonianmag.com/smart-news/ingenious-brick-he...

I would hope GitLab did research on its own customers to be confident that the benefit significantly outweighed any negative impact.

I think Alex Jones is a fool and spews all kinds of garbage into the public discourse, but I don't know why he would be banned but not countries officially recognized as problematic by Google's governing authorities (the U.S. government).

That is "as long as it's legal" doesn't seem to be the governing principle for Google right now.

If you dispute a law, your issue is with the government that passed and enforces it, not companies abiding by that law.

This is not a legal requirement. Amazon and Microsoft do not do this, nor do Heroku, DigitalOcean,and countless other small and large companies. Google's interpretation of the law is unconventional and almost unique. (To the best of my knowledge, the only other company who interprets the law this way is Oracle, and that's saying something).

The IBM Softlayer Cloud does, or used to do it, as well.

It's hard to tell until it's decided in a court. Until then it's everyone's best guess. Googles compliance officers might find it not worth the risk.

There might also be other reasons one would prefer to follow the spirit of the law and help put pressure on two dictatorships where at least one of them seeds instability in many countries.

Other cloud providers don't do it. The idea that letting TCP packets be routed to the Google Cloud datacenter falls under "providing services" is clearly an incredibly expansive reading.

I expect Google to make a little bit of an effort here.

> If you dispute a law, your issue is with the government that passed and enforces it, not companies abiding by that law.

It's another vector to apply pressure to get the issue resolved. There's the direct course (you to your congressperson) which likely won't matter, unless you have enough like-minded individuals all applying the same pressure.

Then there's the indirect course (business to congressperson) which will likely matter more, since the business can donate larger sums of money, all at once.

Politics is the opposite of business, as far as cash-flow is concerned. Business wants sustained, regular amounts to allow for forward planning. Politics wants bursted, large amounts of cash to immediately address issues of the day.

OVH is probably the only major non-US based public cloud provider[1]. It also happens to be the host of Wikileaks.

[1] https://www.ovh.com/ca/en/public-cloud/instances/

Hetzner would like a word with you. Not only that, but there are tons of low-profile "cloud" hosting providers who only market to a single nation or a small region. Their websites are usually not in English. A couple years ago I did some digging on prominent South Korean websites, and they were all hosted by national providers.

"On January 11, 2016, Hetzner blocked the St. Petersburg site of Novaya Gazeta, the leading oppositional, non-governmental newspaper in Russia. The newspaper marked the act as political censorship without any legal procedure."

https://en.wikipedia.org/wiki/Hetzner#Incidents

Kudos on your Wikipedia research, but I don't think this has anything to do with my comment.

I pointed out OVH as a cloud provider with a strong stance on the internet without borders.

Hetzner doesn't come even close in this regard.

Perhaps in your mind you did, but the comment I replied to was not so specific.

You might be right. Nevertheless, Hetzner is not an alternative to GCP.

Can confirm Hetzner, I've been using hetzner for months, in addition to AWS and GCP.

Coming soon to the USA. Scroll down to the map labeled "OVH Public Cloud regions" on the link you provided.

In fact, OVH Cloud[1] is already available in the US, but operates under an entirely separate legal entity to prevent from the access to non-US customer data[2].

[1] https://ovhcloud.com/

[2] https://www.ovh.com/world/a2294.how-ovh-will-balance-develop...

Devil's advocate: IANAL but my understanding is that the kind of safe harbors that exist in other areas of IP law, e.g. that network providers are not responsible for copyright violations, either do not apply to export controls or have not been subject to sufficient legal review to establish a precedent that would protect Google from export control liability from software that is hosted by customers on their infrastructure where the customer has not undertaken the necessary measures to implement the export controls required by law.

If you think the law is unjust, then target the law, don't target the companies which err on the side of legal caution. After all, certainly we'd prefer for companies to err on the side of regulators in many other cases...

There are billions in fines of evidence that they don't care about the law and are willing to ignore it as long it benefits them. They are "legally cautious" purely strategically.

While this sentiment is applicable (and agreeable) now, I can't help but think it is a little short-sighted.

Can we really say <insert_megacorp_name_here> will do or not do 'X', or has some immutable policy 'Y', with any reliability? Historically, have we not seen that these policies change with the direction of the wind, and/or are applied inconsistently to suit narrative 'Z'?

What is true today may not be true tomorrow. There is no bastion of moral decency in cloud infra, and to make decisions based on a current stance is facile and probably pointless.

I'm far more interested in what awareness the GitLab folks have of this issue, and procedures in place to mitigate it, if any.

I agree this probably hurts if you're in those countries or doing business with entities in those countries, but what business would stop an architecture overall that they believe is the right move, only because of the small Iranian/Cuban market?

Either way, the best way to fix this is to lobby Congress.

You're right that for most, there isn't a business case to care about access to these sanctioned countries (there are more than just those two that Google is blocking). I wouldn't say you won't be locking out any users: people in Iran obviously do use, say, source control. But sure, the monetary loss is very limited.

But ideology can be a powerful reason. And people in Open Source can be very ideological. Certainly for me, this makes Gitlab a no-go. I want my code to be accessible by everyone. I do wonder what kind of pressure could be applied to Gitlab in terms of free software developers shunning the platform.

We are sorry that there are legal restrictions that are imposed for these countries. There isn't much we can do about this, but we have an open issue discussing this [1]. You can also see the U.S. Department of the Treasury link [2] for more details regarding these sanctions. We planned this move because we wanted to improve our stability, and you can read all the technical details that affected our decision in our blog post [3].

[1] - https://gitlab.com/gitlab-com/migration/issues/649

[2] - https://www.treasury.gov/resource-center/sanctions/Programs/...

[3] - https://about.gitlab.com/2018/06/25/moving-to-gcp

GKE is fantastic to work with, we rolled with that on a nascent project in favour of Heroku and other PaaS. We had one or two rough spots but we’re not paying nearly as much as we would to have the entire thing managed by other parties, along with the accounting and awareness overhead that incurs. Kubernetes has a learning curve but it delivers an elegance too.

The ulterior motive is that understanding Kubernetes now has serious professional value in and of itself, so if the product fails your experience certainly doesn’t.

That said, I’m surprised Gitlab didn’t announce a new nav redesign at the same time ;)

But the cost of Heroku on free, or nearly free tier, isn’t comparable to GKE. When I tried to launch a demo cluster it was basically launching numerous VMs just to support itself. I don’t know the monthly total, since I abandoned that idea pretty quick. But I imagine it’d be over $50/mo to run a tiny web app.

My point is that every offering has its place. Heroku is great for getting going.

Recently I’ve been experimenting with Dokku. I think it’s a totally viable alternative to Heroku. I do value my time, and understand the value of PaaS. But when Heroku starts charging insane amounts for Redis memory, it made me look for easy alternatives and Dokku fits the bill well.

You can have it both ways by leveraging the free / hobby tier of Heroku with bringing your own data store. Nothing binds you to their resource providers. Just add an environment variable for your own database or Redis provider.

Then you lose some security though, because I think Heroku uses large IPs ranges [1]. There seems to be no way to allow database access from a private IP address, other than using expensive proxying addons. That means you probably have to open up your database to 0.0.0.0/0, which is not good at all.

[1] https://stackoverflow.com/questions/15512360/get-a-finite-li...

Nothing stops someone from spinning up a Heroku bash terminal and trying to access your internal DB either so it’s not markedly worse than pure Heroku.

You can get better firewall control than just open/shut

Would you mind sharing what you have in mind please? I'm genuinely interested in how that would work.

Sure, I have a PG database on the internet. It's firewall allows only traffic from my whitelist IPs. Additionally these PG servers require SSL[0] from only my CA so even if firewall is open, no access.

[0] https://www.postgresql.org/docs/current/static/ssl-tcp.html

Is Heroku Postgres locked down by default? Maybe I’m misremembering, but I’m pretty sure I’ve used the Heroku database url on my local machine before.

Redis has authentication.

Heroku has "private spaces" which will give you a fixed set of outbound IPs, AWS VPC peering, and a firewall via Trusted IP Ranges.

Heroku is surprisingly nice as an enterprise customer.

https://devcenter.heroku.com/articles/private-spaces#stable-...

GKE is free. You could run a micro vm for 5$ with kubernetes orchestring it.

GKE itself is free - you have to pay for each node you use though, and GKE requires some core services that take up most of the space on 1-2 standard nodes in your cluster.

Anyone noticed any performance changes?

Anyone from Gitlab have any data to share? I had a look at https://monitor.gitlab.net/d/000000003/fleet-overview?refres... but don't know what I am looking at :-)

General browsing around feels snappier than it did before. I just tried importing some new repos from Github and it was fast (like 2 or 3 seconds for tiny repos, under 10s for larger ones). During the aftermath of the Github acquisition repo imports were taking days, but that was probably because of the load I guess.

Would be good to back this up wit some data :-)

I’d also be interested in some actual performance metrics/benchmarks. I’m a big fan of gitlab, love gitlab CI, and I Iike (most of) the UI more than github’s, but my one biggest gripe with it it is how sluggish browsing it at times could be, groaning if I accidentally clicked the wrong file because now I have to wait/stop that and go back and then wait for the one I actually wanted. It’s certainly improved over the last year to the point where I only occasionally had to sigh as I navigated through a repo, but it’s been creeping towards GitHub level performance at least.

However, I did take a few minutes just now while I’m farting around on my phone on the toilet to go and browse through random repos, opening random files, looking through issues/merge requests, etc, and it definitely seem noticeably snappier, opening large files from my phone seemed to be as quick as github, no sitting twiddling my thumbs waiting for a 200+ line file to load like I had previously experienced many times. Hopefully someone can confirm this with some actual numbers, but first impressions are pleasant.

If the performance improvements are actually significant, and hold up overtime, this will certainly help gitlab’s ever growing popularity as the one complaint I always see get brought up in gitlab discussions is it’s performance (aside from a few that don’t like the UI — but that’s a bit more towards personal preference rather than objective metrics).

It’ll likely take some classic 80s-90s Microsoft “business strategy”, or more recently what they did with Skype, for gitlab to ever have a chance at overtaking github. But, as a gitlab user, and fan of competition/not having essentially a single (closed source) entity being the “hub” of all open source code.

All this said, while I’m more than happy to see major performance improvements to gitlab.com, what would make me even happier is to see some performance improvements for self hosted instances. But, I realize that takes quite a bit more work to improve since it likely involves quite a bit of code/app level optimizations, rather than just throwing more computing power at it. Our instance at work is usable, but good lord can opening large files/directories be annoying, and since unfortunately gitlab isn’t really considered a hyper critical system at my workplace, deploying it to largely scaled & optimized GCP/AWS/Azure infra isn’t likely going to be an option anytime soon. So, gitlab just please don’t forget about us lowly folks that have it deployed to a small cluster/single server who can’t just throw more and more resources at it.

Also, if there any gitlab folks reading my rant, and you want to grant a wish of mine/make my work life significantly more enjoyable, if you guys could get back to the people that have been in contact with you about getting a education gitlab ultimate license (I work at a large university)[1], I’d likely forget about the performance issues entirely. They said they reached out to you guys, however they said they were told the education license program wasn’t ready to be completely rolled out or something?

[1] https://about.gitlab.com/education/

We're working on making GitLab as fast as possible. We have a list of all ongoing efforts to improve performance of GitLab (the product) at [1]. There's also a list of ongoing efforts for improving performance and availability of GitLab.com specifically [2] (performance) and [3] (availability).

As for the differences compared to being on Azure. We don't have any definitive results right now. We'll have to wait for Monday so we can get more performance data. Most likely, we'll have a better image if and how performance changed in the middle of next week. That being said, we did observe an improvement in p95 response time for our web and api nodes - it's a roughly 30% to 40% reduction. We're working on a public monitoring page so you and the community can browse through these metrics at your own leisure.

---

With regards to the Education program - it's now in full swing and available to everyone. You can apply via [4]. We sent out an automated email to let people know it was still in preparation some time ago. It should have been followed up with an email stating that it's fully functional too. Not sure why the university you work at didn't get the second one. Let me know if there's any holdups with your application after you apply through the linked form.

[1] - https://gitlab.com/gitlab-org/gitlab-ce/issues?scope=all&utf...

[2] - https://gitlab.com/gitlab-com/infrastructure/issues?scope=al...

[3] - https://gitlab.com/gitlab-com/infrastructure/issues?scope=al...

[4] - https://about.gitlab.com/education/

Oh yes, I’m definitely aware of your guys hard work on increasing performance, and it definitely hasn’t gone unnoticed/unappreciated. I’ve been using gitlab for a couple years now (drew me in with free private repos, and now have happily locked in with all the CI/devops tooling stuff, having that all in one place is what I love about gitlab) , and the difference between the performance now vs two or so years ago is incredible (that’s even with all the features that have been added since). I just wanted to express my only complaint I have had/frequently see, which as of recently has hardly ever been an issue. Also, really thankful it’s open source, as I find myself looking through the repo fairly often to see how you guys handle things in such a large app, as I’m the sole developer on an ever growing Rails app, gitlab’s code has been one of my gotos for ideas/seeing how something could be done. Hoping eventually I’ll get familiar with it enough that I’ll be able to make some contributions to those performance improvements instead of just complaining online about it ;).

> With regards to the Education program - it's now in full swing and available to everyone.

Awesome to hear! The group that maintains the gitlab instance is just a few volunteers that meet once/twice a month to work on maintenance/upgrades, and the update I got from them about the education license being on hold was ~2-3 weeks ago, however this month’s meeting was cancelled for whatever reason, so they likely have gotten the update that it’s now available, and just haven’t had the chance to work on getting it setup yet. But, I’ll ask them Monday, and send them the link if they aren’t already aware. Can’t wait to finally get to use some of those awesome ultimate features at work.

Appreciate the response! Always like seeing you guys pop in to these threads with some extra tidbits/insights/help.

A lot of the performance gains you get from GCP come from taking shortcuts over private fibre.

Traffic inbound to a GCP-hosted system first goes to a local point-of-presence, then makes relatively few hops to reach its destination via Google's private fibre network.

You can get a discount if you let packets make their own way to the destination over the public internet, but in practice that will make sense for cost-sensitive bulk operations, rather than interactive usage.

Dave from the GitLab.com infra team here. The numbers are obviously early, but we are seeing an improvement in p95 response times for all calls to api, web and git. In many cases they look to be 60-70% the times we saw for the same time Saturday last week. We expect to have real data as we go into the week and will share what we see with normal load from Monday.

We are working on a new public monitor page to share soon too.

Backstory: https://about.gitlab.com/2018/06/25/moving-to-gcp/

Real backstory: https://news.microsoft.com/2018/06/04/microsoft-to-acquire-g...

Why does this dumb story keep getting posted on every GitLab thread? GitLab started the move to GCP way, way before the acquisition from MS happened. They are not related.

> Why does this dumb story keep getting posted on every GitLab thread?

I suspect it's because some people really like microsoft and are personally offended that a customer left.

Or it's just the penchant across Internet forums for the 'real truth.' Reddit is overflowing with it. People often like to believe they have knowledge that few others possess, special insight. They're not fooled by the mainstream media, they see around corners and through the guise, they've got it figured out. There's a massive culture for that today, which is where the Alex Jones, flat earth, etc. popularity boom comes from. Those people used to be a lot more isolated, now they can link up and spread their special insider knowledge, which leads to every story having an alternate 'real truth.'

>I suspect it's because some people really like microsoft and are personally offended that a customer left.

I would do the same thing. Not sure what you're implying. Now get back to hobby and improve pmos (pls).

The GCP move was decided upon months before Microsoft acquired GitHub[1].

[1]: https://venturebeat.com/2018/04/06/why-and-how-gitlab-abando...

No doubt, but the Microsoft acquisition must have pushed the priority of the move through the roof.

Negative. I work at GitLab. I can assure you that GitHub getting acquired did not have any bearing on the speed at which we made this migration happen. It's been in the works for a long time, and in none of the meetings I've attended has anyone said "Github now is owned by Microsoft, we need to get on GCP FASTER."

Someone should have then. On the other hand, you only talk about the speed of the migration, could also be an overly-specific denial and you've known about a potential acquisition before.

Call me a tinfoil nutjob, I still don't buy it and would move away from azure too in that situation.

Do you have any evidence backing that up? It seems unlikely that, say, Microsoft would endanger a large and growing business by artificially sabotaging GitLab in a way which would be quite obvious.

Who cares about evidence and "what's more likely". Would you host your product on your competitor's servers? I'm not saying that's why they switched, but from a business perspective it makes sense to remove that risk.

What’s the actual risk? Once you’re off in conspiracy land, why not worry that they’ll pay off a supplier or something? In the real world, blowing an Azure contract or SLA would cost more in lost business than they paid for GitHub entirely.

I can't think of any good reasons to keep your product hosted at the hands of your biggest competitor.

Yes there are examples out there but I think this is a very different situation where proprietary technology is a main advantage.

Any reason to suggest otherwise is dwarf by this reason alone. Good move and logical.

The move has nothing to do with Microsoft acquiring GitHub.

It's less about getting away from Microsoft and more about getting closer to Google: https://techcrunch.com/2017/10/09/gitlab-raises-20m-series-c...

While other commenters have criticized the "hidden truth" angle of this comment, I'd like to point out that some insiders may have foreseen GitHub aligning with MS a little earlier.

I had a hunch, way before Microsoft announced the acquisition, that if GitHub was ever sold it would be to MS, as it aligns with their outreach to open source developers. At the same time, they were quite active on, and happy to endorse GitHub, which is a kind of cross-promotion large corporations only do when they're stakeholders of the smaller company (or at least have some contract defining the terms and legal consequences related to this PR activity)

Microsoft became and are extremely active on Github, but were they really uniquely endorsing it among large tech companies? AFAIK Google, Facebook, Twitter, Amazon, Netflix all run all their open source stuff through Github. Even Apple for the open source stuff they actually want people to use.

Github being owned by Microsoft has 0 relevance for any competent technical organization.

Saying anyone who cares about Github now being owned by Microsoft is incompetent is borderline ad-hominem.

We have seen embrace-extend-extinguish from MS before. Even if you believe they've changed there's no reason to label everyone else as incompetent.

That's not what I'm saying. The news has 0 relevance to teams, whether those teams care or not.

Github was a for-profit organization losing massive amounts of money with stalled development and weak features compared to competitors. It's only major asset was the large community. It now has a many more resources to stay around long-term and finally start building better products. Your data is also 1-click away from being exported and git is already distributed meaning every developer has a complete clone.

What possible issue is there to worry about?

Has anybody tried GKE and EKS from Amazon? What's your opinion on the later?

Here's a nice recent blog post about various issues you have to deal with to even set up a EKS cluster in AWS:

https://medium.com/@dmaas/setting-up-amazon-eks-what-you-mus...

I get the impression that compared to GKE, EKS is not yet fully baked as an easy to use product. It still involves too much manual fiddling around (e.g. see above) and operational complexity.

I'd advise any clients of mine who want to run Kubernetes in the cloud to stick to GKE for now. The lower prices on GKE are a nice bonus also.

That said, I don't discount what Amazon can do in this space. If they make the setup/admin experience as easy as spinning up a VM on AWS then Google will have a serious competitor. But not yet.

BTW, for smaller shops and solo devs, I hear good things about the upcoming Digital Ocean Kubernetes service. It's in beta right now:

https://www.digitalocean.com/products/kubernetes/

I've spent the past two weeks trying to get Gitlab and Amazon EKS up and running and that blog post is pretty much spot on. I'm actually halfway through exploring some of the ingress options discussed in the post but I think I'm just going to try another provider now. Thanks for posting.

> BTW, for smaller shops and solo devs, I hear good things about the upcoming Digital Ocean Kubernetes service.

How would smaller shops and/or solo devs benefit from using Digital Ocean Kubernetes service over GKE?

I recently had a deep dive on EKS as part of an architectural review for a large scale (200+) microservice deployment. The recommendation was to strongly avoid EKS. Bad enough that the platform is brittle and expensive, the Amazon engineers sneered at Kubernetes as a "fad" and pushed us heavily towards ECS. I put it down to hubris but not a hope in hell we'd put our services onto a platform built with that attitude.

I've used it for the last month+ now and it's not quiet there yet. Setup via terraform was mostly fine although the default subnet sizes were too small (since the networking layer seems to over-allocate IPs you only get 30% of the IPs usable for pods). The metrics backend is broken so metrics based auto-scaling of pods doesn't work. Also, r5 nodes aren't supported by the networking plugin since someone forgot to add them into it. User authentication for access to EKS is somewhat awkward. Also, the user role that creates the EKS cluster has magic admin access which I can't find a way to control.

EKS is basically half-managed. They originally had a better setup for master nodes but GKE has caught up with free masters and regional deployments (1 per zone) for the same high-availability.

EKS masters will still run most of the cluster services to free up your worker nodes, but that's because they have very poor control over everything other than the masters. And since that's the most important part of the cluster, you will run into a lot of manual work and broken systems getting the cluster up and running on EKS.

GKE is about 10x smoother and everything works as expected, and they are rapidly improving things with much better storage and networking options coming soon. At that point the experience will become 100x better than EKS.

The recent thread about Kubernetes on Azure has some experience reports for EKS as well: https://news.ycombinator.com/item?id=17700360

I missed watching their livestream as it was 3am, and my YouTube search-fu isn't good enough to find the archive link.

Is there an archive link, or is it just gone because I didn't watch right then?

We didn't do a livestream since we were concerned about showing credentials in the terminal in case things didn't go according to plan. We did record it and plan to publish it since it went according to plan.

Great, thank you for clarifying that. I haven't yet had to carry out such a massively involved migration plan, and so I'm always interested in learning from the process of others.

And congratulations to the whole team on the move. I know it must be a relief now that it's behind you.

Thank you.

It's not quite over though. Monday traffic will be far higher than the weekend. People will need to login, which will put unusual stress on those endpoints.

Also, there is a small chance someone experiences malformed or missing data. There was only a single repo that we know experienced this, which we're already working on. So I'm only talking about unknown unknowns. So we will be on high alert to respond to any such requests and retrieve data from the archives in the previous infrastructure.

GitLab VPE here. This instinct proved correct. We'll look into editing the video this week after the team gets a breather.

Interesting to see this now and then go back and read their post from 2016 where they stated they knew it was "time to move away from the cloud."

https://about.gitlab.com/2016/11/10/why-choose-bare-metal/

See https://about.gitlab.com/2017/03/02/why-we-are-not-leaving-t... for why we're still in the cloud.

Thanks! Good read.

Why did GitLab go to Azure in the first place?

Usually because of discounted or free credit.

Maybe they want to be acquired by google

You doubt it? Google is an investor in GitLab. They also push heavily for GCP to their customers offering USD 200 free credits (above the USD 300 you get by default). Zero marketing mentions regarding integrations with AWS, Azure or any other cloud providers. I wouldn't be surprised if they are hostile in terms of integrating with other Cloud providers.

You even get marketing promotions about GCP in the consoles empty states. Sometimes it feels like a cheap Google adware. I cringe when I see so many Google references in GitLab.

GitLab is basically a subsidiary of Google Cloud at this point.

I really like GitLab since their service and user experience is top notch. I use it in all my side projects. I just wish they weren't so heavily biased to push and promote GCP. If they advertised themselves as a cloud agnostic and completely independent Git solution, I would be more trusting of their service. But the reality is the opposite. Sadly.

Hi, GitLabber here. There are several inaccuracies in your comment. Hopefully, I can clarify and comment.

> Google is an investor in GitLab

Not true. GV invested in GitLab. GV != Google. (Just do a web search to learn the difference.)

> push heavily for GCP to their customers offering USD 200 free credits

True. It's free credit for our users, of course, we let them know :) Note that this is Google's standard partner credit. All their partners offer this. It's not unique to GitLab.

> Zero marketing mentions regarding integrations with AWS

Not true. Amazon is also a GitLab partner, and We officially support EKS: https://about.gitlab.com/2018/06/06/eks-gitlab-integration/

> Sometimes it feels like a cheap Google adware.

This may be accurate. I think we got this wrong the 1st time out of the gate. Here's an issue where we improved the look and feel to be less 'adware' and more informative. https://gitlab.com/gitlab-org/gitlab-ce/issues/48804

> advertised themselves as a cloud agnostic

We are a cloud-agnostic company and we do advertise it. (e.g. even in our launch post for the GCP marketplace we make a point to say "you can install GitLab almost anywhere" https://about.gitlab.com/2018/07/18/install-gitlab-one-click...) Although, perhaps not enough. This is good feedback that we need to make this message more front-and-center.

GitLabber here. GitLab publicly states our goal is to IPO on Wednesday, November 18. 2020 and that we don't want to be acquired so we can preserve our values: https://about.gitlab.com/strategy/#goals

That is oddly specific

Reply on news.ycombinator.com