"While we aggressively rate-limit login attempts and passwords are stored properly, this incident has involved the use of nearly 40K unique IP addresses. These addresses were used to slowly brute force weak passwords or passwords used on multiple sites. We are working on additional rate-limiting measures to address this. "
I'm sure a similar situation would work here, especially with a decent geoip database. If you get 3 - 5 failed attempts on an account separated by more than a few miles (and not along an interstate :-)) you can probably assume someone is using a botnet to try to break in.
Before I switched my sshd ports a few weeks ago, I often saw break in attempts on my servers that would imply checking only a single username and a single password.
My guess on that is that bruteforcers, rather than testing thousands of usernames with thousand of passwords on a single host will try a single username and single password on thousands of hosts.
http://bsdly.blogspot.no/2013/10/the-hail-mary-cloud-and-les...
I double checked my log archives, it seems not to be exactly the same, though. In what is reported here, a same username will be tried with several password, allowing something like a minute to pass to make a new attempt.
In my logs, this is really just a single attempt. Other break in attempt may happen something like two days after, but with an other IP and username.
It may just be yet an other evolution of the same botnet knowing its previous pattern has been spotted, though.
(and in case you wonder, yes, I always have the nose in my logs, having a dedicated monitor for that :) )
- Of all users, only a small subset would have private repos
- Of those users, only a small subset would have private repos that would be of interest to third parties
- Of those users, only a small subset would have a weak enough password to allow brute force
To reverse it, of these accounts that were hacked, I can't see many of them having private repos that would be of interest.
And if so, then this would seem a bit excessive.
I know that's a lot of ifs, but it seems reasonable. I would be interested to see the number of total accounts vs. the number of accounts w/ private repos.
- That private repos are the only thing worth targeting. What if you could inject a trojan into a popular open source project? You could do a lot of damage that way, probably way more than on private repo, because so many people incorporate them in their products. Imagine they hacked the Rails repo, for example. Worse, some repos host binaries, for which a meddling would be harder to detect (a bad idea, but doesn't mean it doesn't happen).
- That the users being attacked are random and not specifically targeted based on who the user is and what the work on. Not sure if that's the case or not, but I see no reason to assume it.
A good bet to find people with private repositories, would be to aim for all or a subset of those who belong to an Github "organisation".
Likelihood of valuable user greatly increased.
Looking at the security history page I see a lot of failed login attempts. Makes me glad I enabled 2-factor-authentication!
If someone would slip in rogue code - it's quite likely some to many would actually run it and deploy it. Especially if it's a fast moving piece of software - like being so rapidly developed that distribution packages can't keep up for either time or stability reasons, leading people to compiling/deploying from source themselves.
It's a great way to distribute Open Source software. (Previously Mattt hosted it on a personal Amazon S3 account which he paid out of his own pocket; now bandwidth is generously paid for by Github)
Now that is dumb. I expected more from GitHub.
The way around this problem is to add source IP into the rate-limiting algorithm so that I cannot mess someone else up.
Unfortunately that can be worked-around somewhat by using 40k different source IP addresses.
No perfect answer. Personally I think GH did ok here.
That would prevent a practical joker at work from locking you out unless he went to a lot of extra trouble.
You attempt a login, they send an out-of-band request for confirmation. You do the confirm, and they let you in. Like a superfast streamlined version of the "reset password" email.
http://stackoverflow.com/questions/549/the-definitive-guide-...
TL;DR - compute the average number of system-wide failed password attempts, and if it's over the norm, impose small delay on all users (except those that login via a persistent login cookie).
Two-Factor Authentication is great but even without, it would seem that they could more aggressively lock out any new IP address which is failing password retries several times. (but continue allowing log-in from trusted IP's.)
There is a new SSID with WPA2-EAP replacing the captive portal and the usual DHCP/NAT business replacing our massive waste of IPv4 space, but that was implemented last year. This introduces the opposite problem: one person out of thousands can't remember their GitHub password so GitHub mysteriously drops all traffic from everyone on campus.
IP addresses don't identify people, computers, or usefully discrete units of any kind.
While it might be possible to do all this correctly, it's just so easy to do wrong that it's hardly worth it. Given that there are existing better ways to do it, it's the kind of thing you probably don't even want to mess with.
Of course, this works only if the number of attacks is small enough.
An even more secure approach is what Linode does. If you log in from a non-whitelisted IP, they require you to whitelist the IP via an emailed link, then you can login.
But that's overkill and the first one is a lot better for casual websites. You store the user's last login IP and you wait for maybe 1024 failed attempts not from that IP, which ought to be enough to suggest a bruteforce attack. You're protecting "bluedog4", not "password" or "1234" which are going to get cracked anyway.
At any rate, don't store really sensitive stuff on Github. It's a bad idea for many reasons, security flaws being one. In particular, keep in mind things like AWS server credentials which might go in your repository.
IMO something similar to SSH security policies like allowing logging in from a set of IPs and/or without password i.e. using public key, may be a good idea.
foobarjones:123456
foobarjones:passwd
foobarjones:test
foobarjones:123456
drsmith:123456
ilikeorangejuice:123456
...
foobarjones:passwd
drsmith:passwd
...The attacker probably tried a password like 'github' on the first pass and 'password' on the second pass.