Denial of Service Attacks
github.com
github.com
Google does this conversion automatically on their own outage pages.
show New York time 14:25 in your local time zone: ~$ date --date 'TZ="America/New_York" 2014-03-14 14:25'
show New York time 14:25 in Alberta time zone: ~$ TZ="Australia/Alberta" date --date 'TZ="America/New_York" 2014-03-14 14:25'
On a GNU/Linux system, for more timezone strings see ls -Ral /usr/share/zoneinfo/
You can get the local time with JS, and you can get UTC, and from those two you can narrow it down (you know the current UTC offset, but that's it), but you're not going to get a single answer.
Now that I think about it, you could auto-suggest based on offset and embed a list of locations per option if they needed to pick between daylight savings or not, for instance. There's room for an interesting widget or service, but it would need to be updated almost as much as a US sales tax calculation widget ;-)
I've always seen it refered to as AEST/AEDT, reducing the ambiguity.
[1] http://en.wikipedia.org/wiki/List_of_time_zone_abbreviations
Awww, that's so optimistic.
"At some point in their career, every programmer must pass through the fires of timezone hell."
(Esoteric software-engineering topics could make a wonderful video game, you know?)
It took a bit longer than I expected.
Along with the recommendation to never inventing your own cryptography, you should also never write your own date/time routines. Use well tested functions in your database or programming language libraries.
Anyway come feb 29th 2008 I get a call about some users were having issues. After not to long I figured out it was caused because of the leap year and his logic did not account for leap years. So I called them back and told them it would fix its self by tomorrow. I never actually bothered to fix the bug because a) I thought it was unlikely that the pos system would still be used in 4 years and b) the code was so bad in all areas that trying to fix anything had a good chance of breaking something else.
Lets just say that job was very character building
Much of the code assumed that all times were in PST and the commenter had apparently been incorporating data sources from other time zones into the code. I thought it was pretty funny.
> Greenwich Mean Time (GMT) originally referred to the mean solar time at the Royal Observatory in Greenwich, London, which later became adopted as a global time standard. It is for the most part the same as Coordinated Universal Time (UTC), and when this is viewed as a time zone, the name Greenwich Mean Time is especially used by bodies connected with the United Kingdom, such as the BBC World Service, the Royal Navy, the Met Office and others particularly in Arab countries, such as the Middle East Broadcasting Center and OSN. It is the term in common use in the United Kingdom and countries of the Commonwealth, including Australia, South Africa, India, Pakistan and Malaysia, and many other countries in the Eastern Hemisphere.
> Before the introduction of UTC on 1 January 1972, Greenwich Mean Time (also known as Zulu time) was the same as Universal Time (UT), a standard astronomical concept used in many technical fields.
> In the United Kingdom, GMT is the official time during winter; during summer British Summer Time (BST) is used. GMT is the same as Western European Time.
Why github I wonder? Perhaps it provides a challenging target. Perhaps github is used as a testing ground for a more profitable future attack.
We often get technical writeups after a DDoS attack, however we very rarely get a writeup sumising the motive behind the attack. I can't believe every attack is simply driven by 'because they can'.
Perhaps some developer was not going to make his deadline or wanted to take down the network to give him more time? Who knows...
Because HN is now full of people who already know the answer and who like to troll HN.
I thought submitting or referring to a good (possibly obscure or unknown) article would be more helpful in that instance, including to myself.
There are various possibilities, nevertheless, bringing down Github is surely a juicy objective.
What they found was that, at one forum, there was a convention where new, as-yet-untrusted sellers of DDoSing-as-a-Service are expected to take down some big, technically-respected target (e.g. GitHub) to prove their mettle, before anyone would hire them. And conveniently-enough, some new DDoSaaS seller was advertising right then, telling everyone to "look and see that _________ is down. That was me! Hire me now!"
It further turned out that the site-owner doxxed the account, found them to be a thirteen-year-old(!), and called their parents to tell them what their child was doing on the internet.
I mean, if you're into this, it's certainly fun to launch DOS attacks against large "evil" things such as government services, large corps and Micro$oft becoz w1ndoz sux0rz, but... Github? Why?
-"our agency has received intel that this site is a spawning ground for computer hackers" see: freenode ddos
Sort of like testing without getting massive amounts of repercussions.
That's just a (dumb) hypothesis. The cost of doing so is probably not worth it.
I don't use Git at work (SourceSafe ftw), but if I had to, I would use Github as a long-term backup storage, not as the nexus point of my dev env.
hard to recreate the logic tho O.o but it was something in the vein of "if i learn to set up git correctly, i shouldnt need a unique root, so what is it then.... a backup with vizualization? I dont need that for a private repo i can use offline tools"
Git is federated, as in Jabber. A branch has a place where it lives. If you do something with a particular branch, you need access to the place where it lives but not to anywhere else.
Monotone is distributed, as in Usenet. A branch may only exist on certain servers, but it is not logically tied to any one server (or set of servers). There is a global namespace for branches.
I think other systems (hg) tend to be more like the Git model, because permissions handling is far simpler.
Though when you think about it, taking out github is a very effective way to basically kill the productivity of many dev shops. At the place I'm working now, we are basically dead in the water while github is down (can't do deployments, can't merge pull requests, can't run automated testing, etc).
But laughter == good and I don't have to think about ethics and things, right!?
Malware, DOS, DDOS, etc nowadays is driven by simple and plain crime. Spy's stories are really the exception.
In fact, some software companies run Github on their private servers (https://enterprise.github.com/) that prevents them being affected from a DOS to Github's servers. Granted, these companies probably made the decision to use their private servers for other performance and security reasons as well, but avoiding DOS could very well be a reason.
Did this article contain anything particularly useful for anyone thinking about DDoS hardening? I didn't find anything. I guess it's not really supposed to be a technical article, just a smattering of buzzwords to let you know how hard they try.
The postmortem-half-apology has become quite an art form; as getting it right can actually draw a lot of positive publicity, and getting it wrong can be brutal. But I can definitely see how this post would feel like a pat on the back to whoever launched the attack.
It's actually a pretty bad position Github's being put in. They sit at the crossroads of playing defense against DDoS and trying to dispel or at least ameliorate any blame for the downtime.
My point was, if they have indeed become the internet's DDoS proving ground (as several others were speculating), then while you can see how much effort they're putting into these postmortems, I can see it becoming a vicious cycle.
Then the challenge is, how does Github placate their users without basically pinning a ribbon on the attacker? The funny thing is how the "best practice" checklist for a postmortem (say what happened, say how you thought you were safe, say how something unexpected broke your assumptions, apologize, say what you're doing differently in the future) basically ties their hands.
A pretty bad position for Github all around.
What isn't useful is the narcissism of your comment, and the assumption is that everything should be targeted at you and people like you.
"Commit locally github is down"
PITB, yes. But you can still get work done.
Plus you are forgetting that lot of automated jobs get triggered on github changes. Many shops kick off all kinds of tests, deployments, and other things based on changes to the github repo.
My point was simply that you can still be productive when github goes down, if you want to be.
I don't think you're "wrong" for the developer use case, but the reality is that it can bring large teams to a screeching halt.
Also, our deploy process is to do a `git pull` on the remote server, then run the build process, and finally to deploy the built stuff. When GitHub is down, we don't have a process for this.
I agree, both of these things could be avoided by having a different procedure in place, but that would obviate the need for GitHub altogether. Why use it when we can already share code and push it to our servers to be built? The point of GitHub is to provide a nice upstream that you can push to and pull from a la SVN because for most projects that model works really well. git provides all the niceties of local branching, rebasing, etc. while GitHub makes it easy to collaborate.
Having said that I'd love to hear how others handle this type of challenge.
Though when you try out a new attack vector, the community (hopefully) publishes enough details about the attack to help the next target more effectively deal with that particular attack.
If someone is attacking whitehouse.gov or a similar target, I somewhat understand their reasons why (though I don't agree with them). github.com, on the other hand, while a for-profit corporation, is also a valuable bit of Internet infrastructure that makes the world a better place. Attacking targets like github.com, Wikipedia, and others helps no one, and forwards no coherent political agenda.
So I'm going with the "for kicks" / jerkwad theory.
I deal with this crap all the time.
Attacking Github means you deny many more than only Github as a target. Which I guess, makes it more valuable than another target.
http://www.theregister.co.uk/2013/01/31/github_ssl_man_in_th...
https://en.greatfire.org/blog/2013/jan/github-blocked-china-...
On their spare time they take down botnets: http://www.prolexic.com/knowledge-center-ddos-vulnerability-...
http://arstechnica.com/security/2012/08/ddos-take-down-manua...
Shameless plug for hackmiami, if anyone is interested in learning how its done up and close they run frequent talks/meetups locally: http://hackmiami.org/
github.map.fastly.net.
199.27.76.133
If that's indeed what they're using, the only way to tell would be to look at BGP announcements when they're actually under attack.
"A simple Hubot command can reroute our traffic to their network which can handle terabits per second. They're able to absorb the attack, filter out the malicious traffic, and forward the legitimate traffic on to us for normal processing."
There are lots of articles on HN about DDoS attacks on various websites or online services. Most of the discussion is about the bandwidth used and the technical mechanics of the attack and defense.
This is interesting, but there's little discussion of the economic motivation.
I assume the kind of infrastructure used to launch this attack is not free. I understand people or groups might be using this as a way to further various political agendas or simply for bragging rights. I also understand DDoS attacks might be an extortion tool.
In the former case, wouldn't the attacker try to loudly and publicly claim responsibility? In the latter case, wouldn't the defenders take pride in their "we don't negotiate with extortionists" stance while they're in disclosure mode?
Or maybe this is just some rich guy's private hobby, and he does it for the amusement he gets out of reading about people's reactions when they can't figure out who's responsible?
It seems like the set of rich guys who have the technical skills to do this kind of thing without getting caught would be kinda small. And if they hire people, the bigger their organization gets, the likelier they'll hire a law enforcement plant -- or simply someone with a conscience -- and the game will be up.
Organized crime might be a possibility, but I assume those guys are interested in making money, not just committing crimes and wreaking havoc. So what's the business model that motivates these attacks? If it's extortion, why do the targets feel comfortable revealing the attack, but uncomfortable revealing they're being squeezed for money?
That's kind of awesome
http://www.prolexic.com/why-prolexic-best-dos-and-ddos-scrub...
If an attack was launched only that whitelist would be allowed until the attack was mitigated.
So while certain legitimate traffic would be blocked for sure, people who connect through fixed ip addresses that were whitelisted would get through and be able to do what they needed to do.
Thoughts?
For a website of GitHub's scale, I don't think it would be very effective, though maybe it could be helpful in combination with other measures.
"Just have your botnet do a few regular old requests to the network a few days before launching."
Not talking about "whitelist sites that have made access in the last x days".
For example on HN it would be easy to create a white list. They do it now recognizing new people who signed up and keeping track of activity as well (by points).
You could either have people identify the ip address that they accessed from and further limit the whitelist to a certain period of time and activity additionally.
The idea is not to be 100% perfect but enough so that if you are a regular user of github from an IP address at your office (as opposed to wifi cafe) you will be able to get through.
This is, by the way, how registries limit access to their system. It's all whitelist you have to pre identify the ip addresses that you will access the system from.
The whitelist only comes into play when under attack. And for sure yes if you are connecting from a new place you will be blocked. But others will not be blocked and there will be some access for some people.
It will let a small percentage of non-whitelisted IPs in, but would filter the majority of them out.
Comcast is really in a class of it's own regarding one-sided peering policies, but the other providers like Cox for example are fairly easy to peer with.
BCP 38/RFC 2827 would change the DoS game, but it's been a best practice for longer than most of this audience has been alive and nobody yet gives a shit and/or they are too lazy to automate the implementation. So operators waste their lives cleaning up after bad actor ASNs that they can't even identify. I shouldn't be mitigating 65 Gbps destined for a controversial customer, the attacker should be removed from the Internet before I even notice.
You can tell from my tone that attacks are part of life for me. I'd venture that denials are the second largest problem facing the Internet today, behind the organizational structure of critical systems like DNS and ahead of spam and surveillance. However, there is now a sizable DoS prevention industry so I wouldn't be surprised if BCP 38 drifts into even more obscurity, but that's the cynic typing.
It's not in my interest to "Citizen's Arrest" someone with a pwnt node.
Really? You have to round-trip through Campfire to control your network?
If they're all in Campfire anyway, there's no overhead here.
1. It's scripted so you don't have to think about it at 3am.
2. The rest of the team can see it happening in realtime so you don't have to explain what you're doing via a side channel. They can see it happening.
3. It doesn't require specialized knowledge of routing to enable it. If the on-call engineer sees an attack and calls someone for guidance, it's super easy to tell them "type /mitigation enable" for instance.
4. Of course we can run the exact same script or login to our routers and manually change our BGP announcements if we need to.
DDoSing a government site I can understand, sure. (Aaaand now I'm on a list.)
Dear github dudes, netflow is your friend.
A decimal order of magnitude is a factor of 10. And would likely represent a problem.
Really not hard, you don't use the term 'order of magnitude' in binary. Instead opting for 'bit shift', or doubling.
I think avoiding the temptation to false precision is more important. Inconsistent units and the retention of insignificant digits in order to make numbers look bigger (or smaller) drive me up the wall, though.
Not that close, if you ask me.
If you mean something between 5 and 15, you are using base 10. *16 are 4 orders of magnitude in binary, but 1 in decimal.
That said, everybody asking that same question, please, do not use binary orders of magnitude. Our language suffers every time somebody does that.
I'd be surprised to hear someone using binary to talk about bandwidth... Or pretty much anything else.