Bitcoin exchange hacked via Rails exploit, funds stolen
bitcointalk.org
bitcointalk.org
First and most importantly, Airbnb and Uber are not disrupting industries burdened primarily by consumer safety regulations; they are disrupting industries burdened primarily by barriers to entrance that are designed to direct economic rents to politically favored actors. Huge difference.
There is no plausible 'consumer protection' story for preventing licensed livery cab drivers from picking up curb hails, whether on the iphone or otherwise. The law is there to protect the incomes of people who buy cab licences.
There is no plausible 'consumer protection' story that would explain why building codes for permanent residence are not good enough for temporary residence as well. The law is there to protect hotel operators from vacation rental competition.
So let's not compare Bitcoin with Uber and Airbnb, because they are completely different animals.
Bank security regulations, OTOH, ARE designed to protect consumers, although I'd argue that they're mostly unnecessary in practice. Legitimate banks don't get hacked because there are billions of dollars at stake for the banking institution, and their business literally depends on their ability to secure payments. The incentives are there with or without bank regulations.
Bitcoin sites, on the other hand, regularly get hacked because they are fly-by-night operations written by idiots who are probably also trying to steal from you. It's the fake-money equivalent of using www.send-monie-through-me.co.in and then being surprised when you get ripped off.
Bottom line: there is nothing wrong with regulation designed to protect consumers from actual threats. There is everything wrong with regulation designed to protect business from competition. The latter is what needs to be 'disrupted'.
Additionally, there are a lot of tourists that visit SF. Those two demands for places to sleep, from tourists and residents, are at odds. The price you can charge for a hotel room in SF is substantially greater than what you can charge on a nightly basis in stable rent.
So what we're seeing happen in SF is people who have rental units that would normally be on the rental market are now pulling those off the rental market and putting them on the AirBnB market, which serves tourists instead of residents. Because frankly, why wouldn't you? If you can rent your place for $200/night to tourists you can make a hell of a lot more money than you can renting to someone for a year.
The hotel regulations aren't only to help hotel consumers, they're to help the renters too. Now, you could argue the natural market economics should just play out and the city should allow as many hotels to exist as the market will support. But that's not a city I want to live in. I value prioritizing housing for residents instead of tourists.
I remember someone from Vancouver saying the same thing about that city in a previous HN thread on this topic.
Earthquakes are just an excuse used by NIMBYs who want to preserve their oh so precious "bay view".
So long as the foundation is bedrock, there is nothing preventing the building of highrises in San Francisco. We have the technology.
I talked about it more here http://news.ycombinator.com/item?id=4815087 http://news.ycombinator.com/item?id=4815247 http://news.ycombinator.com/item?id=4815537
San Francisco might be "ruined" by your definition, but how is it any of your right to tell people what they can and can not build on their land?
Zoning is central planning at it's worst.
Sure cramming people in like sardines makes it easier to make things efficient - but it's very realistic to make less dense populations efficient/sustainable too. Urban developement isn't some kind of ecological optimization problem. There are factors you are completely dismissing, like overall happiness, contribution to the community and the nation as a whole, cultural value generated etc.
> but how is it any of your right to tell people what they can and can not build on their land?
Are you serious? There is the whole idea of community and sustainability. If a community deems a certain construction project detrimental to the overall health and wellbeing of its members then they can stop projects. If I don't want to live next to a highrise and the accompanying noise, traffic, pollution, I have a say in what my neighbor can build.
I'm not personally telling people what to do (because I have no authority). I'm engaging in a public debate over the SF community's values and priorities.
Which is not created by suburban sprawl, either. You seem to (without support) indicate that this is not possible with denser communities.
That is simple egoism, don't coat it in nice language.
Non sequitur.
This is not a very thoughtful comment. There are obvious reasons why properties zones for permanent residence aren't appropriate for transient residence; the latter type of occupancy is accompanied by crime and abuse.
You seem to be falling into the trap of considering "consumer protection" only from the perspective of the tenant.
There is a reason hotels require "special use" zoning exceptions in cities, and it's not because Mariott and Hyatt have captured the city council; it's often the residents who create uproars when those exceptions are granted.
You are talking about renting apartments to unchecked strangers for days and I am talking about people renting out their apartment on AirBnB. There's some relationship between the two sets but I don't think they're equivalent.
Regardless, is there evidence for the claim that crime and abuse of non-tenants is higher for either the set you're talking about or the set I'm talking about than in the general population?
I live in a high rise that also contains a hotel, but with different lobbies and elevators, and that doesn't bother me. I think this is because a hotel has a management staff that maintains common area decorum and holds occupants responsible for their actions during their stay. An apartment rented on airbnb has no similar oversight or responsibility for common facilities.
[http://delmar.typepad.com/brianbrady/2011/06/owner-occupancy...]
I imagine the rules about what safety equipment a small ocean-going yacht and an ocean liner must have are pretty different too, for similar reasons.
When you say "There is no plausible 'consumer protection' story for preventing licensed (sic) livery cab drivers from picking up curb hails" (I think you meant unlicensed) it's easy to dispute that point.
Here is the first hit on Google for "cab rider ripoff": http://www.nypost.com/p/news/local/taxis_taking_wBtAr13EzaKS... - "At least a dozen hacks have been caught hitting unsuspecting passengers with pricey tolls for bridges and tunnels that the cab never actually crossed".
And the result: "We are confirming these data, and if appropriate, will likewise seek to revoke their licenses," Yassky said. "We will continue to comb the GPS data for any similar incidences."
That type of legal solution is not possible if taxi drivers are unlicensed.
And the well-publicized instances of Airbnb problems (e.g. prostitution) are already demonstrating that at least some of the regulations are in fact necessary.
Finally, the point about consumer bank regulation is, if your bank account does get ripped off, you're insured to $250K by the FDIC, which BitCoin doesn't have. Though perhaps that is a market opportunity for an aspiring YC company?
In any event, you're overstating the case to make your point -- keep the language reasonable if you expect to make your point.
The original comment was very specifically worded to only cover licensed livery drivers, not random unlicensed drivers.
As an apartment owner in a multi-unit apartment building, I don't want the neighboring apartments being used as short-term rental properties - and the building regulations forbid it. It's absolutely a quality-of-living and consumer-protection issue protecting property owners from the risks associated with transients.
For instance it gives property owners in a residential neighborhood an assurance that a neighboring building can't decide to convert their rooftop to a nightclub potentially disturbing the neighboring buildings with noise, foot-traffic, car traffic, drunks, trash, fights, etc. If you gave individual landlords the right to make these decisions without wider oversight, you'd run into a lot more issues like these. Sure - today they can petition to override existing zoning regulations and that sometimes happens, but residential issues are larger than just an individual apartment or building.
Maybe it's just that I get peeved when people get all "landed gentry" and start thinking that somehow because they are privileged to have been able to buy a piece of property that they have a moral right to control the lives of their neighbors.
That said, I support regulations "within reason" and I suppose it all comes down to a quantitative difference in where we draw that line.
You can make an argument about not restricting the free market, but shelter is such a basic human need that I think it merits a healthy amount of regulation.
I think Uber and Airbnb stand on different ground here. The building codes issue is a red herring; Airbnb faces more severe problems. I can think of plenty of "traveler protection", "hotel protection", "tenant protection" and "landlord protection" stories. Real stories detailing Airbnb's failure to answer these issues are already circulating the internet:
http://www.google.com/search?q=airbnb+nightmare
(to be fair, these stories seem to focus exclusively on bad tenants, while I can see bad landlords being an issue as well)
Uber only uses licensed sedan drivers. They are already subject to safety regulation, but unlike taxi cabs they are also checked by reviews from passengers. The most frightening experience I ever had in a vehicle was in a taxi cab taking me from the airport in San Antonio to my hotel. He was exceeding 90 mph, and driving recklessly, ignoring the turn signals of other drivers on the interstate. Where was your touted regulation then?
Not surprising, considering that Uber currently carries a tiny fraction of the traffic that the cab companies do. As the company scales, it's not hard to imagine that they will need to bear more responsibility for background checks on their drivers, and assume liability for damages those drivers may cause. Full-time drivers will begin battling one another for preferred territory, and Uber will have to mediate the conflicts. In short, Uber will start looking more and more like a traditional cab business, and less and less like a peer-to-peer matchmaking service.
Agree with you on your other points though.
"I was told confidentially by an IT Security specialist from a major bank that the public would be shocked if they knew the amounts that are stolen daily from online financial institutions via ACH and wire fraud. Typically, breaches and total numbers are not revealed because they don’t want to advertise a weakness and they certainly don’t want to alarm customers. Some of the more vicious attacks are State sponsored. I believe him. That’s what bitcoin is up against as it progresses into the mainstream. The leading security experts are in that world, already protecting against the barbarians at the gate. They are not in the bitcoin world."
That said, I don't think it's an impossible task (is anything?), and I'm sure some day there will be a large disclosure through some means, internally-assisted or otherwise.
Modifying or updating anything on the server required physical access.
That server was located in a secure vault.
Your home is your home, so if you die in a fire it's your look out.
But a hotel, or a temporary residence, is not your home, and if you pay money to someone to provide a service they should meet minimum standards for safety.
This is a good thing. It allows small businesses to compete but without using "safety" as an area which can be cut.
But I will address this one major factual error: Legitimate banks don't get hacked because there are billions of dollars at stake for the banking institution, and their business literally depends on their ability to secure payments.
Banks are not in the business of securing payments; that is what payment processors like Mastercard and Visa do. Banks are in the business of investing money which has been deposited with them. As a result of (state and federal) legislation, banks are liable for making depositors whole in the event of theft, so they are motivated to invest significant sums in security.
This is the same effect that has made the anti-vaccine movement popular.
Also, this incident has nothing to do with regulating bitcoins. It has to do with Rails and this particular exchange site, whose reputation is now damaged and who's going to lose business. Note how free market works great in this case: the organization costs people their lost money and will most likely go out of business. Unlike big banks.
Why so? Even banking websites build on frameworks and if you'd have chosen Spring for example, there was a Remote Code Execution vulnerability in 2010. And even if you roll your own framework, you're just as likely to introduce a critical flaw. The Dutch governmental DigiD service runs rails [1]. The critical difference between the BC service and a bank or the government is that a responsible party would have secured their app immediately. The DigiD service was taken down pretty quickly and stayed down until patched. There were multiple workarounds that did not involve major patches and even if you didn't know which of your apps was vulnerable, you could filter the payload at your load-balancers if you had some [2].
[1] http://lwn.net/Articles/532224/ [2] An xml tag with the type "yaml" was required to trigger this. It's a pretty specific payload that is very unlikely to be used in a regular request.
This is already. You can legally trade things for other things. There is no law saying that you have to use Euro (or your local government issued money)( for everything.
The only think you have to use it for, is taxes or paying fines.
"Along with the power to coin money, Congress has the concurrent power to restrain the circulation of money which is not issued under its own authority in order to protect and preserve the constitutional currency for the benefit of all citizens of the nation."
There are actually some voices saying that Bitcoin needs a taxation protocol.
That sounds horrible. If a bank gets hacked and "loses" my money, they owe me that money. Federal and state law requires them to put that money back into my bank account, at the bank's expense. (Note, this is not the same as FDIC insurance, which applies in the event of a bank failure.) The free market still applies: on top of getting their money back, customers can take their money to more secure banks.
Make it legal to receive whatever I want to receive as a payment. Let businesses regulate the currency market and determine what currency is reliable. Oh wait, except that then government cannot tax you, of course.
You can receive whatever you want to receive as payment; this has been a basic principle of English-based law for hundreds of years. The currency requirement is merely that any debt obligation must be satisfiable through the use of currency equivalent to the value of the debt. Also note that the government reserves the right to tax you regardless of the currency you use. This has been basic law in some form or the other for hundreds of years, and is explicitly stated in I.R.C. section 61.
Unless the bank goes bankrupt. Basically, if the bank plays fast and loose with customers' money the customers shoulder the risks whilst the bank owners get the rewards - and there's no way customers can tell whether this is happening, since they neither have access to the bank's internal records and systems nor the skills and resources to make sense of them.
Actually, the only reason the baks have to return the money in the first place is because of Government intervention, and even that's not enough. Unfortunately, thanks to binding arbitration the US has a free market of sorts in dispute resolution, and the banks and financial providers have so much more market power than consumers that they can effectively pressure arbitrators into siding with them. If they don't, the bank won't do business with them and they can't find work, whereas most consumers only need to use arbitration a few times in their lifetime at most.
Banks pay for FDIC insurance coverage as part of their capital requirements for being a bank.
Tip: This is happening. This is how banks have and will always make money.
It is the role of governments to regulate to what extent the bank can use your money and for what purposes in order to minimize customer risk.
I just don't buy into the abrogation of common sense argument for regulation. If you let random people stay at your apartment, there's a chance they'll destroy it. If you're really worried about that then buy renter's insurance or don't sublet your place! If you give your bitcoins to some random website, there's a chance it'll get hacked! It's not like those people didn't have option of putting their money someplace more secure, like a bank.
I think regulation in general is really important for making our society a decent place to live, but when it makes reasonable activities effectively illegal that is generally a sign that it has passed that point and started making things worse.
I'm not sure about Uber, but this is most definitely not true of AirBnb, which has come under fire from many neighborhood associations. It's not just your quality of life that gets reduced when you rent your house to bad apples, it's your neighbors' as well.
Atlantic, Wired, the NY Times, Chicago Tribune, and the L.A. Times have variously run horror stories for renters who made the mistake of using AirBNB to book rooms (see, e.g., Toshi hotels and their variants). The whole point of hotel regulations is to protect the guests, not the hotelier.
> The whole point of hotel regulations is to protect the guests, not the hotelier.
I wish this were true.
Perhaps it's not vital that a hotel(or taxi) is licensed today, because we can easily see its realtime feedback from previous users. But then again, perhaps those users don't notice that there is no emergency lighting and the fire alarm is disabled, or that the driver has multiple convictions for dishonesty.
This Tetris-like complexity-collapse model is very common in biological evolution.
When it works well, the new simpler system will still solve the problems that the old complex system solved. It will just solve them more elegantly.
Regulators and incumbents need competition. No new product is ever better than an existing product in ALL respects, only in some features. Hitting the features that existing regulations are meant to ensure might not be #1 on the feature roadmap, but it's on there.
The problem is when features that are less important to customers are prioritized by regulation (and therefore by guns) over features that are incredibly important to customers. Clearly, the existing taxi regulations did not incentivize rather important features like "convenient for taxi customers" but instead were about edge cases that are important, but only at scale.
I agree that companies need to take a look at how their industry is regulated and what purpose those regulations serve. But the fact that companies can come into these types of industries, openly skirt the regulations, and still be massively successful shows that the existing laws aren't meeting the needs of the people who use these services.
And how exactly has government solved "the problems inherent in the peer-to-peer model"? I'm not sure what problems you're talking about in the first place.
That's the point. :-) A number of them have been enumerated above: protecting banking customers from loss in the event of theft; regulating the location and safety of hotels; providing some means of recourse against a dishonest cabbie.
Rather than more regulation, perhaps more transparency is a better solution. 3rd party certification would do a much better job at security than a government regulation. And have much less abuse and overhead.
Essentially the same thing has happened here, but in the digital realm. A thief (or thieves) broke into the backend systems of an exchange which was holding bitcoins for its customers, and the thief stole all their bitcoins. The exchange and/or its customers have suffered losses, but every single bitcoin stolen by the thief will continue to be as valuable as any other bitcoin. In other words, Bitcoin will continue to be the same exact commodity -- its integrity has NOT been compromised.
The moral of this story: if you own bitcoins, make sure they are stored in a truly secure system. Many Bitcoin exchanges claim to be -- but really aren't -- truly secure!
Shesh. A Bitcoin has no inherent value. If such incidents become common enough, nobody will be willing to buy bitcoins for dollars or accept bitcoins as payments for goods,which means that the thieves will sit on a bunch of useless bits.
No, that would only be true if people were forced to store their bitcoins on vulnerable bitcoin exchanges, which they're not.
Thus, on this basis, like all useable physical goods, gold has some inherent value as a commodity.
1. Bitcoins can't be counterfeited. If you receive them with even just a few confirmations on the blockchain, it's probably going to be there forever.
2. You can confirm the legitimacy of Bitcoins, en masse, without any cost.
3. You can store bitcoins without any cost.
4. Bitcoins are much easier to transfer to other people.
5. You can form contracts and advanced redemption conditions for transferred coins.
If Bitcoin had no "inherent value" then it should be worthless. Maybe Mises Regression Theorem is wrong?
Bitcoin can be worthless and yet hard to counterfeit; I can sign a random number right now and it's worth nothing. I can Tweet that number or create a permanent record in any number of other ways.
Your second point is the same as your first; it again presumes that there is some value to bitcoin.
I can store a lot of things without meaningful cost. That doesn't make them valuable. My application server with seven gigs of dev logs isn't a treasure trove either, despite the ease with which I managed the data relative to bars of gold.
Bitcoins are easier to transfer than gold coins. That is true. But spent facial tissue is also easier to transfer than gold.
You can form contracts with all manner of instruments, from cows to gravel to pork bellies. Every commodities trader in Chicago has a story about a friend of a friend who ended up with a garage full of pork when they failed to sell a futures contract in time.
Do you really mean to say gold's rust-resistant properties are to blame for it's exceedingly high price on the market?
Fact: In 2013, Bitcoin is the world's easiest way to send a standardized indicator of wealth from one person to another. Forget anonymity, forget open source, forget deflation, forget amateurs building Ruby on Rails websites poorly!
Bitcoin's #1 benefit is the ousting of intermediaries. Because Bitcoin transactions can occur directly between two people without any third party involvement, the platform upon which people trade Bitcoin for goods and services becomes irrelevant. With Bitcoin, the platform only matters to the extent it aids in the market discovery process.
I for one see inherent value in a currency I have control over 24/7, that I can literally park in my brain never to touch the public Internet, carry with me everywhere I go, send to anyone, anywhere at any time, instantly and without an intermediary. If you see no value in that, honestly, what is there to say. Good luck with your financial goals in 2013.
I don't know the answer, but I can tell you it's miniscule.
Then how do you know what storage locations are truly secure? If you have to be a computer security expert to safely use bitcoins, then that will surely decrease the demand for the currency and therefore reduce its value.
It was only a day or so from when a patch was available to when a public exploit was everywhere. Maybe they were on vacation, maybe the site was built by contractors who have since moved on, maybe they don't read hackernews or other programming news sources. I expect to see similar stories in the coming weeks.
Having said all that, if our upgrade holdup had taken any longer than it did, we would have implemented one of the mitigation strategies.
I don't know that that's what you're saying you did but we need to be glacier-blue-ice-clear about this. Nobody gets to wait on bugs like this. You patch or workaround immediately or, most probably, you shut your app down.
EDIT: I guess that qualifies as a mitigation strategy, but when I said that, I was talking more along the lines of the patches, or like another person I know, even more dramatic steps like forking Rails. There are regressions in the 3.2.x updates since 3.2.9 that affect some sites.
Bottom line is that there was a lot of bad timing here that sucked up a lot of time in securing a Rails site.
I wanted to be careful not to point a finger at you; it's just that this is exactly the kind of crazy mistake I can see a web startup making.
Obviously, the timing sucked, but nobody had any control over that.
https://gist.github.com/4512579
http://dev.metasploit.com/redmine/projects/framework/reposit...
You also cannot leave 6 years of vulnerable Rails-versions up on rubygems.org without even backporting your patch (yes, they're still up there now).
It makes no sense to blame the users of a web-framework, most of which are not security experts, for not responding instantly with the perfectly correct procedure to an event of this magnitude.
Put the blame where it belongs, on the idiots who "handled" the issue by releasing a "recipe to exploit"-Advisory. The same idiots who ramble about "not breaking old apps" when asked why the vulnerable gems are still up on rubygems.org and continue to be quietly installed by any Gemfile referencing them...
I don't know why people weren't taking a hard look at the params processing code path in Rails for 6 years, but they clearly weren't. The SQLI bug from last week pointed people at that code, it got its first real shake, and that's all there is to say about it.
A bit more creativity should be allowed when handling a vulnerability of this magnitude.
For example, publish a patch that escapes all input in curious ways, presumably to prevent SQL injection. Pad it, obfuscate the actual fix with code-noise, make it annoying to read. Then release it as some handwavy, semi-plausible "follow-up" to the previous SQL-injection, urging everyone to upgrade in small caps.
Yes, a handful of blackhats will see through the bluff. However, it will likely be the exact same blackhats that already had discovered the issue after the initial SQL injection advisory.
This would have given the Rails-community a head-start, rather than unleashing every script-kiddy on the planet at once.
What needed to happen is what happened: a patch and a series of workarounds were produced as soon as possible.
Nobody got to choreograph this one. The control you think the Rails team had over this, they did not have.
I actually remember this incident (albeit blurredly, hell, was that really 10 years ago...).
I dug up a few snippets[1] and I think the main sources of the animosities back then were certain regressions caused by privsep (not a hassle-free change) and a bit of ego-clashing (Alan Cox, Theo).
Your summary of the events may be about right, but is "not wanting to dance to someone else's tune" really a good argument against an attempt at responsible disclosure?
[1] http://www.baylisa.org/pipermail/baylisa/2002-June.txt (at the bottom)
And then downplaying the severity by not encouraging developers to take the follow-up patch seriously?
That's a flat out terrible idea.
That seems like something that would ruin any future credibility of the project. No further security vulnerabilities could be trusted as being accurate (and any further patches would just be begging for additional scrutiny).
Also there is no "downplaying" in declaring any kind of problem as a remote SQL injection vulnerability. That is still obviously urgent enough for everyone to patch immediately. Yet it doesn't attract blackhats in the same way as blarting "remote code injection".
There were people on this very site who were commenting about whether they should concern themselves with this patch (initially because people erroneously attributed the vuln to SQL-I, and then later because they "weren't using XML anyway")
Those kinds of things happen when you don't clearly describe what a vulnerability is, and when you try to mask how big of a deal something is.
This was a huge vulnerability. It was critically important that everyone running a Rails app fix it immediately. Shouting that from the rooftops was absolutely the right approach. Cloak and Dagger bullshit to try and hide that is unequivocally a bad idea.
For me this very article (bitcoin exchange being hacked) demonstrates that despite the harsh wording, the advisory didn't reach everyone in time. Not even those who should really care (like bitcoin exchanges).
I still think the explicit disclosure did more harm than good, by drawing maximum attention from the blackhat-community without really improving the reach amongst the oblivious. I still think a "staged" disclosure might have worked better, even at the risk of the timeline being short-circuited by a malicious party spilling the beans early.
However, there's little point bemoaning this particular baby or bathwater much more now.
I'm more interested in the steps that the Rails-team will be taking to lessen the blow by future security incidents. As I've said in another thread I'd be in favor of an optional (opt-in) kill-switch, to be triggered only in drastic cases like this one. Perhaps that is a point where we can agree again - otherwise we'll have to part in disagreement. :)
I disagree with Moe that labeling the latest bug as an SQL-I instead of RCE is a good strategy to ward off blackhats.
I'm all in favor of giving people more time to upgrade, but this issue was very unusual. Multiple groups were all finding it at the same time and who knows when the first "full disclosure is always the right answer" group would have started singing? Like it or not, full-disclosure people exist in the world, and responsible-disclosure people need to be aware of their existence.
If just one person/group had found this, RoR could have done a better "get ready to upgrade next Tuesday at 6am" followed by a "here is the patch, details to follow tomorrow" on Tuesday at 6am.
Nothing about this situation was good, but hiding it would have only made it worse. When you're serving shit sandwiches for lunch, it's best to let everyone know that's what's on the menu. End of story.
You pull off the plug ASAP.
Better have customers unable to use the service for one day than have the customers lose their money while you stumble trying to figure out how to patch it. This is elementary for any mission critical system. Just imagine your neighborhood nuclear plant delaying the insertion of the control rods during a meltdown because it has to wait for some shinier parts to arrive.
The fact remains though, that this exploit was _so_ severe, that, depending on how attractive a target you were and how disastrous it would be for you to be compromised (financial services? High on both scales) -- it would have been a better choice to _take your app down_ until you can fix it, rather than leave it up with the vulnerability. The vulnerability was "an attacker can run whatever code they want on your server, and they can discover that you are vulnerable by cheap automated port scan." It's as bad as it gets.
But if it was somewhat important, kill the application servers until you can be sure to have upgraded properly.
When systems are designed with security in mind, rather than simply throwing together a web/sql application, people put a lot of time and effort into constructing barriers to protect the integrity of their data.
Take for instance the CACert certificate authority. They designed their system in a way where the master key that signs certificates is stored in a computer that isn't connected to any network. The actual servers then talk to this computer over the serial port using a carefully crafted API when they actually want a certificate signed. This means signing certificates is slow and the key is inaccessible. So if all their servers got remotely compromised, a hacker would never be able to get the key and at best would probably be able to sign a short list of certificates before being detected.
Other people have suggested that this black box should store public/private key pairs generated from the user's password for each user on the exchange. So when a user signs up for an account on the exchange, Javascript code generates a private key from the user's password, client side. The corresponding public key is sent and stored in the offline transaction signing box. Whenever the user wants to initiate a withdrawal, the transaction signing box creates a random number that needs to the signed with the private key that corresponds to the public key it has on store. This way an attacker need to compromise a server and install an eaves dropping application that replaces new users' (or existing users changing their password) real public keys with its own. Just breaking into the server wouldn't do the attacker any good at first.
There is really no excuse for exchanges not using cold storage.
It's like a CA that creates the certificates in PHP right there.
You need the seperation and you need to closely monitor and control the transactions requested from the web interface to detect any fraud or misuse.
I can dream up architectures which limit manipulations, require the user to constantly type in passwords, etc. A company not security conscious enough to update is unlikely to have done that. But suppose they did, what happens? EVEN THEN you can turn the website into the digital equivalent of an ATM skimmer, and steal money. That is, of course, assuming that I have not managed to turn shell access into some more direct compromise of your whole network.
Here is the moral. If you're directly handling money or a money equivalent on your website, and someone has shell access, they will be able to steal from you.
That said, if the backend is also vulnerable to the same or another exploit, that's not going to buy you much.
It's a forum post without any details - your question is raised, but has (so far) not been answered.
Because these exchanges focus more on promoting the "world changing" ideology than they do on taking their users' money seriously.
This was no undocumented zero day hack.
While a front end compromise is always going to be bad, splitting the two gives you more options and more ability to spot when the front end is acting unusually.
What happened to a bank's customer's funds if it got robbed prior to 1933?
I met an ex bank robber once. He and his "gang" attempted to rob a branch that was near a factory. The factory employed a number of immigrants who were fond of cashing their entire paycheques on pay day, so it would bring in a lot of extra cash every pay day. They timed their robbery for when the payroll money was present.
Unfortunately for them, the scheme had been rumbled, so the payroll delivery was fake and the police were waiting in plain clothes for them. He was shot in the neck but survived and learned to play bridge in prison, and in the fullness of time was released, where I met him and heard his story.
Any ways... It seems quite possible to me that a bank could be robbed but remain solvent.
The nearly immortal protagonist often ends up being the banker in a small frontier towns. He routinely has to deal with mobs of people who attempt to "nationalize the bank" and are dumbfounded to discover that the bank does have all of the money in a safe.
Example: http://www.centralbank.ie/mpolbo/mpo/pages/reserve.aspx
Banks are typically capitalized by a variety of assets, of which cash is typically a small amount.
In addition, the bank typically has its cash holdings distributed at a number of different branches.
Your account at the bank is technically it's liability. Essentially, you've loaned your money to the bank with an option to redeem it at any point in time (demand-deposit account, or checking account) or up to some period of time after asking for it (time-deposit account, or savings account).
So, let's say a bank's branch is robbed of $10,000. What happens? The banks shareholder lose $10,000.
Let's say all the cash and physical assets at a branch is robbed. The bank will typically cover those assets with assets from other branches. The bank's shareholders swallow the losses. Sometimes the bank will sell additional shares, or some of its remaining real assets to help provide it with more liquidity so it can continue to cover funds requests by account holders.
Let's say a bank has all its assets in cash in a single branch location and that location is robbed, meaning the bank now has no assets left. That bank is now bankrupt. It goes to bankruptcy court and you are a creditor. Once the banks remaining assets are sold, you may get a few pennies.
This is a major failing of the existing bitcoin repositories. They aren't banks and so aren't regulated as such. They're just storage lockers where you hide your cash under a mattress. In the event of a theft or fraud... oh well! It'd be interesting to see someone get a bank charter and backup those bitcoin deposits with real capital.
Governments can and do offer an insurance system whereby they will ensure that customers receive a substantial part of their monies back - this aids against bank runs (where customers panic and all try to withdraw their money as cash and so cause the bank to implode).
http://www.fbi.gov/stats-services/publications/bank-crime-st...
http://www.allgov.com/news/unusual-news/robbing-banks-is-not...
"Using confidential data on the value of every bank heist in the U.K. from 2005 to 2008, the trio came up with an economic model of bank robbery.
They found that the average revenue from a British bank robbery in 2005-2008 was only $31,600, although excluding the one-third of robberies that came up dry boosts the average to $46,600. But those proceeds have to be divided among the gang, and while extra gang members raise the average take, the haul per person decreases.
The average take per person per successful job was $19,792, equivalent to less than six months’ average wage in the UK, although being armed increased revenues substantially. While that may not sound too bad, multiple jobs greatly increase the risk of arrest and incarceration, as 20% of heists ended that way.
Data from the FBI paint a similar picture. In 2011, there were 5,086 bank robberies in the U.S., generating $38,343,501.96 in revenues for the perpetrators, or an average of $7,539 per heist. Excluding the robberies where nothing was taken increases the average only to $8,457. Not only is the return low, the risk is high: out of 13 people killed during bank robberies in 2011, 10 were robbers."
If only people would understand this about school shootings and the like
"Number of incidents in which deaths occurred: 3"
all 3 were the perpetrators rather than employees / customers. On the other hand 2 hostages being the employee's family members is disappointing. It's really sad that people really do involve the families of targets just for a bank robbery.
Seriously, I don't see any comparison between this and a bank robbery. It's not like banks in 1933 were only just figuring out that they should lock the door where they stored the money, or that they should repair any holes in the wall, or even that paper walls, though they are quick to build, have a very bad record on security.
What are you talking about? People are still famous for being bank robbers, e.g. http://en.wikipedia.org/wiki/List_of_bank_robbers_and_robber... . FDIC insurance has nothing to do with robberies, it insures against banks going bankrupt.
In most cases, even a large robbery wouldn't ruin a bank: They could borrow against or sell their mortgages to another bank for cash to pay depositors.
This is different than a safety deposit box: The fine print is that you are responsible for its contents and if the bank is robbed, they will not replace the contents or reimburse you.
So, if you put your life savings into gold and put the gold in a safety deposit box, you lose. If you put your life savings in a deposit account, you will be reimbursed unless the bank fails outright.
(Prior to the creation of the FDIC in the US or deposit insurance in Canada).
Edit: See the response from raganwald
"Before the wild speculations beginn, the service will be recovered and we pay the losses out of our own pockets."
from this summary: https://bitcointalk.org/index.php?topic=135926.msg1447905
"Before the wild speculations beginn, the service will be recovered and we pay the losses out of our own pockets."
If they can, of course. But still, a very nice gesture.
Then worse, you have tons of internet criminals coming up with clever strategies to part people with their Bitcoin. Will these guys reimburse their users or end up giving an excuse and disappearing into the ether? Only time will tell.
That said, Bitcoin itself remains strong and probably one of technologies with the biggest potential in quite a long time. These issues will continue to occur and spawn drama for the foreseeable future. At some point the good Bitcoin news will drown out the bad Bitcoin business news. But that's not going to happen anytime soon. So let's get used to it.
Perhaps the "Rails generation" will gain some engineering, product selection and QA skills now.
There's a big reason banks operate the way they do with the kit they do.
[1] Auto-Unmarshalling yaml in xml, seriously? Who wants yaml in xml?
[2] I'm sorry to single out spring here, but this is a nice example since it's the same kind of attack: Instatiation of an arbitrary class, in this case by modifying the class loader and loading the class from a remote server: http://support.springsource.com/security/cve-2010-1622
Rails pushes low time to market. That is all. Its built with precisely no engineering design or quality control on top of a poorly specified rickety language in a community of hype.
However, my original generalizes the problem as a human issue which is where the real problem is:
Did they do a risk analysis on rails - no
Did they verify their architecture - no
Did they perform input/type checking - no (sorry but statically typed languages win here)
Did they act responsibly - no
Would you trust them with your cash? Probably not then.
This is the opposite of banks who have multiple layers of security to prevent all the associated risks of the business.
Would a bank run their OLTP on rails? Hell no, because they did the risk analysis above, which is my point.
Would a bank run their OLTP on rails? I don't know. I'd rather say that rails is a bad fit for that kind of problem, but that's not the goal. Rails is built for web-applications of a pretty specific type, quick build and release cycles while still keeping a solid focus on code quality [1]. So your problem is that folks see an opportunity to make money, take the first tool that seems to fit and build stuff that doesn't hold water. But that's language agnostic. You certainly realize that 10 years ago java was the "poorly specified rickety language in a community of hype".
[1] I never dreamed I'd be defending rails. gosh.
Actually, this bug was discovered precisely because people began to perform a more in depth analysis.
>Did they perform input/type checking - no (sorry but statically typed languages win here)
LOL. It's 2013. Can we stop having this preposterous argument?
The argument is definitely not preposterous. Are you saying guarding against bad inputs and enforcing type is bad? A language which uses no type inference has less assumptions therefore is likely to be less error prone. I've proven this hundreds of times over the last 30 years of writing code in things from communications electronics to financial quotation platforms.
And you know what, I have a feeling they did have multiple people looking over the code, and it's been vigorously refactored over the years. Rails 3 is a somewhat different beast from Rails 2.
Careful auditing reduces bugs but does not eliminate them. Crowing about "proper engineering" is very nice but is somewhat farcical given the state of the art in the industry.
>Are you saying guarding against bad inputs and enforcing type is bad?
No. Of course you guard against bad inputs.
But that's wholly separate from enforcing the type of the objects. And it's not like people are eval()ing stuff willy-nilly.
For example float vs decimal types in finance. You really want your bank running on floats when doing interest calculations?
From Wikipedia: Ruby on Rails - Initial release July 2004
Eight-and-a-half years later, in depth analysis has come about to find this? (The exploit is present through 3.2, meaning it's been around all that time).
[1] Sinatra didn't, but that's < 1000 LOC.
I would gladly take the other side of that bet.
A decade ago I was working on a site using Perl/Mason/Apache. When we were bought by eBay, we were put through a thorough pen test. The ONLY security hole they identified as needing fixing was a redirect that could redirect to any URL anywhere. (The people testing us were shocked - they had never before seen that few problems.)
To the best of my knowledge, no holes have been discovered in that architecture in the following decade either. (Looking through Apache vulnerabilities, there have been a couple of remote code exploits. But not on all platforms, and we turned off every feature we didn't need.)
If you take the attitude up front that you won't have magic, this kind of bug doesn't tend to slip in. If you take the attitude that there will be a lot of magic, then this kind of bug does slip in.
You admit that the framework you used had a couple of exploits and you were not affected because you turned of all features that you didn't need. The current rails vulnerability does not affect you if you turned off all features you didn't need. Same argument. So we have a hole in Mason, I already cited one in Spring, someone else cited one in .NET http://news.ycombinator.com/item?id=5043839.
Think about it: for most apps, probably 95% of your code is a framework that you didn't write. When this is criticized, the response is typically "open source unicorns! speak no more!" Moreover, there seems to be a different level of forgiveness just because it's open source. If ASP.NET WebForms had the same level of security holes in the past years as Rails ... just wow.
I love frameworks, and I think an average team would on the whole write less secure code than an open source framework would. If we want maximum transparency into our code, we'd go back to CGI written in C, but our productivity would tank, and looking back, the boom of recent years never would have happened.
However, we need a different attitude. Let's be honest: the arguments about open source are mostly a lie. Yeah, I went there. No, I'm not advocating closed source. Too many hang their hat on the ideals of open source. We're not talking ideals. We're talking rubber meets the road, cash leaves the bank reality. "Anyone can view the source - many eyeballs makes it more secure." The truth is, there aren't that many eyeballs. How many download, or (for the real technology ninjas) git clone, and trust what they're working with (whether app or framework) by a faith that rivals any religious organization?
As an industry, we need to read more code. We need to push back against the Barnes and Noble developer, the one who paid $29 for a book and now calls themselves a developer because they finished it front to back. Not everyone needs to be a senior engineer, but we have too many with that title after 3 years of building framework apps who couldn't code their way out of a CS101 class if they had Donald Knuth and Dennis Ritchie as their personal tutors.
This is pure supposition, but I think if at least 10% of those who used Rails read Rails, it'd be far more secure.
That's a joke, right?
https://www.google.ca/search?q=asp.net+remote+code&oq=as...
>we trust the framework authors implicitly
You want to export your common web app code to a framework for all the same reasons you don't want to write your own crypto libraries - the more people look at it the safer it is.
Far more damage has been done to the web from people not using a good ORM or a toolkit that escapes your view layer against XSS, or forgot to add request forgery protection.
I never claimed we shouldn't leverage well-written projects. I do challenge the idea that there's many eyeballs: most download and have faith. There's a bunch of eyeballs, yes, but not nearly as many as we tell ourselves.
You're totally right about frameworks resulting in better, more secure apps. However, much of that is relative: when most apps are hand-crafting SQL, JavaScript, and form handling, a framework is heads above other apps, especially when most attackers go for low-hanging fruit. However, when we get to a point where most apps are in frameworks, where would attackers turn their attention? Is an app inherently secure, or secure because it's less of a target? (Think the Mac vs. PC security arguments)
Not really, because in .NET separating the "language" from the "framework" is a little thornier.
And the difference is all the more moot from a practical point of view: you still have to rush out and patch everything.
> However, when we get to a point where most apps are in frameworks, where would attackers turn their attention? Is an app inherently secure, or secure because it's less of a target?
You're not wrong in that widespread deployments of the same code base are what make these kinds of attacks possible in the first place; that's one downside of this kind of monoculture.
I can tell you however that these frameworks make it easier to write secure code in the first place. My business partner in my consultancy is a security researcher, and his job is to break apps. Frameworks like Rails make his life "harder" (in so far that one looks good by finding vulnerabilities); there is a ton of automated tooling that can probe and exploit a myriad of common-developer-mistakes-or-misconceptions.
Code you wrote within your team is audited by 10 people, tops. Code like Rails has been seen thousands of eyeballs. Even with several orders of magnitude of more attention, stuff like this still slips by. What are the odds you are better off?
Possibly 2-3 people have read it who know their shit. The rest are just consumers.
Frameworks only centralise the security concerns - they don't necessarily make it better. That is in the hands of the implementor and their ability to build bullet proof abstractions. One fuck up and your system is globally compromised. It is the right solution, but you need the right people. Rails hasn't had the right people.
And if we're using that metric, it's extremely unlikely your average team member knows his or her ass from their elbow, either.
I agree with your second point, which is why I am a senior member of an architecture team in a company with 80 developers. It's our job to make sure ass and elbow confusion doesn't compromise the product.
I feel like once you have 100k+ in there you become a target for some very persuasive people...
The two transactions, debt providing cash to the government, and seigniorage (difference in value from the metal in the coin and the fiat value asserted) providing cash to the government for the coin issuance are approximately equally inflationary, over time, assuming that the debt remains outstanding. If such a coin were undertaken, in all probability, it would be repurchased by the Treasury and retired, and debt would be issued. Indeed, the a return of the coin to the Treasury could be via issuance of debt directly to the Federal Reserve.
A little perspective on Federal Reserve Bank's process to create cash:
Federal Reserve Balance Sheet - by James Hamilton on Econobrowser
http://www.econbrowser.com/archives/2008/12/federal_reserve_...
are approximately equally inflationary, over time.
Your arguments appear to be directed at people who support the Republican party or are concerned about the partisan points here. Bush, Obama, Bernanke, Greenspan, Krugman, and the whole gang are peas in a pod. Bush won reelection in 2004 by having Greenspan inflate a housing bubble with Krugman cheering[2] the Fed Chairman on. Obama won reelection in 2012 by having Bernanke re-inflate[3] a housing bubble with Krugman cheering the Fed Chairman on.
[1] http://www.forbes.com/sites/jackperkowski/2012/06/26/china-b...
[2] http://www.nytimes.com/2002/08/02/opinion/dubya-s-double-dip...
To fight this recession the Fed needs more than a
snapback; it needs soaring household spending to offset
moribund business investment. And to do that, as Paul
McCulley of Pimco put it, Alan Greenspan needs to create a
housing bubble to replace the Nasdaq bubble.
Paul Krugman: August 2, 2002
The Federal Reserve said it will expand its holdings of
long-term securities with open-ended purchases of $40
billion of mortgage debt a month in a third round of
quantitative easing as it seeks to boost growth and reduce
unemployment.Bitcoin are uniquely identifiable by nature. Has there been any attempt to create and maintain a manifest of "tainted" (i.e., stolen) bc that could be referenced during transactions?
If a receiver of bc knows that they are tainted, and that the next receiver might refuse them, then they might refuse them as well.
I understand that I'm hand-waving over the "Alice reports the bc she just paid Bob as stolen" problem, and I don't have an answer to that---maybe users subscribe to this service that investigates bc robbery---just thinking out loud.
This would create a chilling effect on trade since now you have to worry about the legitimacy of the other party, which brings us right back to the problems we have with credit cards, etc.
http://anonymity-in-bitcoin.blogspot.ie/2011/07/bitcoin-is-n...
I feel bad for anyone that actually got screwed because of this.
Also, you already trust an anonymous form of money: cash. Why so? Mostly because insurance exists against cash theft. As the Bitcoin ecosystem develops, eventually an insurance market will develop too. It sounds like Bitcoin is too unmature for you. So, wait. In this theft case, the exchange itself is providing this insurance to you: they guaranteed they will cover the stolen funds themselves.
Also, the beauty of Bitcoin is that you don't have to trust an exchange like the one that was the victim of the theft. You can decide to take care of your own security by storing bitcoins on your own computer. There are a few efforts in progress to develop credit-card sized hardware wallet in order to provide extremely safe bitcoin wallets to non-technical users who are unable to, or don't know how to keep a computer secure for hosting a Bitcoin wallet. Again, it sounds like you need to wait for these efforts to mature before judging Bitcoin.
Given that simple scenario, think about who is watching your money..
That's great its ensured though. Good deal
You are trusting your money with someone who can't even make simple critical updates to an app.
Furthermore: unless some established institution wants to work with bitcoin.. like a bank, then youre leaving your money up to the hands of some individual developers who, as made apparent by this post, dont really have their shit together.
I could be wrong, there could be some massively secured bitcoin bank somewhere with a large team of security experts, insurance, etc... but based on the fact that most bitcoin banks i see use an unstyled bootstrap theme I can't imagine they really know what theyre doing.
cliffnotes: I have yet to see a bitcoin bank website that doesnt look amateur as hell. I bet money that whatever bitcoin bank was hacked had all kinds of pages on it about "how we have a full team of security advisors making sure your money is safe!" yeah right.
bitcoin itself is cool, the mediums used to store your money are what I'm not trusting.
They never replied but did fix them.
I asked around and was told "oh, yeah, they changed the defaults recently so there is always a transaction cost, you should pay closer attention to mailing list X, it's more up-to-date than the API documentation."
I decided I didn't like dealing with something where I had to assume the fine print was out to screw me and walked away. I have other things to do with my life.
(I probably am misremembering a few details of something that took place a few years ago.)
"Ass-kicking"? Is this high school football or professional programming? If a language or framework has major security holes, it should fail in the marketplace, no matter how much "ass kicking" it has done.
Rereading your post, I'm going to assume it's a troll post. After all, when I see your username, I can't help but imagine, "Do a couple of quick sets down at Gold's, then come back and slam out some Rails, yeah brah!"
It is a fair enough conquest to learn Rails the proper way, knowing exactly what is happening under the hood. Rails productivity doesn't come from how easy it is to generate some pre-built basic controllers or models without knowing what is involved, Rails productivity comes from how well it is engineered and connected in the whole picture. In fact, all Rails pro's will tell you that they have stopped using those "automagical features that make productivy fast" as soon as they were able to understand what they were doing, for the sake of more granular customization. So, in 2 words, you won't have a good working knowledge of Rails in a week-end, expecially if you don't know ruby. You'll just learn to use some "generators".
> If a language or framework has major security holes, it should > fail in the marketplace
It should in a perfect world designed by an IT professional which fortunately it isn't. It will not in reality, and fortunately we have plenty of cases for this.
> I'm going to assume it's a troll post. After all, when I see your username I can't help but imagine [...]
You have a poor imagination.
IMHO, this is a misconception, though I must admit it's one that gets pushed by a lot of parties. There's a lot of moving parts in rails and a lot of magic shortcuts, but IMHO you should actually know what goes on behind the scenes before using rails for a serious project.
> If a language or framework has major security holes, it should fail in the marketplace
So no more Spring, no more ASP.NET? Come on, we should be beyond that. Holes appear everywhere. What matters is how they are handled.
"Before the wild speculations beginn, the service will be recovered and we pay the losses out of our own pockets."
Is there a Best Current Practice for all steps in BitCoin - for people wanting to buy or sell bitcoins; for people wanting to trade goods for bitcoin; for people wanting to run bitcoin financial services or exchanges?
By they way chrome shows their https certificate also has some problems.
I only had one affected app. I just bumped my Gemfile to rails 3.2.11, gem update.
When they say "multiple times" I hope that means they're using PBKDF2 or bcrypt, because if they were simply using SHA1 then their users are even more doomed.
[0] https://bitcointalk.org/index.php?topic=135919.msg1448056#ms...
Now how many real banks do run their website using Rails? Ruby?
You guys certainly aren't as stupid as to believe this is the latest major 0-day Rails exploit to create havoc right?
And now comes the answers containing the logical fallacy: "All languages/frameworks have security issues". Which is rubbish in that it would imply that all the languages/frameworks do offer exactly the same level of security...
You could argue that "big, enterprise" systems are likely to be more secure, but experience with things like Oracle databases around 8/9/10 would indicate that's not always a good measure.
You could argue (and people do) that the framework being open source is a good thing, or a bad thing.
So absent that information how would someone factor that into their choice of system?
http://blog.trendmicro.com/trendlabs-security-intelligence/j...
Cheers!