Fraud caused disappearance of 99% of Mt. Gox Bitcoins | Hacker News Reader

Fraud caused disappearance of 99% of Mt. Gox Bitcoins

the-japan-news.com

148 pointsjpatokal71 comments

From all the theories I've seen, the most likely to me is that this was a business that slowly drifted into a whirlpool of fraud and insolvency, probably starting with good intentions.

Most likely, they had a hard time maintaining the right ratio of bitcoin/yen/USD to match the reality of their customer's deposits (through incompetency) and got hit hard when the "wrong" price fluctuations occurred... when this happened, and in which direction the fluctuation happened, I cannot say.

After that, they were deeply into a fractional reserve situation and thought "hey, we own a large part of the Bitcoin market, we can probably play the price a bit to make our customers whole again, without anyone knowing what had happened."

In this way, they gradually drifted from "cutting corners and doing the ugly things required to keep a business afloat" (aka mild, veiled fraud) into outright fraud.

Your theory seems reasonable to me, and I know you're not defending them, but ...

How difficult is it to keep the right amount of bitcoin and USD in deposit? It seems pretty simple to me. Customer deposits $1.00, then I keep $1.00 in deposit in my business account. Customer deposits 1 bitcoin, I keep one bitcoin. The account for customer deposits will only ever be touched by automated systems processing these transactions.

If I'm charging fees as part of depositing/withdrawing/trading currencies or BTC, then those fees are immediately swept into a separate account, the entirety of which is revenue. Kept strictly separate from deposits. I'd regularly audit both accounts to confirm that the dollar amounts of each deposit/withdrawal, as well as total, match the records of actions in my systems.

Mt Gox only ever handled about 40 trades per second. Seems like you could keep everything balanced with a typical database, using transactions to implement atomic trades / deposits / withdrawals. Perhaps interfacing with money transmission systems is slightly more difficult, but once a transaction has reached the point of having probably-completed, you initiate the transaction to modify the customer's balance of bitcoin and currency.

So I guess I'm agreeing with you that if they failed in this way, they must have been incompetent.

P.S. I'm just making all this up. I've never worked in payments or accounting. I expect a good security or finance auditor could suggest much better controls. I'd pick up an "accounting for dummies" book and learn about the basic control principles used in accounting to prevent fraud. Perhaps I periodically download statements from my bank, print them out (or receive them in the mail) and audit those records against my own systems, by hand. Do the same electronically on a continuous basis. Set up an off-site write-only logging system to capture all transactions, and regularly audit to ensure that the system logs match bank transaction logs.

This is just wild speculation now...

...but my best guess is their trading engine was so unstable that they ended up using one server to run a stripped-down clearance house algorithm for executing the trades as quickly as feasible, and a separate system that was the "source of truth" for how many physical assets in bitcoin/yen/USD were actually in possession by the company.

I'm guessing that there was absolutely nothing "atomic" about the system Marc Karpeles had built, their storage system was just an amalgam of disconnected, duct-taped databases living in different places, that were "kinda, sorta" in sync with each other, with lots of unpredictable, undefined behavior.

MtGox was pretty buggy and many of those bugs lost money, which could easily make them insolvent. Also, interfacing with Bitcoin at scale (e.g. managing thousands of addresses) required a lot of custom programming.

> I'd set up a write-only logging system to be used for auditing company systems

I bet Marc Karpeles had 5 people calling him at once that the trading engine was down again so THAT's where he put is efforts... He didn't get any calls from people saying "By the way, I hope you have robust auditing procedures."

Like some people live paycheck to paycheck, I'm betting the MtGox guys were living from critical bug report to critical bug report, without time for "luxuries" like "atomicity" or "database auditing"

(Let me say up front I am in no way intending for my comments to be a defense of the behavior of these criminals.)

It has been credibly alleged [+] that Mt. Gox's general level of engineering acumen can be summed up as "No tests, no staging environment, live coding by SSHing into production."

[+] Unnamed employee of Tibanne, the parent company which supplied substantially all of 0-employee Mt. Gox's staffing: http://www.pcworld.com/article/2105920/lost-in-translation-t...

On the topic of SSH: It appears that they used their own version of SSH reimplemented in PHP: https://web.archive.org/web/20140102104411/http://blog.magic...

True, but a system like I'm describing ought not to be terribly complicated to set up. Perhaps it's simply a remote syslog daemon under the control of a trusted exec. Preferably offsite or in a separate cage. Or if SAAS, under a distinct account.

Maybe we don't block transactions on writing to the log, but whenever discrepancies are detected of more than a short time (transactions more than 60 seconds old not appearing), we page all founders and senior leaders. The trading engine stays up, but we draw much attention to the logging outage.

(I'd also want controls like offline wallets and accounts that would prevent an attacker from obliterating our assets in a single withdrawal)

I'd like to note that Gonzague Gay-Bouchery is an oft-ignored part of this saga. He has gone to a lot of effort to silently vanish (deleting most of his online footprint just before the announcement last February) and has been mostly ignored, but reportedly had a large role in the business side of Mtgox, while Karpeles handled the code aspects.

If you're familiar with the Willy Report (Google it), the simplest working theory is that someone figured out a way to award themselves USD/etc balances inside of the Mt. Gox system, purchased large amounts of Bitcoins with them, then transferred them out in the ordinary course of business. This caused positive improvements in the set of metrics that Gox and the community actively monitored, and the metrics in which the discrepancy would have been obvious were apparently not actively monitored. (e.g. Comparing the sum of all USD/etc liabilities as reflected in the database with the sum of balances at Mt. Gox's single-digit number of bank accounts.)

You could call it the most total failure of accounting controls in history, if one had any reason to suspect that Mt. Gox possessed meaningful accounting controls. There was no reason to suspect this, but many people were willfully blind to that, because it appeared that Bitcoiners were making money hand-over-fist.

Edit to add: One reason Gox may not have done the math on the assets vs. liabilities is because they knew they were insolvent from some point in early to mid 2013, as a result of some combination of external events, including the since-reversed freezing of their US bank accounts. The clearest publicly available evidence of insolvency was persistent unwillingness to allow people to withdraw USD/etc and ludicrous excuses for why this was the case. (At one point, the CEO alleged that their daily wire volume had overwhelmed the second largest bank in Japan. This kind of claim is very useful because it is trivially falsifiable.)

> You could call it the most total failure of accounting controls in history

Enron?

Nah, Enron's prominence is purely political, it was useful for various narratives. There are far worse examples, and Mt. Gox sounds like it's in the running.

E.g. there's cheating on your general ledger (which at 50,000 feet might be the sort of thing Enron did), then there's not keeping a general ledger at all, which is being posited in this discussion.

Even I, who's never run a company, but did watch my parents run some while I was growing up, know how essential it is to keep a general ledger. Otherwise you have no real idea what your financial state is.

not keeping a general ledger at all

To be fair, you don't need a general ledger if you are setting up a website to trade playing cards. Say an Online eXchange for Magic the Gathering.

Every time an Mt Gox story comes up, the most incredulous thing to me is, why was anyone even expecting these guys to act like a bank? Why did anyone even consider trusting them with actual money?

Because bitcoiners were willing to believe anything which fit their narrative, logic and diligence be damned.

LOL Bitcoin fanboys have shown up on this thread I see judging by our down votes...

You'd probably still want a ledger there, if you're trading physical goods. That way you can correlate confirmed receipts on both ends of whatever cards were traded.

No probably. Presumably you're doing this for a cut of the action, so that's income on one side, then there's your out of pocket costs to run the system etc. Knowing how these two compare can tell you all sorts of things, including how much you can safely take out for yourself if things are going well enough, vs. knowing you'll have to pay next month's co-lo fees out of your pocket.

Now, of course, at that level something as formal as a general ledger isn't quite required, but you still need to do the same things in e.g. watching the organization's bank balance.

I am admittedly a bit of an accounting fanboy, and may jump to more rigorous tracking of things I control or are accountable for than is needed. For instance, upon discovering a large portion of the intelligence funding in the United States is "off-budget", I was concerned about their financial rigor (your budget should include all the things you spend money on -- it's odd to think the US Government is quite a few steps and security clearances away from being able to answer "How much do you spend on pens a year?")

It wasn't an online exchange for MtG cards, it was an exchange for MtG:Online cards (digital collectibles), and, IIRC, other than buying the name, the bitcoin exchange didn't have much in common with it.

Mt. Gox was popular. In an environment without some sort of central accrediting authority or standards body for Bitcoin businesses (the very idea would be abhorrent), popularity confers legitimacy, and convenience confers popularity.

I also suspect there were also more than a few people who trusted it implicitly because it wasn't a bank, they just assumed it worked because Bitcoin.

When you begin from the premise that the models followed by the financial world are at best obsolete and at worst a criminal facade, of course you might not bother referring to them if you start a business, or care to find out if an exchange follows the best practices of an industry you consider corrupt and useless.

I'm not saying this describes all bitcoin holders or Mt. Gox customers by any means, but the politics around Bitcoin do seem to lend themselves to an unwillingness to hold third parties up to any sort of standard.

Bigger than Enron? Not even close: http://usatoday30.usatoday.com/money/energy/2002-01-22-enron...

And that's just the primary, don't forget about the fall-out (Arthur Andersen implosion, for one (miraculously, they survived)).

Only in a technical sense; Arthur Anderson is in a line of business where a criminal prosecution is pretty much a death sentence, a conviction absolutely so, even if later reserved. Per Wikipedia:

In 2002, the firm voluntarily surrendered its licenses to practice as Certified Public Accountants in the United States after being found guilty of criminal charges relating to the firm's handling of the auditing of Enron, an energy corporation based in Texas, which had filed for bankruptcy in 2001 and later failed. The other national accounting and consulting firms bought most of the practices of Arthur Andersen. The verdict was subsequently overturned by the Supreme Court of the United States. The damage to its reputation, however, has prevented it from returning as a viable business, though it still nominally exists.

How difficult is it to keep the right amount of bitcoin and USD in deposit? ... P.S. I'm just making all this up. I've never worked in payments or accounting.

Mt. Gox had a financial control problem less complex than that faced by the typical supermarket. A supermarket has cash going in and out, multiple cash registers, multiple cashiers on different shifts, credit cards, checks, benefit cards, and automated checkout lanes. That's just the front side - there's a comparable operation at the back, where merchandise comes in and is accounted for. A big supermarket has a higher transaction rate than Mt. Gox did.

If a $10 bill goes missing in a supermarket, the management will know at the end of the day, and will probably be able to figure out where it went. This is routine cash control. It's a solved problem.

Mt. Gox had no financial controls; we know that from press reports. None. They don't even seem to have had a general ledger. This is rare in real businesses, but it's common in fraud schemes. Fraudsters don't want accounting controls, with everything they're stealing on the record.

Anyway, on Jan 3, we'll hear more. That this story came from the Tokyo Metropolitan Police is significant. The police haven't said anything until now, except that they were investigating. That silence has ended. Arrests may be announced. There's a good chance that Mark Karpeles is about to experience the standard 21-day interrogation used in Japan.

> If a $10 bill goes missing in a supermarket, the management will know at the end of the day, and will probably be able to figure out where it went. This is routine cash control. It's a solved problem.

Not necessarily, differences are a daily part of working as a supermarket cashier. Most supermarkets have agreements with their employees (sometimes mandated by law), varying from full employee liability to full coverage by employer. It's actually quite common to have differences in your register, e.g. because customers let you keep small-ish returns or give "real" tips or you as cashier in a high-frequency environment make a mistake in sorting in a bill into the right compartment - or fraudsters trick you (http://www.weser-kurier.de/region/delmenhorster-kurier_artik...).

The point is that cash differences in a supermarket get noticed within hours. Some level of error can be accepted, but if somebody comes up $40 short several days each week, they're going to be looking for a new job. Mt. Gox claims they didn't know they were being drained of assets for months.

$40 short weeks in a row, or in this case, ~650,000 $500 bills short in a couple years :)

Exactly. I have a friend who while high school in the '60s worked in one of those highway restaurants, if you travel in the right parts of the US you'd remember their name.

Of all their cashiers she had the highest "volatility", you might say, from day to day. That conceptually annoyed her superiors, but they loved her because she never made a big mistake, and her daily errors averaged out very nicely.

Not to mention the 90+% conviction rate in Japan ...

Fun fact: in no year since 1957 has the Japanese court system failed to convict at least 99.9% of indicted suspects.

Even more fun fact: this is not the strongest statement one could make. The last number I remember citing for it was something on the order of two dozen not guilty verdicts, nationwide, out of about 125,000 prosecutions in a year, but don't quote me on that.

And you're not afraid? Do arrests seem legit when you hear about one?

I'm generally not afraid of the police when they're not actively questioning me on suspicion of me having committed a crime. That has happened eight times. Six of the eight times, it was on suspicion of stealing my own bicycle, the relevant evidence being that I was a) not Japanese and b) riding a bicycle. One time, after the cops had called my bicycle's anti-theft registration number into HQ and received the radio report "That bike is registered to one Patrick McKenzie", I was accused of stealing Patrick McKenzie's bicycle and identity. [+]

Does this answer your questions?

[+] I realize this sounds like the kind of story an Irish storyteller would make up to prove a point and feel that I must add "I swear by all the saints, by my hope of heaven and my fear of hell, this did literally happen."

It's the perfect crime. Steal someone's bicycle, along with their identity, and ride the bicycle with impunity.

How did you convince them you were him and that you hadn't stolen your own identity?

What next? They make you prove you've never been cloned?

I showed him a particular immigration document issued to me that he is not, strictly speaking, allowed to ask for prior to starting a custodial interrogation. He then started second guessing Immigration's decision to offer me a visa.

That was good news, since it convinced a colleague he was overstepping his authority. I was released 5 minutes later, with an admonition to lock up my bike properly to avoid the scourge of foreign bike thieves plaguing Ogaki.

And the other two times?

That seems incredibly scary to me. When something is that insane, what stops it from taking the next step of insanity? I'm sorry that it happened to you (let alone six times), and am glad you made it out ok.

One part of it is that the Japanese police won't arrest someone unless they think they can nail them. A lot of murders may be ruled suicide because there's no convenient scapegoat, he-said-she-said rape cases not pursued, etc.

OK, but that's approximately true of criminal justice in the US. For instance: wiretaps are administratively expensive to pursue, and so police don't request them until they're sure they've got enough evidence to win the warrant. But civilians aren't persuaded by this logic; instead, they see it as evidence that the whole system is tilted in favor of the police.

It should be as disquieting (or not disquieting) in Japan as it is in the US.

If you look at the criminal justice system as having some amount of diagnostic sensitivity at each stage in the process (having both false positives and false negatives as all systems do), then our comparatively lower conviction rate means that (assuming end-of-the-day-justice is equal), we should be even more willing to waste people's time developing cases that have no merit (because more of the diagnostic sensitivity is in the trial/conviction stage). So it's not unreasonable to assume it's easier to get a wiretap in the US than it is in Japan -- another explanation is that Japan is substantially more likely to have false negatives (as implied elsewhere -- but the false negative rate for many crimes in the US is quite high, as roughly half of all murders go unsolved). Could also be that the diagnostic sensitivity is extremely bumpy in America or Japan (read: basically one or two steps / people get an extremely high share of the criminal justice discretion).

Should be easy enough to prove or disprove by comparing the number of X-type arrests made to the number of X-type crimes committed.

If it's absurdly high, or significantly higher than some comparable country, then you've got a problem - with a 99.9% conviction rate.

99.9808%, given those figures. Some numbers I can find online pin it at 99.97%, which in either case is comparable but greater to that of Communist China (99.93%).

Just curious, searched and couldn't find anything meaningful, what is the "standard 21-day interrogation"?

http://bbc.co.uk/news/magazine-20810572

http://news.bbc.co.uk/2/hi/8290767.stm

Police can hold suspects for 23 days; those interrogations are private; Japan only recently re-introduced trial-by-jury (2009!); etc

Here's a nice 28 minite radio programme talking about the problems. http://www.bbc.co.uk/programmes/b01phktd

To make a long story short: the general standard of Japanese policing at present is quite similar to the prevailing standard of US policing in the 1950s. The primary investigative tactic is coercing confessions, and I do not use the word "coercing" lightly. Japanese cops can jail someone on suspicion of wrongdoing for 20-something days prior to having to bring a magistrate into the picture, during which time they're routinely physically and emotionally abused and made to think that signing whatever the cops put in front of them will be the quickest way to end the ordeal.

Doesn't this get abused heavily?

It is systematic abuse; I'm not sure it makes sense to talk about it being abused...

Indeed. Japan is a police state. A remarkably polite one, but things at this level are done for the convenience of the police, prosecutors and judges (at least prior to adding juries to the system, but I'm quite skeptical how that'll turn out for quite a while). Closing cases is more important than finding the real criminal, and atrocities like their handling of the Aum Shinrikyo cult suggest they're not so good when put to serious test.

Are they juries? A lay judge is different, and given the implementation of it, it seems barely better than what they had before, since a lay judge is typically going to just defer to the professional judge most of the time, anyway.

I figured the system would end up being "juries" only in form and didn't investigate it, but it's of course it turns out to be much worse as you point out. And then there's this gem from Wikipedia which echoes the early issues with common law juries, which were not much respected by the ruling class if the case was important and they rendered the "wrong" verdict:

"Others have written with concern regarding the harsh secrecy provision in the statute which includes the risk of criminal penalties for those lay judges who would publicly share confidential deliberation room discussion even after trial proceedings are complete.:

I'm sure there's a lot more where that came from, plus of course as you note extreme deference would likely be paid to the professional judges. The above is from the end of https://en.wikipedia.org/wiki/Lay_judges_in_Japan

"Atrocities like their handling of the Aum Shinrikyo cult"? Are you referring to the guy falsely blamed for the Matsumoto attack? The police are indeed likely to blame for leaking his name to the media, but he was never even formally charged, much less convicted:

https://en.wikipedia.org/wiki/Yoshiyuki_Kouno

The investigation of the Tokyo sarin attacks, on the other hand, seems to have been carried out pretty much by the book and I'm not aware of anybody outside the tinfoil hat brigade who contests that they got the right people.

After the sarin attacks, they did a good job. I'm referring to the multiple times they dropped the ball prior to that point, which among other things allowed the sarin attacks to happen. Their screwup on the Matsumoto attack is an obvious example; they they harmed an innocent in an incorrect focus on him is in a way much less important than letting the culprits later do much worse.

I don't know if this is actually "standard" or if there are just outliers that receive press attention, but there have been reports of innocent people confessing after being interrogated for 15 hours a day [1].

[1] http://www.nytimes.com/2007/05/11/world/asia/11japan.html?pa...

The line between legitimate business and an illegal operation can be as thin as the line between genius and insanity.

An alternative explanation could be that Karpeles, who was convicted of fraud in 2010, decided to stick with the fraud thing from the start.

Here's the thing I don't get. By default, you're receiving bitcoin and sending out dollars/yen. By default, there should be no discrepancies! I feel like you would need to be actively incompetent and trying to do tricky things for this to go out of sync.

If I had a box, accepted apples and oranges, and would give them back to the owner if asked(taking some slices as payment fees, but at transaction) , at what point can things disappear? The only thing I can think of is race condition style things, but that's a single point of failure right?

Maybe I'm missing some complexity here

SO not a surprise that an unregulated exchange of securities was mismanaged and collapsed. Explain to me again how Bitcoin is supposed to work? Because from the outside it looks like an electronic Ponzi scheme.

I'm skeptical of bitcoin's success as well, but for different reasons. I am however totally confident in the technology itself, and it doesn't really play much of a factor in this story as far as I can tell. It kinda sounds like you're conflating bitcoin exchanges with the technology itself.

Nothing to do with technology. Has to do with arrogant developer with no experience with financial regulation, thinking they can just ignore centuries of safeguards because they know math.

Well this could be interesting if it's more than hype to sell more paper.

Karpelès having quite a trail of fraud and misconduct, I'm curious to learn the results of the Magic The Gathering Online Exchange investigation.

For those who don't know yet about the past of Karpelès, he started his career when over a disagreement with his employer he moved some of the company's clients' data over to his own servers, redirected a company domain towards one of his then resigned from the company. When the company confronted him, he refused to give back what he took and offered to buy the domain he stole, so the company went to court with Karpelès admitting he did it to the police but telling a different story on his blog [1]. With Karpelès not appearing at the audience because he had left the country he got sentenced in 2010 to 1 year of prison and 45 000€ in damage [2].

This is only part of a trail of lies, deception and fraud from Karpalès for his personal profit that has started to surface, we know have a blog he made in 2006 where he confessed of getting caught for a youth error of doing bad things with online payment systems for 2 years[3] or how he used lies to get money for a job he didn't do[4].

Of course that doesn't mean he did stole the coins, that's why I want to know about the investigation.

[1]: https://web.archive.org/web/20070630213730/http://www.magica...

[2]: https://www.documentcloud.org/documents/1238981-img-20140720...

[3]: http://web.archive.org/web/20140302234940/http://blog.magica...

[4]: https://www.cryptocoinsnews.com/exclusive-tibanne-co-ltd-sen...

It's astonishing that financially minded people, used to doing due diligence for potential investments in their day jobs, would trust a company with their own personal money without digging into its founder's backstory!

Most (all?) of that material was in French, which seems to have been a surprisingly effective barrier. (French proficiency is not terribly common in the US or Japan; and how many people are so paranoid about random counterparties that they will hire translators to go through everything any of the counterparties have ever written looking for red flags? Have you ever done that?)

I don't recall even the Mtgox haters bringing up any of the French backstory until not long before the end.

I'll try to find a print copy when the Yomiuri hits news stands on the 3rd and post the gist of it, but don't get your hopes up on this shedding a lot of light on the story.

Please, and thank you! I've been avidly following every piece of news about Mt. Gox, so it would be great to hear a translation of the news story. It probably won't have any new details, but I'm very interested to know how Japan feels about the situation. A translation of a printed story might not be the best way to find that out, but at least it's something.

Reddit seems to have some info from the print edition already: https://www.reddit.com/r/Bitcoin/comments/2qyyy9/tokyo_metro...

It's almost like people with no education or training and a history filled with criminal activity can't be trusted to run financial institutions.

It's almost like there is a great irony in the fact that people who mistrusted government run banks sought an anonymous unregulated cryptocurrency and then immediately got ripped off because it's anonymous and unregulated.

> government run banks

What's one of these? It's the other way round - banks run governments.

Should probably be read as government-regulated instead. For some value of "regulated", anyway.

Is that to suggest that regulated banks (or any kind of government-regulated institutions) don't ever rip people off or otherwise engage in criminal or fraudulent activity? Or is your point that this type of corruption is innate to any system involving humans?

I for one am just interested in its potential as a technology foremost; this isn't about mistrust of the government for me.

This is still a mystery? I thought it was widely accepted that Karpeles stole the money for himself to buy Starbucks.

Reply on news.ycombinator.com