Introducing Bitcoinica API (The first RESTful Bitcoin Trading API)
blog.bitcoinica.com
blog.bitcoinica.com
patio11: "My semi-amateurish opinion is that as a non-registered securities dealer with all accounts having margin capabilities the (near 100%) likelihood of your Rails app exploitably broken is not the key source of risk to your business." (http://news.ycombinator.com/item?id=2974695)
See also e.g. http://news.ycombinator.com/item?id=2974032 and http://news.ycombinator.com/item?id=2973429 plus tptacek's reply to http://news.ycombinator.com/item?id=2973732.
On the other hand, Bitcoinica may not be less secure than Mt. Gox...
This is not my first project. I have been coding in Rails for almost 3 years, with some failed projects. I also know some security basics because I have hacked websites before (non-destructive).
There is always relative security. Ruby on Rails itself is a relatively secure framework. Rails sites have been much less frequently hacked than PHP sites. (Because everything is well wrapped.) I'm also very conservative about using Rails (rarely override methods myself, not even toughing find_by_sql even when it's the easy way).
Passwords are stored hashed and salted using BCrypt. Although no explicit security auditing has been done, all the Rails app logs (including requests and IP addresses) are extracted and backed up nightly. Soft-deleting is enabled for all operations, etc.
I'm not distracting myself from the real problem, but I have to say that I'm not that kind of random kids who write crappy apps for fun.
My use of Heroku is also partly because of security. I use Mac and know a bit of Linux, but I know I can't really set up a Linux box and be sure about the security. Instead I have chosen Heroku's integrated, read-only and cloud-based platform.
Sorry for not replying most of the security-related comments. Not because I can't face opposing arguments, but instead, I think they are valuable.
However, I hope everyone can be a little bit more reasonable and realistic. Since nothing is secure, security is everyone's problem. Although I don't have anything to prove my security experience, at least this is not the first time I deal with money. I once developed a mini payment gateway (PayPal and credit card) for my friend who used it to process transactions totaling more than $50,000. There're a lot of fraudulent orders, which means that people have an incentive to steal, but the payment system itself has been safe.
I can understand that people are skeptical considering my age (or even my Chinese name). But will the Bitcoin community be better off if I shut down the project now? I have chosen Bitcoin because its low barrier to entry and open community, not because I have no bottom-line or security concerns.
What I really want to disagree is the view that I must end this project now. There are basically too many people emailing me just to thank me for delivering the features that the big guys (Mt. Gox and TradeHill) can't finish in months.
I have taken the extreme minimalistic approach in development of Bitcoinica - no Bitcoin wallets and no unusable features. I make it a pure trading platform - you can't even do money laundering on Bitcoinica. System wise, I don't have to SSH the server to configure anything, and I don't have to even see the database credentials myself.
Anyway, I completely accept all the negative viewpoints about Bitcoinica, but I just want to explain clearly that I may not be the person everyone's thinking about.
Sorry. Security isn't "everybody's job". You've chosen to advertise a financial application. Security is your job. In a very real sense: sucks to be you.
What I mean is, security is every financial platform owner's big challenge. This includes banks, escrow service providers, payment processors and e-commerce solution providers. Sometimes it's hard to know how secure a system is before anything happens. But I have already explained my experience, ability, current situation, my strategy, how my product works, my objectives and what I have done to the public to help everyone make informed choices.
Not everyone will like every single product. I have been careful about everything I can.
I think the best thing I can do for you is to be blunt about how unconvincing they are.
Very few people with Rails or appsec experience will read them and think that you know what you're doing with regards to security.
You should not be taking "thousands and thousands of dollars" from people. Full stop.
To continue the basketball analogy, you're using a trampoline (Heroku) to reach the basket (a secure system) when you really should be learning how to jump (encryption, anti-intrusion, etc.).
He think it's unconvincing, I accepted. The security standard is very different. What I can say now is, compared to other Bitcoin exchanges which handle millions of dollars of transactions, it's likely that Bitcoinica has almost the same level of security.
However there are many ways to look at a system and see weak points, before you get hacked. Are you vulnerible to SQL Injection attacks? (You have already been shown that RoR sites can be vulnerible to SQL Injections attacks and your ignorance of that fact http://news.ycombinator.com/item?id=2973796 ). You cannot just say "Nothing in 100% secure" as a way to shut down conversation. Yes, there are unknown problems that could happen, but there are known security weaknesses. Are you covering for them?
Remember where you are. This is a website where people are encouraged to pour significant fractions of their life savings, and ideally the money of others, into high-risk, high-reward endeavors. If this crowd has this many people yelling at you to stop, in the level of detail and with the level of experience at and openness to this sort of high-risk high-reward activity on tap here, maybe, just maybe, you should listen.
(And I'd class this "high-risk, low reward" personally.)
But now I have a big question, if your users love your product and they keep investing, but there are many people out there who may never have tried to understand how the product works are yelling at you to stop, what will you do?
I have to admit that many Bitcoin investors are not rational at all, and they don't know what they are doing. But there are also a lot of Bitcoin miners who want to short sell to protect their investment along with some speculators who want to long positions with margin. The market is already here.
I have never claimed that I'm not responsible for the security measures, I just said that many people were over-sentitive in these matters. Bitcoinica is probably the first Bitcoin-related website that doesn't have Bitcoin wallet. And vast majority (I'd say nearly all) of security incidents happen with stolen or lost Bitcoin wallets. I know my bottom-line clearly - I know I may be good at writing secure apps, but not good at keeping a wallet secure.
I don't know what's your attitude towards Mt. Gox or TradeHill. I may have over-used these two examples but in fact, the system works the same way. It's just that I have only 3 years of experience of Ruby development and have done this project more quickly than others.
I'll be working with TradeHill together to expand the service further, because TradeHill requested so. And many users want to see Bitcoinica continue.
Bitcoinca was launched a bit more than 2 days ago. I know how high the stake is, but I'm not going to close it. Our finances are running very smoothly and trading volume is higher and higher.
I will consider hiring a security expert after a month or so. Currently there are two adults working with me in terms of financial systems (they are professional forex brokers). I think everything has just started.
You don't get to put off adding security; for a financial application like yours, security needs to be a foundational element from the beginning. You can't just add it on later.
What if somebody gains access to your database and changes the financial transactions records? Do you store them in a way they can't be modified? Do you have a cron that constantly checks the historical consistency of your transactions, and if there's a problem, shuts the system down? Do you have a mirror DB or a transaction log running on an independent server, so you have a chance to compare if your DB got hacked?
Changes in transaction records do not directly result in financial losses because all the money is stored in more reputable exchanges and banks. Yes, integrity check of transactions is a must and we have it along with the generation of candlesticks (which also checks the integrity of quotes).
Databases are automatically replicated and backed up. Application (transaction and request) logs have log drain set up and will be pushed to a syslog server for archiving.
The databases are only accessible by the instances running this application. The filesystem of the application is read-only.
1) Who are zhoutong and bitoinica? Where is this based out of? I went through the web site and it doesn't seem to mention this. Is he really who he says he is? A 17-year old who built a sophisticated web site in 1 week by himself? Why did he say that he built the web site himself, but in the blog of bitcoinica, it's written as "we"?
2) Why do they deposit everyone's money into a single account? Who owns this account? How do they keep track of everyone's money? REAL brokerages have people's money in segregated accounts, meaning that the brokerage firm CAN'T TOUCH THE MONEY even if they collapse. How do we know they won't one day turn around, transfer the money to China or Singapore and disappear? How do we know this isn't another Madoff case?
3) How the heck does he afford to give everyone 5:1 margin? The simple story of using the money that no one else is using doesn't fly, it's not that easy. You need to have excess capital in order to do this. What is bitcoinica's excess capital?
4) They do NOTHING to ensure counter-party risk!! What if Bitcoinica implodes and goes under? Again, what is their excess capital to ensure that customers don't share in any losses? If they suffer a catastrophic loss from bad trades, is that loss divided amongst all the customers?
I'm not saying they are crooks. I'm very interested in their platform, because I do a lot of algorithmic trading, so I'm familiar with this. I just want answers to these very basic questions before I deposit any money into this thing.
There're currently no explicit laws regulating Bitcoin transactions and Bitcoin trading.
All these points apply to Mt. Gox or TradeHill as well. There are no law restrictions on this kind of trading.
I use "we" because I represent my company xWaylab Inc. (registered in US). I'm the sole developer and project manager but I'm not the only shareholder.
For margin part, since we offer short selling, generally the combined portfolio is quite balanced. Currently our finances are quite healthy. We will never bet against our clients.
I don't really hope anyone will set a double standard in the Bitcoin market. I think everyone is a fair player. You can choose not to trust Bitcoinica if you're skeptical. But a few dozens of people have chosen to give us a try and they have deposited thousands and thousands of dollars.
Believe it or not, Bitcoinica is currently the third largest Bitcoin Trading Platform, following Mt. Gox and TradeHill. (8000+ BTC traded within merely 48 hours of establishment.)
I still believe in time. If Bitcoinica is still around after a few months, maybe there will be more people joining in.
If you stole everyone's money (not saying you will), and then I complained to the FBI, they would look at me and say "Are you stupid? You deposited money into someone's bank account 10,000 miles away, and you actually expected the money back?"
"A few dozen people have deposited thousands and thousands of dollars." And yet you have enough money to offer 5:1 margin? If there are 50 people, if all accounts are of roughly equal size, then it takes just 10 people to max out their margin and the entire Bitcoinica has run out of money!!! But assuming a normal distribution of account sizes, and assuming that the ones who deposited the most are the most aggressive, then you would need much less than 10 to max out before you ran out of money.
There are better ways to set up this company so that you can protect yourself and your customers. Even more surprisingly, you offer no protections to YOURSELF. This is a huge red flag. Why would you not be afraid of suffering consequences if bitcoinica melts down or if there is a huge fraud?
From what I gather, you are asking people to deposit money directly into your account (another huge red flag). You are supposed to be held accountable, if there is fraud or if there is a catastrophic loss to bitcoinica, but you're not protecting yourself, at all. The only way is by remaining highly anonymous, and by living in a foreign country that could be outside the reaches of US law.
If you cared about protecting your customers, you would set up a system of segregated accounts, so that one customer can't destroy people's accounts.
I think bitcoinica is a fantastic piece of coding, it's slick and seems to work nicely. But in order to protect yourself, I would return everyone's money and just allow simulated trading. Even as a simulated trading account, bitcoinica is still a fantastic accomplishment.
Then, once you have created a way to protect yourself and protect customers financially, by using things like segregated accounts, and by going through proper channels, etc, then it might be okay to start accepting real money.
Having my broker also be my trading partner is a bit of an uncomfortable position to be in. If I score a huge upside, you would have a strong incentive to just disappear.
There are people who long and people short. If these positions were opened at the same time and price, they can be canceled out directly. Otherwise, we will trade at other exchanges to hedge.
We are partly responsible for the crash yesterday because we dumped more than 800 BTC into the market when the order book was quite thin due to large amount of short sell orders. The spreads were large due to lack of liquidity.
We have roughly the same financial status regardless of our clients' profitability. We may lose some on guaranteed liquidity, but the loss will be compensated by the subsequent (small) increase in spread.
We can control the risk level any time. Running out of either currency has never happened since launch at least, because our reserves are huge now.
By the way, did you know that by registering your company in the US, you became subject to all of these laws, plus US tax laws, US accounting laws, etc.?
The options clearing corporation [OCC] is the organization that performs clearing and alleviates most of the counterparty risk. They demand deposits from various member companies to ensure that, if a member defaults, at least the other parties get their payments.
Organizations such as SIPC offer protection for investors in the case that brokerage firms belly up. They are government-backed. It's hard for a private company to do that.
Leverage involves a contractual agrement which generally involves some sort of clause that allows the broker to go after your company assets, and sometimes even your personal assets, if you blow up. And the brokers generally have enough cash to provide necessary float (in case of a failure to meet requirements).
zhoutong, it's useful to study how existing futures markets work, and try to see how you can replicate some of those key facets (like counterparty risk)
So, while the concept sounds interesting, it's very likely that if I get in early I will lose all of my Bitcoins to some exploit, and that if I get in late I will not be able to rack up enough Bitcoins to buy anything.
The only downside is that while I have no idea how I would be able to get a small amount of money (?25 Euros?) to the site. Bank transfers don't run on the weekends and while there is a bunch of ways to get money to mtgox, I don't know if any of them can be fueled by paypal/google checkout/amazon payments/... which operate on the weekends
- Instantly available
- Reliable (maybe law-regulated, bank-operated or systematically-controlled)
- Non-reversible
> Ticker:
> URI: GET https://www.bitcoinica.com/api/quotes/[currency_pair].json
That's RPC over HTTP, not a RESTful API.
edit: well, apparently HN sides with meaningless keyword-babble, great.
http://en.wikipedia.org/wiki/Representational_state_transfer...
- The URI is RESTful. (Explicit collection and member actions, with common namespace.)
- Supports both JSON and XML. (Internet media types)
- Bitcoinica API utilizes three of the four RESTful Web Services methods - GET, POST and DELETE. (HTTP methods)
But I'm not sure about the hypertext part, because I don't understand what hypertext means here. Is Bitcoinica's interface considered hypertext-driven?
> I'm not sure about the hypertext part, because I don't understand what hypertext means here. Is Bitcoinica's interface considered hypertext-driven?
It means that the docs should only have to document a single URI (the entrypoint) and the formats, which should have links to the various parts of the API.
For example, you could have an URI, say 'bitcoinica.com/api'. Then you GET that, and it returns a JSON file with the following key/value:
"ticker": "http://bitcoinica.com/api/ticker"
{ "btc-dlr" : "http://.../api/ticker/btc-dlr", "btc-eur": "http://.../apu/ticker/btc-eur", ...
This means the URLs can change without breaking all the existing clients.
In general, I'd say this is "kind-of" RESTful. It doesn't follow HATEOAS and it uses custom formats² (it's inevitable, since there's no standard for this). Having URLs for each concept and reusing HTTP methods and auth is RESTful, though.
¹ Hypermedia as the Engine of Application State ² JSON and XML aren't really formats, they're more like transport mechanisms. Formats are KML, MathML, atom, etc. The point is that knowing it's XML or JSON tells you nothing, your clients are still completely tied to their API. It's just nice not having to write a parser, but it's not a standard format.
Not having this datatype is a big drawback of json for RESTful responses. That's why I typically design APIs to use xhtml responses instead. They're just as parsable (just a big bit slower) but they're richer and they allow the API to be tested with a browser that has no knowledge of the semantics of the API. I consider the ability to use the API using a generic client to be a good test of properly implementing HATEOAS.
Imagine where we would be if every website expressed links using a different syntax, and browsers had to be explicitly programmed to understand a given website's syntax in order to use its links. The web couldn't exist that way.
Frankly, if I GET an URL from your API describing a certain user's profile, and I get a blob of unstructured text, I won't use your service for long.
XHTML + RDFa is a different case, but that's hardly well understood, especially the specific ontologies. It's really no better ATM than using a custom JSON based format.
<div data-role="user">
<span data-role="name">...</span>
<span data-role="email">...</span>
</div>
{ key: "value" }
{ key: { attr1: "attr1val", attr2: "attr2val", value: "value" } }
<div>value</div>
<div attr1="attr1val" attr2="attr2val">value</div>
What's odd about this is that in Javascript, attributes (aka Properties) can be attached to anything. You want your string variable to have some properties? No problem. You want some of those properties to be functions? Go for it. You want your function to have methods? Enjoy. I'm surprised that JSON didn't include this capability, since it exists in Javascript and is also a big distinguishing factor between JSON and XML.
Getting back to hyperlinks as a datatype, JSON uses "" for strings, [] for lists, {} for dictionaries/objects, and key: value for key/value pairs. Why not add <...> for hyperlinks as a special kind of string? The ... could just be a url, or it could be a url followed by a comma-separated list of standard attributes like rel="..." and label="...". It's a small addition, but I think if it was right there in the markup syntax it would become widely used, and software which parses and works with JSON could depend upon it being there and build higher-level functionality on top of it.
<div data-role="user">
<span data-role="name">...</span>
<span data-role="email">...</span>
</div>
What's odd about this is that in Javascript, attributes (aka Properties) can be attached to anything. You want your string variable to have some properties? No problem. You want some of those properties to be functions? Go for it. You want your function to have methods? Enjoy. I'm surprised that JSON didn't include this capability, since it exists in Javascript and is also a big distinguishing factor between JSON and XML
Now every JSON parser would have to embed a Javascript interpreter to be complete. That would make it completely impossible to implement in a huge number of cases, not to mention the headaches with security (the halting problem, for example) and performance.
The standard says that specific data-* attributes are only intended to be used by the site that generates the markup, but I think it's fair to say that an API using them can give semantic meaning to them for the use by any code using the API, without violating the standard.
Personally, I drank the RDF kool-aid some time ago, and I think the best solution for many services is either RDFa augmented HTML, or Turtle for a more "pure" data response.
No definition of XML does.
> But XLink does
By having the client understand xlink exists and accessing xlink attributes you get links. This is no different than specifying that there are URL values corresponding to specific keys in your JSON-derived media type.
An XLink-containing document is a media type derived from XML in the exact same manner.
> and because XML has namespaces, it can be integrated into any other namespace-aware XML application without risk of collisions: http://www.w3.org/TR/xlink/
Not really relevant to the discussion, is it?
I liked RDF for a while too. I even came close to designing and building an RDF database for my employer at the time. Too bad it didn't catch on.
No, there's no need for it, you can just define it in your own media type, which is expressed in JSON. JSON is a meta-format with a few datatypes, XML is a meta-format with a single datatype (strings).
> Not having this datatype is a big drawback of json for RESTful responses.
No, it's not.
> I consider the ability to use the API using a generic client to be a good test of properly implementing HATEOAS.
That's nonsense. Your browser is not a generic client, it's an interface to a user. There's nothing "generic client" about a user in front of a browser. And as far as the actual users of the API (code) are concerned, your urls could be in `a href` or in `span data-whatever` it makes no difference, as long as they're built with that understanding in mind.
And that understanding is the API's media types definition.
I don't want to define my own media type, because if I define my own media type only clients that are designed to understand my media type can use my API. I want to use a media type that clients can already interact with in standard ways. I want my clients to be able to do something like this:
var url = hyperlink(resourceData, linkRel).href;
When I use xhtml, this works (with jQuery):
var url = $(resourceData).find("a[rel=" + linkRel + "]").attr("href");For example, the JSON Activity Streams[1] is a standard format, supported by plenty of 'big names' (Google, Opera, Gowalla, etc), which defines a schema under which certain keys ("url") are to be considered URLs/links.
A second-best alternative would be a standard way to express hyperlinks in JSON using the existing datatypes. I don't think a key:value pair with a key named 'url' is sufficient though; hyperlinks need metainfo so you can differentiate them from each other and give them semantic meaning, like the 'rel' attribute does in xhtml. I think labels are important too, though they're not always needed.
The problem with this approach is that it's reinventing RDF and XLink in JSON, and RDF and XLink have already failed to be broadly adopted. I think part of the problem with them is the need for an add-on interpreter in addition to the syntax interpreter. If RDF or XLink interpreters were baked-into HTML and XML parsers so that the linking info they provide were as much a part of the syntax as elements, attributes, and strings, they might have become widely used.
But you still need to know what that number, list or dictionary means. By itself, what does [3, 5, 6, 2] mean? Nothing. Likewise, you'll know that string X is a link because you know the format.
And the exact problem exists with XLink. Let's say I'm parsing a user profile and there's an XLink there. To where does that link point to? The user's personal website? To his mother's profile? To a song he likes?
XLink is only useful if you don't care and are just displaying it to the user. Most of these APIs are supposed to be consumed by machines, hence they need to have more structured formats. Having that, XLink is irrelevant.
> hyperlinks need metainfo so you can differentiate them from each other and give them semantic meaning, like the 'rel' attribute does in xhtml.
As I said previously, that's provided by the format, which the client needs to understand anyway. I know an 'url' inside an 'Image' object represents the URI I can access the Image data from. I know an 'url' inside a 'Friend' object represents that person's profile page, etc.
> The problem with this approach is that it's reinventing RDF and XLink in JSON, and RDF and XLink have already failed to be broadly adopted. I think part of the problem with them is the need for an add-on interpreter in addition to the syntax interpreter. If RDF or XLink interpreters were baked-into HTML and XML parsers so that the linking info they provide were as much a part of the syntax as elements, attributes, and strings, they might have become widely used.
RDF is another example of a format that's useless by itself. Sure, you know that X is an URL, but an URL of what? The client needs to understand the domain ontology. Knowing that, it's obvious what is and isn't an URL.
The 'meaning' of the list, within the context of a specific API, is of course specific to that API. That's the semantic meaning I've been mentioning. My ideal is where you only need to document the semantic meaning, and not the syntax that is used to represent that meaning. The syntax should be standardized.
The other part, identifying a certain string as an URL, is much the same as identifying the 'meaning' of the list - it can and should be done by the higher-level format that sits on top of JSON.
I don't count that as a positive, since it would at least prevent them from calling it restful.
> The URI is RESTful. (Explicit collection and member actions, with common namespace.)
URIs can not be RESTful, this does not even make sense.
> Supports both JSON and XML. (Internet media types)
Supporting "JSON" and "XML" is completely useless: they're meta-format, in and of itself outputting "JSON" is completely meaningless as it does not tell the client anything.
Defining the exact media types returned (which this API description barely does) is the vast majority of a RESTful application's documentation. In fact, it should be all of the documentation save for a single URL, which is the API's root entry point.
> Bitcoinica API utilizes three of the four RESTful Web Services methods - GET, POST and DELETE. (HTTP methods)
Better, but still not sufficient to make an API restful. Plus orders are created via a POST, that's not correct, the semantics of POST are updating (creation is PUT)
> But I'm not sure about the hypertext part, because I don't understand what hypertext means here.
And you don't think that might be an issue in your judgment?
> Is Bitcoinica's interface considered hypertext-driven?
No, and being hypertext-driven is the most important criteria of a RESTful application.
PUT semantics are more like "update_or_create". POST is like "create_child".
http://stackoverflow.com/questions/630453/put-vs-post-in-res...
I wish people would read about REST [0] before they use the term. Proper use of HTTP is only one prerequisite of an REST over HTTP implementation.
[0]: http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hyperte... is a common point missed by most folks.