functional_test

As others have mentioned, you can find variations of this around online, but if you wish to roll you own (perhaps to add other features or what have you), you could knock it out in about 20 lines of Python using the built in CSV library and Openpyxl [1].

[1] https://openpyxl.readthedocs.org/en/latest/

Did you actually read the article? Here's the most relevant quote:

> The reality, however, is that China has struggled to create enough white-collar jobs for its soaring population of college graduates. In mid-2013, the Chinese government revealed that only about half of the country’s current crop of college graduates had been able to find jobs, while more than 20 percent of the previous year’s graduates remained unemployed.

> The bottom line is that when two parties enter into a contract voluntarily, that should be the end of the matter.

That is a very black and white view; real life has a certain nuance that just isn't captured by blanket statements like this.

You'll miss things if you limit yourself to a single level of abstraction: while it makes sense to the two parties to look at it like that, there is also a larger-scale, societal interest that certain sorts of contracts not be allowed. Thus, society makes rules about contracts and employment. For example, we ban slavery, indentured servitude, and child labor because as a society, we've determined that those are exploitive (even if you can get children who would willingly work for you, or people who would sell themselves to you).

These sorts of laws are what make us a society. For an example of a what happens without them, take a look at Somalia.

Find an employment attorney. Pay that person for an hour or two to read the contract for you. They will be able to offer much better advice than HN.

Thank you for posting this (it was quite a surprise to see his name here).

I was a close friend of Emil for a long time. Before he passed, I had gotten busy with work and hadn't spoken to him in a while. I saw some of our mutual friends the weekend before it happened and had planned to call him. Really wish I had made that call sooner.

Exactly. It's so general that he's on base this time, but it's dubious if he actually has the experience to go into more detail.

Take what Jonathan Kinlay says about HFT with a grain of salt. He worked at a friend's firm briefly before leaving without making a single profitable strategy (although his excel models were profitable of course). While what he says in this article is in the ballpark of correct, he is not the best source of information on modern HFT.

Wikipedia also has a good discussion of the basics as well as some generalizations:

http://en.m.wikipedia.org/wiki/Fractional_derivative

Turns out there are even some applications, albeit rather esoteric ones.

I did not. I described canceling orders as no big deal. Spoofing is illegal and very much a big deal. Spoofing implies canceling but canceling does not imply spoofing.

In the example I gave, the spoofer's intent is to encourage people to cross the spread into their resting order to avoid paying the spread themselves.

On the other hand, say someone has been resting an order for a long time, for example to buy at 9 because they think that's a good price. Until the bid is at that price, they are unlikely to get executed so they'll keep the order regardless of their position. But maybe they have a large long position on when the bid reaches 9, so they decide to cancel their order to prudently manage their risk by not buying more. This is obviously important in a healthy market -- firms that fail to manage their risk run the risk of cascading failures (if their clearing firm can't cover their losses.

It's pretty easy to tell one from the other most of the time, especially for regulators with access to account tagged data.

This is blatant misinformation. Please stop spreading it.

Canceling orders is something _all_ participants do. It's a vital part of risk management and a healthy market. Some markets do have minimum quote lives, but they typically fail as market makers cannot manage risk there, so trading moves elsewhere. The only time I've seen a market place survive adding MQL's is when they were on the order of milliseconds, not minutes.

There are certainly strategies that are manipulative. For example, having a hidden order on the ask and posting an inflated bid then pulling it once your ask gets filled. Since you posted it with no intention to get filled and with the intention of manipulating the price, that's spoofing, and illegal.

But the fact that someone placed an order and then canceled it when the price got close means exactly nothing -- everyone does it as a part of normal business.

Ah yes, downvotes outnumbering real responses. Classic HN.

I did miss that actually. Thank you for responding to me with something real. I'll reply again once I've had the opportunity to take a look.

That is not a valid comparison, and as someone who has been affected by drunk drivers, I take great issue with your trivialization of a serious issue.

But I'd expect nothing less from HN.

I only brought that up to counter his claim that data loss is inevitable. Of course my anecdote doesn't mean it's not common =) But anecdotes are all anyone else has, and every time I've read one about someone losing data, they either hadn't read the documentation, or just didn't understand the semantics of what they were doing. Very very rarely, especially these days, has it been an actual DB bug (though I will admit I got Mongo to core one time on 2.4 doing a compaction).

And it's a little disingenuous to point at the issue tracker -- as you say, everyone has open issues. The specific things that are mentioned though have been fixed: writes are checked by default now, the global lock has been broken up into per table locks, etc. There may still be common issues that aren't being addressed, but if there are, I'm not aware of them.

TokuMX supports most of Mongo's features besides the aggregation framework last I looked. Better locking, transaction support, better indices too -- you should check it out.

This is the last one of these posts I'll ever respond to, I promise. And I'll give you the same response I've given every other time before:

You should not base your decision of database (or anything else for that matter) on marketing copy. For something as important as your primary data store, you should at minimum read the full documentation and run some tests with dummy data to see if it will even plausibly work for your use case.

I used MongoDB successfully for years with a large data set (>1TB) and 100% production uptime for more than 3 years. I never lost data. Your claim that you will unavoidably lose data is baseless and without merit. In fact, every issue you listed has been fixed, again, counter to your claim.

Personally, these days I prefer TokuMX if I'm looking for something compatible with MongoDB, but these baseless attacks on MongoDB have to stop.

EDIT: Every time I make a post like this, I get some downvotes without responses. Please tell me why I'm wrong. If it's just that I'm abrasive... Well, you would be too if you were addressing the same thing for the Nth time.

Dang, I have the highest respect for what your moderation has done for HN.

That said, unkilling this is just wrong. This sort of drivel is what I'd expect on Tumblr -- nit picking minor things, focusing on people's phrasing rather than meaning, not accepting common terminology as if that advances some sort of social justice, and asking people (inherently Bayesian creatures) to ignore their priors for the sake of "equality".

These articles detract from the real issues, and serve only to further separate women in technology, rather than integrating them. There's a reason this was flag killed (and it was not sexism).

I reply to these posts on HN relatively frequently these days. These sorts of accusations about Mongo are out of hand, and not congruent with reality. I say this as someone who used MongoDB extensively.

I don't have time to address all of the problems, but let's look at #1 on his list:

1. MongoDB issues writes in unsafe ways by default in order to win benchmarks

This is objectively false. At the time, Mongo chose to turn off write confirmations by default -- this was clearly documented in many places including introductory documentation. Surely no one would have deployed a database without even a cursory skim of the docs, right? The notion that it was done to win benchmarks is baseless. Furthermore, in response to the community's reaction, the default was changed.

This is a recurring problem with the complaints about Mongo. You cannot use ad copy to evaluate a database (or any product for that matter, but definitely not something as important as your primary data store). You must read the documentation. You must do a trial run and make sure it will meet the needs of your use case.

You can't be upset about the results you didn't get from the work you didn't do.

Canonical released Pollen/Polinate a while ago to address this problem. The idea is that each virtual machine queries a pool of entropy servers to seed it's own generator, and as log as at least one entropy server isn't compromised, you should be fine.

Can these kind of comments stop? Please?

This has been trumpeted pretty much constantly on HN, and even if it's true, does it matter? Is anyone seriously evaluating their database on the basis of marketing copy? I suppose if you do, you can't be surprised if you have a bad time.

Isn't it right there in his comment? Seems that he doesn't like the idea of having to work for free to "rank higher" if recruiters start to use these sorts of scores. Since no other profession has to work for free like this(see: the carpenter analogy), it seems a reasonable position.

I can see where you're coming from. For my particular apication that's not a concern (and I have sequence numbers to know when I've dropped data).

I'll start by saying that I agree with almost everything you've said. I can think of one time where a loop with a sleep might be appropriate though -- if you have something, perhaps some IO channel like a log, and you want it flushed regularly, but not every time a write is performed. With a long enough sleep (>100ms) the performance overhead ought to be low. Is there a better way to design that though?

I had ~3TB before I migrated to TokuMX (same semantics, but at the time much faster indices and lower disk space requirements). It was replicated twice, although I could let reads on slaves fall behind a bit.

Of course it's not good for everyone's use cases though. Nothing is. Did anyone honestly believe that? I mean, even with the hype, did anyone truly decide to not evaluate their _database_, and believe the ad copy blindly?

Ultimately, I wouldn't have chosen MongoDB if it was bad for my use case. I tried it and other things -- read the documentation, made benchmarks and test cases -- and chose it after a period of evaluation. To do anything less for your database seems irresponsible (I'm looking at you, people who were surprised about the original default write semantics because you didn't read the docs).

Tangentially, the personal attacks ("which is fitting if you are a MongoDB advocate I guess", "that said, I do feel bad for your likely eventual replacements who have to clean up the mess from your poor choices") add nothing to your argument. They make you look foolish. Also, my replacements are still using my system (quite happily), over a year after my departure.

I had ~3TB and it was all on hard disks (although we had quite a bit of RAM). Ultimately I switched to TokuMX which substantially reduced my disk foot print, but I was a very happy MongoDB user.

My use cases may have helped (they were very appropriate for MongoDB, but I suppose I would have chosen something else if they weren't).

This is objectively false. I used MongoDB for >3 years with 100% production uptime and a very large data set.

I get that some people have had bad experiences with Mongo. Sounds like you're one of them. Why not share your experience rather than just spread FUD?

It's worth noting that a lot of the complaints people had about Mongo were with things that were clearly documented (e.g. no confirmations on writes by default back in the day).

You may also really like TokuMX. It has the same API as Mongo (even works with Mongo drivers) but way lower disk use, faster indices, and better query semantics (e.g. updates don't affect the cursor you're iterating right now).

After using MongoDB for years (very happily too, I'm not one of the anti-Mongo crowd), I ended up switching and it's been great.

This sort of snark isn't constructive. The situation is obviously much more nuanced than this, and while this sort of reduction is cognitively convenient, it's not accurate nor useful.

For example, when the money was obviously not going to be repaid, but the Troika continued to lend them more -- at what point does the lender become at least partially responsible? Obviously I don't advocate irresponsible borrowing, but certainly irresponsible lending is also bad.

Ultimately, the bond holders are going to have to take a hair cut, or have their Euros devalued, because the Greek economy cannot possibly hope to repay its current debts. To think otherwise is to hope foolishly for the impossible (all questions of right or wrong aside).

Overall that's great to hear, but I can't help but feel that it's too little too late. Once they fix the generics, are they going to go back and fix the secondary issues too (like the issue with Doubles)?

I know this isn't a popular opinion, but these correctness issues really matter. For example, you can see the impact these choices in the JVM and the standard library had on the Scala community, ultimately forcing Paul Phillips to fork the compiler.

Java actually doesn't have generics. It has syntactic sugar for generics, but they are so horribly broken they can't be fixed.

For example, because the generics are just sugar for casts, you can't specialize them. Because you can't specialize them, you have to put behavior in the "wrong place". Take for example a hash map from Double -> T. Because IEEE floats don't actually form an equivalence class (NaN != NaN), they don't make great keys to a hash table (if you put NaN in, you can't get it out). But, rather than specializing the hash table for floats to give special NaN handling, Java had to make Double(NaN) == Double(NaN), so when they're boxed, they do form an equivalence class.

And if that were the only inconsistent behavior in primitives versus their boxed counterparts, that would be great...

EDIT: To those who downvote, would you discuss your objections? I understand that I may come off a bit caustic, but having been burned by the myriad correctness issues in Java, it's hard not to. These aren't abstract problems that don't come up in practice -- for a public example, look at what Paul Phillips has to say about the JVM and standard libraries.