so the whole thing was just an exercise in choosing a cut point? It would seem like you might want to revisit this sort of thing every now and then, what fer since both web "frameworks", your parsing code, and CPU hardware change over time.
1,638 karma · joined August 24, 2007
Take away the profit motive and it's astounding how much more reliable the trials get. Of course reviewers whine about how they're often equivocal or negative, because the thought of the literature being horribly biased either
a) has never crossed their mind (too busy with protocols),
or
b) is precisely what they want, for "showing progress".
So in order to keep up the appearance of positive results and keep the gravy train flowing, the trials that get published tend to be the ones that "show" "progress".
Mind you, these are clinical trials with preregistered endpoints and protocols. Pretty much the entirety of experimental science outside of physics is much worse. So what I'm telling you is that this is the BEST-CASE SCENARIO for an awful lot of science.
I'm not going to write up "my story" beyond this, because it's the same as most everyone else's story in the field. Killing humans for profit is not my idea of a good time, even if adult trials tend to pay better than the Army.
YMMV. I may be jaded, but I still believe there are populations that, through no fault of their own, end up with few or no options. There is room in this world for integrity -- but get (real) money involved and the bad money drives out the good. The linked paper is dead on.
http://www.nature.com/nature/journal/v497/n7450/full/497433a...
This is the guy that published the irreproducible results paper whose results were (of course) irreproducible (since it did not name any). I.e., "trust me on this", the opposite of science.
Note above that you can pay for more of his sage wisdom, thanks to the generous NatureMacSpringer megaconglomco.
Post everything as a preprint. Submit your trial results to OpenTrials or OpenFDA. Quit killing patients. Maybe work in pediatrics so that, right or wrong, you still won't make any money, so you might as well not kill any excess kids.
I'm not bitter or anything, after participating in clinical trials and analysis for over a decade...
Where would you not have access to a PLoS paper? The entire company is dedicated to open-access publishing.
First link in Google pops up the PDF, FWIW.
http://robotics.cs.tamu.edu/RSS2015NegativeResults/pmed.0020...
(It is a wee bit disturbing that first link isn't to PLoS)
Post all your papers as preprints. Don't bottom for some grumpy old PI. Academia is not scholarship -- the latter is frowned upon within the former, as it takes longer than simply churning out some useless publoid garbage result.
Be like Faraday, Newton, Varmus and Eisen... not like Bem.
Not sure how someone would actually file an infringement lawsuit for not eating, though. This therapy has a little tiny bit of prior art (e.g. several hundred million years).
Patent applications and prosecutions are difficult, much more difficult than I think most realize. Try it yourself (be sure to have at least a spare $13000 lying around) and come back in a year or so with your scars, if you're one of the folks randomly sniping about the process. It's far, far, far from perfect; but it evolves (see for example Myriad, Prometheus, and Alice for important recent cases) and the worst mistakes do get corrected.
JMHO
It does. Although the graph processing use cases that I've seen recently (and their developers) are better served by a faster query engine than e.g. Spark has provided.
Most people, if they can get away with Excel, won't use an RDBMS. (Of course this means that eventually someone will have to come along and scrape all the damned spreadsheets into an RDBMS, but...)
Most people, if they can get away with an RDBMS, won't use Hadoop. (Of course, sometimes you end up with something like DB2 file pointers, which really just say "hey look here's a pile of unstructured data that we couldn't figure out how to handle, and this is where we left it", and then of course someone has to put it somewhere useful, but...)
Now if you're moving around copies of the Internet, or trillion-row "databases" that need nearly instantaneous OLAP, then yeah, you'll be needing a proper distributed infrastructure. However, sometimes you can just rent that proper infrastructure from a vendor with that problem (e.g. Google or Amazon) and then you don't have to support it.
Things get really interesting (as in bleeding edge research interesting) when none of the above solve your problem. But they also tend to push the time horizon for results way out.
JMHO. Eventually software eats everything. It's a question of time scales. If you need results next week, don't rebuild TensorFlow or Redshift from scratch.
I keep hearing about Hadoop and Spark for graphs and models, but in actual practice (WHICH DEPENDS ON THE SIZE AND SHAPE OF THE DATA), an awful lot of data-feeding problems are more easily solved by columnar data stores, or graph data stores. For text mining and NLP, you are almost certainly better off with redundant unstructured storage due to the fundamentally unstructured nature of text and communications. For massively parallel model exploration sometimes it's better just to spin up a bunch of huge EC2 instances. Hadoop and/or Spark aren't necessarily critical aspects of solving the problem. Once you get a redundant, distributed infrastructure to run things on, you may not need to jerk around with name nodes and boring HA grunt work. Sometimes an existing purpose-built implementation (often on top of well-oiled infrastructures) does the trick. Sometimes not.
One interesting project I've kept an eye on (OK I lied, I'm a contributor) is storage of genomic data against a distributed, graph-structured reference. It's really fucking hard. Even with Google, UCSC, the Broad, and the usual heavyweights involved, the milestones have occurred on a timescale of years, and the eventual adoption is projected on a scale of decades. This is with some of the best in the world working on it, in every time zone. Again: DECADES.
So... I'm not against distributed computation. But you had better damned well know what you're getting into. I worked at Google. I work on GA4GH. If you can afford to take the long view, maybe your problems are complicated enough to invest in that sort of infrastructure.
But maybe they aren't, and Amazon or Google or Microsoft has already built what you need because they needed it too, and they hired a hundred graduates from the best engineering schools in the world to support their implementation. Is it worth reinventing the wheel when there are people racing at Formula 100 level out there? Only you can answer that.
There's a reason people don't swat flies with Buicks. Sometimes all you really need is a flyswatter.
Why can't people evaluate the evidence and THEN decide?!?
It feels like these sorts of things make polarization worse.
jesus fuck, this is NOT what I need to hear as we push our salvage therapy companion biomarker into a 160 patient trial.
then again I work in pediatrics these days and nobody ever made any money in peds. So I guess there's that. People expect their 99-year-old grandpa to die of prostate cancer (or apnea, or constipation, or...). It's rather a more unpleasant shock when your 3-year-old does the dying.
Won't make anyone rich but maybe it'll relieve a little suffering in the world. I guess I'd best be OK with that.
Hadoop is a marvelous way to make simple things complicated. Look at what Google does internally (and has been doing internally for many years) with structured data: stick it in a fast structured place (bigQuery, or even MySQL before that, if you go back far enough with AdWords) on a redundant platform and be done with it.
Hadoop, Spark, etc are for unstructured stuff where you're not sure how to attack the problem. If you know how to store and query the data then you know how to attack the problem.
UPS had billions upon billions of rows in their package tracking system long before anyone came up with MapReduce at scale. If you've only got 1TB of data cropping up per year, it's reasonable to stick it in a database of some sort.
If you've got 5PB of new data a year, then you need to start thinking about moving the computation to the data; but even then, sometimes that just means writing stored procedures.
Eventually everything interesting gets tainted. Hence the suggestion never to run anything with any more ACLs than necessary in production systems that touch non-secured bits
I haven't the foggiest idea what people do today in php but if they don't used cached templates everywhere I will be sad.
Even back then we serialized into and "rehydrated" out of memcached and the database (which memcached pulled from on cache misses). It's honestly more of a pain in the ass NOT to use serialized data structures in the long run. It's a good habit that is hard to break once established :-)
As far as AST manipulation. Heh. Betcha 9 out of 10 front end programmers have never written a compiler in their life. Again a good habit (somehow this topic always loops around to either lisp or Parsec) but a bit harder to establish. Template metaprogramming is a great way to get in a logit of trouble fast unless you know exactly what you're doing, IMHO.
(asterisk) wrote and sold a website in php. I don't judge people by their tools. But a lot of things about php sucked, at least 10 years ago when I stopped needing to write it.
In all seriousness I've taught postdocs who were willing to learn everything from Python to C++/MPI to assembler. The key is just to be intellectually curious and enjoy solving problems. R is very often the thin end of the wedge -- its learning curve is more like a wall, but once you've survived that, most programming language quirks seem like NBD. And lord knows the real physical world is far quirkier still.
R is ugly as shit (ok not as bad as Perl, and it is lispy enough not to be infuriating) but it gives you an enormous arsenal of tools. It's not like R is unique in this way, any decent programming language enables this, it's just that R and CRAN seem to have a working implementation of most anything an analyst is likely to need; if you need speed you call out to C++ or FORTRAN, it's not beautiful but it gets things done. So more of your time can be spent in the pursuit of understanding, to poke and prod, visualize your results, solve problems and (sometimes) discover things no one else has seen.
If someone DOESN'T like those last two things, what the hell are they doing in academia anyways? Choose another line of work and make a lot more money :-)
Oh and it's fast as shit (the tight loops are in C++).
It was certainly interesting reading, though. Thanks for the link. I learned quite a bit from reading it and the linked main Inquisition page. It does seem to be used as propaganda out of proportion to its true toll.
https://www.overleaf.com/help/17-is-overleaf-open-source#.WB...
An open-source business that doesn't survive (cough ReplicaDB cough) isn't necessarily better than a closed-but-extensible one that does. LaTeX itself is, and anything you're working on within Overleaf can be exported as text at any time, so I don't worry so much about lock-in.
The underlying toolchain (LaTeX and packages) is fully OSS, and isn't going anywhere; I have supported the LyX project in the past, and indeed that's how I laid out my dissertation, but in the end I find Overleaf more usable.
I still wish they'd support Markdown, though. Sometimes I just want to run off a quick, properly typeset series of equations and lists/tables without all the boilerplate. I suppose you could make the argument "well, if it was OSS, you could just patch it yourself!", and you wouldn't be wrong. But we use the autosubmit feature a lot, so on balance I've decided I can live with this shortcoming.
A side benefit of entire populations grown weary of fairy tales, now the land formerly reserved for tax-exempt politicking can be repurposed for economic growth that benefits even unbelievers.
The irony of this occurring in the home country of the Spanish Inquisition is (to me) very satisfying. The arc of history is long, but apparently it really does bend towards justice.
Be the change you want to see in the world ;-)
(Ha ha only serious. Being an idealist made me cynical. Then it brought me back again. In the end we only have each other, those who we live and love and work with, in this world. It may as well be the best world we can make it.)
I spent the same amount of time (72 hours) getting one hairy equation laid out in Word that I spent compiling my entire dissertation in LaTeX. Naturally the former was a request from my adviser, who did the latter back when she was a student. Dammit.
If you have any serious math in your writing, nothing else comes close. (Protip: Mathematica, the only non-free software package that ever earned its keep for me, will directly export equation cells as LaTeX if you ask nicely. I can't stand doing tedious symbolic manipulation without it.)
http://clincancerres.aacrjournals.org/content/5/9/2281
Simone's Maxims are possibly the best piece of career advice anyone ever gave me.
At least she's learning this early in the game. She's clever and tough and will either go far or be smart enough to make much more money at a real job.
Her prof? Not so much.
You know, like the fruits of searching Google for five seconds to try and locate "the offending phrase"?
What kind of a fucking fossil can't be bothered to at least substantiate their bigotry instead of ranting themselves out of a job? (And no, tenure doesn't cover obvious failure to abide by university policies. Of course it could just as well have been some adjunct making $8/hour)
"Hey there, you with the beaner name. I have absolutely no evidence of wrongdoing on your part, so go fetch me some"
What a twit. I'm not a particularly nice person and have no problem failing students who won't put in the work, but this is beyond the pale.
Yeah see I don't worry so much about this. I purposely follow people with whom I disagree (but who are well spoken) on Twitter to avoid ending up in an echo chamber. So I don't disagree with you about quashing diversity; the last thing I need in my life is to preach to the choir.
But the fact remains that, 1st Amendment or not, you can't yell "Fire!" in a crowded theater and then plead "but, but, my rights!" to evade responsibility. Credible threats are assault (look it up, it's in every state's legal code). I would not be surprised if doxxing was found to fall under similar statutes by an enlightened judge if done with malice aforethought.
Furthermore, an individual's rights to free expression do not include a right to make Twitter spend money or lose opportunities due to their chosen expressions. Twitter != the government. If push comes to shove, their self-preservation must take precedence (legally, since they have a fiduciary duty to their shareholders as a public company).
It's a chestnut, to be sure, but freedom of the press applies to those who own a printing press. Until and unless Twitter is nationalized (heaven forfend), they're restricted (at worst) by common carrier provisions. They can do a good job if it's in the best interests of their shareholders (and recent events suggest that may be the case). It's really an issue of how (how to maximize shareholder value, how to improve public perception, how not to drive away ads...).
Advice to the OP: don't be a martyr. If it looks like you can make a difference, go for it! Just don't pin your identity or sense of self on whether you succeed. The markets are a harsh mistress.
I'd love to be wrong but I've seen too many promising ideas die in clinical trials. Let's hope for the best.
Next time I teach (actually the past few as well) I'm not going to bother with a textbook. Go pull these primary research articles, pull the data, and see if the methods were used correctly. The end.
As previously, I've had some surprised looks when students did this with biological data sets. It's not like psychology is some rogue's gallery; it's a widespread problem. Only solution is to verify results before building upon them.