Ask HN: What are the current (IT) industry skills employment cash cows?
What do you think are the (IT) skills that employers currently pay loads of cash for and are widely searched ?
Spring, LINQ, DB scaling, BPMN, others ?
Spring, LINQ, DB scaling, BPMN, others ?
I'd go with something that requires specialized knowledge, either domain-based (bioinformatics, geology, finance), systems programming (compiler, OS, and distributed computing), or one of the AI subfields (information retrieval, machine learning, data mining, computer vision, etc.)
Stuff that can pays high six figures because they can't outsourced. Computer security (especially if you could get security clearance); geology simulation for oil exploration (high-level domain expertise knowledge); quantitative modeling and trading system design for HFT (domain and quant skills required); risk modeling in actuarial science.
All of these are if you want to work for BigCo. You could also make a lot of money by going consulting in some obscure and corporate package such as SAP modules.
Every idiot out there wants to spray some "big data" retardation all over whatever they're doing, so they throw together a cluster -- if one were really generous -- of a handful of machines so they can use hadoop too!
Meh. It's not big data until you need a couple hundred cores.
However -- it's an opportunity to cash in for devs. Of course hadoop is a needlessly difficult programming model for most people, but never mind that. My .02 is if you can possibly solve your problem by scaling up instead of out, then scale up. Hadoop works well precisely for perfectly parallelizable no communication required problems, which are few and far between. For everything else, it's a giant pita to shoehorn the solution into hadoop. Of course, most managers are retards, so spending maybe $25K on a 256GB ram, 16 xeon machine stuffed full of 15k rpm sas or ssd drives is out of the question, but wasting months of dev time to sprinkle Real Big Data (TM) bullshit on stuff is worthwhile. Sigh. 15 years into this industry and still, everybody views salary expenses / opportunity costs of dev time and equipment as completely separate things.
I'm grumpy today.
However, hadoop has a steep learning curve for anything but toy problems, and I get 2-3 requests a week on linkedin just because of my employer and having hadoop on my resume. So it's great for devs. I know people charging hundreds of dollars as their consulting rates.
Edit: just checked silicon mechanics to make sure my hardware cost wasn't too far off. If you're doing most numerical stuff, parallelization is hard, so: 256GB RAM, 2 of the fastest QPI highest cache procs, and 6x intel 160GB SSDs in a 2U: less than $30K delivered.
It took me a week's worth of work to write a distributed ranker that will work on billions of ids in hundreds of sets; the algorithm is not obvious. On a single node you'd write a for loop that did a sort.
Those are the sort of obstacles that the hadoop programming model introduces. Again, if you haven't used it for real work, you'll be surprised at how few tasks are truly parallelizable without communication. The model is very simple, but accomplishing tasks that don't quite fit in the model is difficult.
Another example is a matrix mult. There is no way to segment the data to eliminate internode communication. It can be accomplished in an iterative fashion, but it's decidedly nontrivial. Or most optimization problems -- I don't think there even are known algorithms for no communication parallelization.