Want a Job? Analytics is the Thing, Says IBM | Hacker News Reader

Want a Job? Analytics is the Thing, Says IBM

businessweek.com

48 pointsmjfern25 comments

Deep statistical analytics is fun. It's also pretty far down the road as a need for most businesses.

My experience is that when people say analytics in the corporate world, what they really mean are simple reports. The only reason it is remotely high paying is that you get locked into a vendor (Microstrategy, Cognos, etc) and have to use consultants with those specific skills.

It's a hard field to break into because people are very hung up on specific tools, eg "oh, you haven't used Informatics PowerCenter 29.4.3 for 6.4 years along with Cognos Magic Report Web Engine 79.63.3 for 9 years? sorry . . . "

As an example of how simple what is mostly needed is: I spent some time working at one of the biggest financial companies in the US. Specifically on a project that was pulling data together from server monitoring (eg resource load), web app use ( hits on each web app deployed by the company), and cost info from the accounting teams.

There were two big goals - use some simple predictive models to understand resource (server) capacity needs in the future and to tie cost per user per app back to revenues generated per user. Amazingly - to me at least - this wasn't being done at any but the grossest levels.

Nothing deep there from an analytics perspective at all. I'm not ignoring that a few companies are doing crazy stuff with huge data warehouses, but most (even very big) companies aren't anywhere close to needing petabyte data analysis.

What is more challenging for most companies is building the capability to do the simple reports. This often involves delicate negotiations with groups reluctant to let you even have the data to build the report - it is a turf thing, eg "No, all financial cost reports come from us" or "oh, that's highly sensitive accounting info and can't be shared." You ignore this political negotiation challenge of analytics at great peril to your project. It is an order of magnitude bigger issue than actual analytics on the data.

Analytics make optimizing the "viral loop" much easier, as described by Dave McClure: http://500hats.typepad.com/500blogs/2007/09/startup-metrics....

Of course when you can break this down by feature and integrate multivariate tests, ad campaigns (inbound and outbound if that is your thing,) and so on.. the numbers can get complex, quickly.

Opinion & gut-based decisions are a terrible way to run a business. Numbers! You should be analyzing your numbers from day 0. I don't think that analytics should be an "eventually" thing, it should be deeply ingrained in your corporate culture from the start.

You ignore this political negotiation challenge of analytics at great peril to your project. It is an order of magnitude bigger issue than actual analytics on the data.

This is absolutely true. For many projects, more time is spent negotiating with various data providers and waiting for them to fulfill their end of the bargain than actually doing something productive. And you are correct, it is mostly due to the fiefdoms that have developed over a long period of time which have then been codified in layers upon layers of politics.

I like analytics as a field but I rather doubt I'd like IBM's conception of analytics as a career. They're looking to provide analytics of middling complexity -- one step above what any idiot can get out of the tool -- as an outsourced service for the cheapest they can possibly get it. The cheapest they can possibly get it is not you. Sure, if you're a Google Analytics or urchin jockey at the moment, you probably have a few years before India/China catch up to the high-demand-low-required-skill opportunity, but once they collectively do, you'll be in the bread line next to web designers who primarily slice PSD into HTML.

If you wanted to do it as a career, I'd think less chart-creating-monkey and more "analytics architect" or maybe "marketing manager with deep skills in analytics to coax extra millions of performance out of our CPC/etc campaigns".

In India, these days there are tons of analytics companies. Mostly what they end up doing is doing extremely simple number crunching mean, median, standard deviation, etc. on survey numbers or other marketing data. The salaries are not that high and as you say, I expect it to become a commodity in near future.

What is sad about analytics is that they would teach you how to use tools first: SPSS, Excel, SAS, etc. but there would be little or no fundamental understanding of concepts. Sure, data mining is a buzzword. Load some data and do a bunch of super cool analysis, but where is the insight? where is understanding of what is really happening under the hood?

You can teach tools in a month, but you cannot teach analytic skill.

Yes, in a month that is. Of course deep understanding of analytics takes time, but it can be learned (but not within a month).

You need one guy with the skill to write a bit of code and a hundred more to do the legwork of plumbing it into the source data for each customer. The latter is what you bill for, and it's commoditized (e.g. "FTP a CSV file from here, run this program, FTP the resulting PDF to the webserver then email the customer to say its done").

.../archives/2009/12/want_a_job_anal.html <- That's the worst case of SEO readable URL I've seen in a while.

I've e-mailed it to a few friends interested in this field and I've already had to tell them ignore what the URL says, it's not what it looks like. And just pray that it doesn't trip up some of their company filters.

this is one case where using a url shortener for email links actually would be advisable.

They should have used an URL-lengthener. Just look what the shortening did to `analytics'.

Want to pay less for analysts? Convince a bunch of people to learn analytics.

In IBM's case they want to pay less for analysts (in their services decision) and flog other companies lots of new hardware and software for their own (cheap) analysts to use. They did the same thing with COBOL back in the day, then the same thing with Java, this is business as usual.

"Want to pay less for analysts? Convince a bunch of people to learn analytics."

Why? Sales commissions are probably about the same today as they were in ancient Greece. I think the rule here is, "Always overpay with free money."

Hal Varian has been making the same point for a while now: http://www.nytimes.com/2009/08/06/technology/06stats.html?_r...

I am sure there is a need there, but I am always wary of vendors looking to define fields of study. One extreme example was a school not far from here that included CICS courses in their computer science (or did they call it information science) curriculum. The problem with vendors (IBM, MS, Sun, Oracle) is that their idea of a workforce is not necessarily one that questions the underlying technology or career definition. Correct implementation of floating point does not owe its genesis, but to thinkers like Knuth and Dijkstra who loudly pointed out the obvious deficiencies.

And as is pointed out here in the thread, the vendor's motivation might be to lower the cost of analytic talent. And wholesale renters of programming talent are fine with this too--no enterprise is looking for programmers to tell them that "you are doing it wrong".

Does anyone remember the advertising for COBOL, that it was nice and english like and that now your accountants would be able to do your programming and you wouldn't need to deal with those pesky goofy programmers?

Or http://en.wikipedia.org/wiki/CA-Telon. I remember one enterprise that finally decided that its paperless effort wasn't going to work, so it laid off 300 Telon programmers.

What I like about the world today is that programmers are seriously involved in these decisions. Did Dennis Ritchie say something that 10 years after Unix, that somehow marketing genius at AT&T decided that it was a good thing?

Wasn't VBASIC invented to make it easier for programmers with less training to do programming? How has that worked out for us?

We've built a business partly around automatic analytics - our approach is to keep the headcount low and the value high.

I wonder if the demand for analysts is a symptom of business waking up to the possibilities, while automated analytics is not mature.

Yes, businesses are definitely waking up with regards to data. They are suddenly realizing that their various systems are currently or are capable of generating shit loads of valuable data that the business has no idea what to do with. Executives and managers learn through conferences, industry events, etc. that OTHER companies are using this awesome data to generate BILLIONS or save BILLIONS etc.

When you see a company like IBM thrust itself into a field like this, you know that the field is entering mainstream business. This is a perfect time to position a firm as a "niche analytics firm" that focuses on high-end projects where IBM just can't compete.

OK. I'll do analytics for $135/hour. Email in profile.

I will do it for $134/hour, email also in the profile. In fact, I will do it for max(N-1, 5) dollars, where N is the lowest amount anyone else is willing to do, except for Fortune 500 companies, where I will do the work for free, or sites with PageRank > 6, where I will happily take a backlink in exchange (restaurants, hotels, airlines and strip clubs can also barter for my time.)

This is what happens to "hot" industries where the pay is far larger than the training effort required. Anything that takes less than 1 year of training, or where someone can BS their way in, is prone to "talent" saturation. Usually the opportunists like myself underbid specialists like rms or drown their voices out with big egos and self-promotion.

Let me add that no one should ever hire cheap foreign Analyticists. As soon as my rates go below $50/hr you can count on me going on the offensive, writing xenophobic articles for Nation's Pride, True Blue American and Kiss The Flag magazines. And once there is momentum we will form a lobby to mandate analytics usage throughout the government (a 30B industry, hopefully) and make sure only people we have accredited, for $999/year, are able to work there.

In all seriousness, this is the script for every "hot" industry out there. It starts with an opportunist out to make a buck, and it ends up being a matter of national interest. Whenever crappy companies can no longer compete, they appeal to government. First the U.S., then they start exporting those "vital U.S. resources" overseas and sell it to other governments as well.

Despite the institutional difficulties that you rightfully present, I'd like to see more analytics used in governing.

Note that Fordham is basically going to provide job training... basically IBM is looking for a high tech version of a vo-tech to train people in the IBM way.

And as soon as IBM can, they will ship the job over to the Philippines where they will pay a decent wage of $500/month.

(When I was in Manila in August I saw multiple billboards advertising jobs at IBM).

Does anyone have exposure or experience with stanford's Data Mining and Applications graduate certificate? Details here: http://tinyurl.com/stanford-graduate-certs my email is in my profile if you'd rather respond privately. I am considering applying.

IBM brought Cognos last year (while SAP acquired Business Objects) so, that is just a promotion.

I was going to bring this up myself. Basically, they are hiring people who are experts in BI in order to do consultations for other companies. That way, they can hire 2500 people who already know Cognos BI (which IBM owns) through education and then send them out to companies to do consultation work.

It's a brilliant idea, and they'll make a LOT of money.

Reply on news.ycombinator.com