Comment by uptownfunk | Hacker News Reader

Comment by uptownfunk | Hacker News Reader

I work in this field.

I am expected to serve clients in a lot of different domains.

When we were doing data science work for an insurance client, I stayed up late for weeks reading about actuarial models and how things currently work as well as learning all the jargon there is in the industry. I could probably pass the highest level actuarial exams at this point (or at least not fail too embarrassingly) [edit: OK I probably couldn't do this, but I could probably pass any/all exams related to the quant side] and also innovate superior premium pricing models using "Machine Learning". Not because I want to become an actuary but that's what's required to do good work.

When I worked in pharma, I learnt from other PhD's/PostDoc's on my team about oncology, a very specific type of oncology in fact, and everything that goes into Pharma companies and their marketing efforts, how doctors operate and behave. Not only that, but I learnt from an industry expert on all the nuances and subtleties that involve analyzing various types of medical claims data. (Hint: It's a total CF)

I could go on and on about the different domains I've had to work in. But the whole point is, being a generalist, in the sense of a dabbler, is utter nonsense. If you want to be a data scientist, you have to be flexible enough to work in any domain, and also have the gumption to become a specialist in the field, do something new in that domain with your shiny "machine learning" knowledge, while making sure your models are not GIGO from spurious statistical assumptions, and making sure you know how to code decently enough that your algorithms/code doesn't shit itself. This probably aligns closer to what the article was actually talking about..

(Edit: Sorry for the confusion, in my world the word generalist has a totally different meaning..)

> [edit: OK I probably couldn't do this, but I could probably pass any/all exams related to the quant side]

After "staying up late for weeks" at some indeterminate point in the past? I'd be amazed. The pass rate for the quantitative finance exams is ~35%, and that's among people who stay up late for months, not weeks, specifically to study for those exams, after having spent years of their careers working on quantitative financial models at insurance companies and pension consultancies.

I'm calling this out not to nitpick about the difficulty of actuarial exams, but to make a broader point: You are a generalist. This is what being a good generalist looks like. Developing knowledge that's on parity with specialist practitioners in a field is an upper bound, and it's not really practical to attain for highly specialized fields. But the amount of domain-specific knowledge that's needed varies widely from project to project, and needing to read up on the subject after work for a few weeks is well within that range.

> this generalist thing is utter nonsense

By your own description, you sound like a generalist with ability to dive deep into the areas needed. You seem to refute the core premise of the article but your description says otherwise.

Honestly, I'm just put off by the word generalist. It makes it sound like just because someone can call a few APIs and create a deep learning model, that, as long as they have a clean data set, all of a sudden the world now revolves around them. Frankly, a lot of it is because of the hype, all of a sudden data scientists are these magical leprechauns that by virtue of their fancy algorithms can make money appear out of thin air.

It really isn't like that, you really have to be able to go deep, as you said. Yes, there is a part of it where you have to be flexible (I wouldn't call it general, because I think it sweeps too much under the rug), so as to go so deep into a topic, you pass this kind of "expert-level turing test" where, were you and a domain expert put in the same room, a reasonable person wouldn't be able to tell you apart, where the weaker version is "another expert wouldn't be able to tell you apart" or something like that..

It sounds like you equate 'generalist' with BSer, but the article's definition of generalist matches what you're advocating.

> Specialists’ work is coordinated by a product manager, with hand-offs between the functions in a manner resembling the pin factory: “one person sources the data, another models it, a third implements it, a fourth measures it” and on and on.

If you're taking the time to learn the domain, source the data and clean it then you fit their definition of a generalist.

The author of the HBR article works in a subject where literally nobody can be a specialist of the type he needs unless they work at his company. He's basically saying "this is the kind of data scientist I need; someone flexible enough to learn about my weird problem." He's also requesting a DS which can do everything from soup to nuts. That doesn't mean it's good for a DS to be a generalist.

FWIIW I am a generalist DS type (arguably just generalist: database engines, blockchain, ML, DSLs, etc), but it's almost entirely because I'm easily bored. From an optimization of my paycheck and hours worked point of view, going deep in one area nets much larger paychecks and a nicer (aka fewer hours) lifestyle.

> I could probably pass the highest level actuarial exams at this point

You must be an exceptional intellect, then. Actuaries famously devote thousands of hours and years of their lives to passing the exams.

It is complete nonsense and indicates to me the OP doesn't really understand the field as well as they think. As someone who has gone through all the exams, the upper-level exams test aspects of the industry in incredible depth and is more about regulatory requirements than any specific model. OP may have a decent understanding of reserve models, cat models, or financial forecasting, but those are not a core part of upper level exams (although you would certainly be expected to know it).

I spent hundreds of hours memorizing flashcards on US healthcare regulations as well as some things about the Canadian healthcare system. By far the worst years of my life.

Sorry, I didn't mean to make it seem like that, you guys do go through a ton of hard work and effort to earn those credentials. It was perhaps too liberal a hyperbole..

Perhaps it depends on one's definition of "generalist".

Having the flexibility, willingness, and capacity to learn almost anything as needed (easily or with intense extra effort) is the hallmark of a generalist by my definition.

I had a different reaction. All the things mentioned in the parent comment are subfields of applied mathematics and scientific computing. Hopping between those different subfields quickly is actually more an indication of deep vertical specialization, and is nearly the opposite of being a generalist.

In software, I’d consider a generalist to be someone whose career goals are only focused on the business problems, rather than career goals focused on developing a specific skill, mastering a specific technology, or becoming known as a famous expert within a specific field or job function.

For example, I once worked with a colleague who was an amazing polyglot programmer. Our boss tasked him with fixing an asynchronous web service, including a ton of work in the frontend.

It had high business value, but my teammate wanted to focus on machine learning in order to create job experience on his resume that would let him get machine learning jobs.

While the bosses were busy fussing over essentially the desire for every engineer to be an arbitrarily fungible generalist, this guy quit and moved on to a place that would let him specialize in machine learning.

But the whole point is, this generalist thing is utter nonsense.

You're entire write-up just described the role of a generalist as described in the article. I think you're assuming a generalist means something that it doesn't because it's pretty clear in the article that the author is describing what you describe.

Why do this?

Reply on news.ycombinator.com