Hacker News Reader

Comment by phillipcarter | Hacker News Reader

In my experience there's a lot more need for people who understand the data and data systems of a company and how to analyze that data than someone who can write some code. This is also technical work in my mind. Data analysis done correctly is very hard, because it's rarely the case that a company's data systems are well-formed, gather the right information without any gaps, and organize it effectively. Knowing how to wrangle that can make you incredibly valuable to everyone, and there's no need to know how to write product code to do it.

View on HN

> there's a lot more need for people who understand the data and data systems of a company and how to analyze that data than someone who can write some code

(discloser: I run a company that builds software for data professionals, so I have a bias)

This is a very interesting topic I spend a lot of time thinking about.

I don't think, in the aggregate, there will be more positions available for folks to "analyze data" (i.e. data professionals) than positions for those that "write code" (i.e. software engineers).

Software has been infused into virtually every industry - that is to say most companies are "technology companies" - the global labor market for software engineers reflects that.

BUT I do think we are in the nascency of data and data analysis (ironic since we closing in on a century of being in the "information age") and the number of folk s doing this professionally is about to grow astronomically.

If you spend enough time, like I do, talking to companies and seeing how they handle data, it's almost staggering how few companies get this right, and that the bigger the company, the worse the effort. Sometimes I cannot even comprehend how big of a wave is coming towards the shoreline.

Honestly I’ve seen the opposite. Business leaders usually ignore data insights, and the pay that goes to the average data scientist or business intelligence engineer reflects that. Building an actual product, aka coding, is where the money is.

I think both your and the parent's comments can both be accurate at the same time.

Yes-data granularity, structure, architecture, and systems. Having worked with many customers in the difficult-to-define "magical" space (as a commenter put it earlier) of using Excel and it's successors (Smartsheet, etc.) to solve problems without realizing it's a kind of code, understanding how the data flows and forms can be leveraged.

Both Smartsheet and Excel are offering a kind of "insight" or ideas feature which attempts to provide a foothold on this data wrangling aspect of an organization's data flow.

To another poster's point, yes, sometimes the insights gleaned are ignored or misapplied for various reasons.

To me, this is the challenge of the digital revolution: getting enough humans to properly interpret and use the data that is staring them in the face, day after day. Coding may not strictly be necessary. Communication skills surely are.

This article is about reaching just a level of proficiency in the technical aspects of your company and product that helps you excel in your non-technical role. The author specifically advises against learning to code because of the time and financial barriers to entry. Is there a way to gain an employable data science skillset that doesn't necessitate those same barriers?

Eh, maybe, but usually you'll have to write software to wrangle/manage the data. And even if you can go without software it would be a lot slower (like a few orders of magnitude slower with a human in the loop). But your main point about the cost of data wrangling vs the cost of "actual data science" is correct.

Reply on news.ycombinator.com