Bridging Economics and Data Science

I'd encourage any economist that has a knack for programming to really put in some time towards the area. I've owned econpy.org for about 3 years now (although I haven't updated it in a long time). I also own economics.io and run econpy.blogspot.com.

It's much easier (relatively speaking) for an economist to pick up some programming than it is for a programmer to pick up some economics. Economists are already familiar with the types of questions that are important to economists, and more importantly, how to frame them. The trouble with economics is that you can't just pick it up overnight as it is a way of thinking more than it is a tool set. Programming on the other hand is something that you can "get working" overnight (economic programmers don't need to be algorithmic theorists -- they just need to be really good at getting/scraping data and organizing it so they can run analyses on it).

Over a year ago, I dropped out of my PhD program in economics because I was not at a school that was going to allow me to do the econ/cs type work I was working on. Leaving my PhD program was one the best things I ever did because it has allowed me to pursue whatever I want to do with the skills I've acquired.

The problem with academic economics is that the data most economists use is so bad and outdated -- such as data from FRED, BLS, and other publicly available sources where everyone and their uncle can download the same CSV dataset that was aggregated by some government employee. The race then is to see who can put together the most elegant econometric model to handle all the issues with the data. The rules of the game change when you create your own dataset and thus have control over while variables to include, the aggregation, the frequency, etc.

Long story short, if you are an economist wanting to do programming, learn to adapt those skills in academia (best way is to find a great advisor -- if there isn't one in your economics department, check the business school as bschool professors are often much more open to highly empirical analyses and care [marginally] less about getting the theory perfect). Or, if you want an easier lifestyle that is much more rewarding, ditch academia for the private sector. You'll find the economists in the private sector to be much more knowledgeable about cutting edge technologies and willing to listen and learn from what you have to say.

I have an econ/finance degree and also worked for several years in finance. Then I quit my job and learned how to code. Now I have a web app with a paying and growing customer base. Knowing how to program will give you special powers when combined with your econ background. Congrats!

He nails the point, and then misses it himself:

the key to success is in cleverly selecting, finding, or creating a data source that answers a particular question

It's about asking the right question and then finding or generating the right data to answer that question. That's what makes it science.

When you need to study 30 plus income statements from 30 plus entities around the globe, you wonder less about the tools and more about getting it done. Then you become a "spreadsheet monkey", but that is only because nobody has been able to create a better tool since 1995.

I understand the pain of seeing people using old tech and taking hours to do tasks that should take seconds, but do not underestimate the importance of specific field knowledge and the fact that many people do not have time to learn coding. If you think about it, it is an opportunity for you to build a bridge between those 2 worlds. For example, nobody has produced yet a decent tool to consolidate financials...

This is the same in the social sciences. I'm currently a PhD student in one of the top Communication departments in the US, and it's painful to see how far behind in technical skills and tools the curricula are (eg., Excel and SPSS). I've been self-teaching Python, R, and SQL, and extending my knowledge base from simple regression-based stats to data mining and machine learning, to make up for it. Not only does that allow me to work on massive datasets (and push the field forward across methodologies), but it allows me to improve more 'traditional' approaches by sharing data and models (eg., with .R scripts).

A financial economist professor once mused that "current phd students are limited in the problems they can solve because they can't program." Of course the programming language he preferred was Fortran. :-)

Excel is the best tool for 80% of what bankers and consultants do. It can middle through the next 10%. The problem is it has just no way to do the last 10%. Either it's too slow or just can't handle the size or computations required.

Like the author, I have an Economics background but have gotten into programming as I graduated from working on Excel. Economics has suffered because of a lack of good data - this is why so many explanations by economists begin with assumptions. I'm hopeful that the data sets now available will improve economic models and that people working in the public sector will put them to good use.

Analysts in finance/consulting certainly haven't had to upgrade their skills, but analysts in media/marketing definitely have had to.

Certainly there are people in this space who can't do much beyond spreadsheets, but there are many analyst now who use python/pandas or R to do work.

The biggest issue I see from the ivory tower I'm surrounded by is that economists typically doubt the results of data mining. Neural networks, machine learning, etc. are all well known toolkits in computational economics (one of my specialties) but the results from their application are rarely believed.

Ditto. I am following a similar path. I am curious if there are any groups dedicated to people like us (econ/consulting/finance people learning data science) to help facilitate the learning process?