Why MS Excel Is a Poor Choice for Data Projects
promptcloud.com
promptcloud.com
If you want to demonstrate how Excel is a poor choice, show how someone would do data analysis using excel, then show how your alternative system, whatever it is, would do it "better", for some definition of better you want to use.
Just stopped there : + not working on linux, paying on MacOS and Windows + proprietary format: hard to get data in / out from other tools (for instance, but not limited to, python lib, R lib, matlab (if some of us still want to use matlab) and all tools built upon those libs) + sucks for any real-sized dataset (try to handle 10k+ rows, not even thinking about 1M+ rows)
Those 3 reasons alone mean that you need to pay for a tool which would hardly do the job a team ask for, if only it works on their OSs, and won't be able to share it with other teams.
Excel's format is publicly available (if a little horrible to read XML), and it imports/exports CSV fine.
Yes you do have to pay for it, but I find lots of R extensions don't work well on Windows, so OS-lock isn't a solved problem. You can use Office 365 in the web browser if you want to access from linux.
The biggest issue is, as you say, don't try to put 1M+ rows into Excel. I agree. But I've worked with lots of people who have 20-1000 data points, and there Excel is great. I'd love to have an alternative to suggest to people, that scales well with them, but we aren't there yet, and Excel still fills an important niche. Assuming people only use it because they are idiots (not saying you said that, but many do) and not looking at why they use it is what is stopping other tools replacing it.
I was part of a startup that was essentially trying to replace excel for a specific market and we made a very similar list (with sources). Sadly I don't have access to it anymore...
Anecdotally, from what I hear from friends who work in the industry, many banks have a rather error-prone practice of emailing excel sheets to each other resulting in version mismatches, versions getting lost, latency etc.
Here are a few reports for you to look at:
http://www.businessinsider.com/excel-partly-to-blame-for-tra...
http://www.cio.com/article/2438188/enterprise-software/eight...
http://blogs.wsj.com/moneybeat/2014/10/16/spreadsheet-mistak...
But I guess the question you are asking isn't "is excel error prone", but rather, "are the alternatives less error prone", and while I don't have any supporting evidence, I would think that when using a "real" programming language, you would have version control, unit tests and so on.
You would if and only if you employed professional programmers at those tasks, which with Excel (apparently) could be done with less/differently trained people. Version control and unit tests are not features of environment, but of professional training and discipline.
But you're right, without professional training and discipline even if the tools exist, they're not used...
I will say the one downside to this practice is that I get stuck editing a bunch of connector code because no one can send the same information in the same format twice.
I think the snowplow analytics guys (again zero association) have better and arguably more transparent/honest documentation. Particularly the whole end to end complete process.
One of the reasons people like Excel (besides the obvious ubiquity of that software) is that they don't have to put their data on some other service. Some even feel it is more secure (this is perhaps somewhat false). That beings said I don't know if I would ever trust a company hosting all my data collection/warehouse needs like the authors of this blog. They might not do anything bad with the data but they sure do have a lot of leverage on you once you completely rely on them.
The other thing is the proprietary data visualization/calculation companies will come and go. I bet Excel will still be here 20 years from now. That is why R and SQL are also good things to learn as well.
> A business organization or enterprise always needs to adopt multi-directional approaches.
Reads like a marketing class assignment from junior high.
Who writes like this in 2017?
We're getting closer than ever to the world's first Turing-complete thinkpiece bot.
A fair limit is 1,000 rows in a sheet, which is enough for most utilitarian use cases (e.g. daily-aggregated data for a year), and certainly enough for bespoke models. I would not recommend using spreadsheets for Kaggle competitions, though.
https://catalog.data.gov/dataset/crimes-2001-to-present-398a...
> The dataset contains more than 65,000 records/rows of data and cannot be viewed in full in Microsoft Excel.