Hacker News Reader

Comment by haskal | Hacker News Reader

Start a consultancy, name it (or if you have one, rename it to) Cassandra[0].

[0]: https://en.wikipedia.org/wiki/Cassandra "Cassandra was cursed to utter prophecies that were true but that no one believed."

View on HN

And in keeping with the spirit of the article charge murderous fees to help people with their "Big Data" Cassandra installations, only to find out after weeks of wading through a management quagmire that the data in question can be stored in a few KB text file.

Incidentally, Cassandra is called like that because she kills the /Oracle/...

Selenium being the cure for Mercury poisoning, but I guess that's neither here nor there. :-)

Devious! I love it.

"Big Data" doesn't mean "makes Excel slow down" or "needs to scroll in Excel" :) If you can fit it on a thumbdrive you can buy from a Walmart, it's probably not 'big data'.

This goes for so, so many things I've seen where it's not even big data, but crap like some small part of their total dataset being a graph (unlikely to ever go beyond a thousandish nodes for any given connected set, and even that'd be unusually high, and certain to be sparsely connected to boot) so of course we have to use a hipster-ass graph db adding thousands in development costs and making the whole thing harder to work with and (demonstrably—this wasn't the first project they'd made this mistake on) less stable eyeroll.

Starting a new project? You almost certainly don't need something other than 1) files (yes, seriously), or 2) SQLite (yes, seriously), or 3) Postgresql or some other multi-paradigm, capable SQL DB. If the former two, please also consider whether you even need a f*cking server or are actually writing something that ought to be desktop/mobile software. That's another expensive, feature-delaying, and UX-harming mistake I've seen.

I started a new job where I was not officially a programmer and my machine was locked down so that I could use basically nothing except Microsoft Office, and after a couple of months, I have taught myself to do nearly any sort of data processing I previously did with Linux, Oracle, Perl, Selenium, etc. using Excel, Access, and VBA. Obviously, this doesn't involve that much data, but my previous job was really not "big data" either and they paid for all sorts of expensive licenses.

I'm enjoying myself much more than I used to, because I've escaped the red tape of development and get to solve problems that matter to nontechnical people without intermediaries predigesting the requirements.

If I was starting a business, I would definitely always try to do almost anything with Excel and Access first before deciding to invest in an industrial strength solution whether Postgres, Oracle or "big data".

This really is true, I worked on a project where we were meant to be getting hourly files into a data lake, the files so small we couldn't reach the recommended size of 256mb per file (compressed parquet in azure adls) - the files were like 1 mb each - a years worth of data was tiny and the processing overhead ridiculous

If your workflows don't bog down your servers, add big data technologies until they do!

I think if you cant fit a single data set on the biggest commonly used hard drive is another good metric.

Reply on news.ycombinator.com