Hacker News Reader

Comment by elteto | Hacker News Reader

You could probably keep 15 million rows in a csv file and still get great performance. Without even mentioning sqlite. Any reason to use such an overkill solution for that problem? Honest question, not passing judgement.

View on HN

I inherited it from the prior CTO and team.

This was one of the better solutions they had. They also had Talend for ETL and Snowflake for “analysis”. You don’t want to know what Snowflake cost.

For the same roughly 15 millions rows…

This is just so incredibly incompetent. What gives? I would expect this in government but in a company incentives are meant to be aligned.

The prior CTO came from much larger, highly regulated startups. His only experience was very large scale systems with complex requirements. He took the approach that worked there to this much smaller, mostly unregulated startup in a very different industry.

It happened because he was the only semi technical person who was an FTE. Everyone else was from a consulting firm that the private equity owners “suggested” they use.

That explains a lot. It’s still a case of incompetence but this gives it a clearer explanation.

Government is not only compatible with competence; it is a profound source of competence. When we recognize our place in an immensity of people and in the passage of ages, when we grasp the intricacy, beauty, and subtlety of public life, then that soaring feeling, that sense of elation and humility combined, is surely civic. The notion that government and competence are somehow mutually exclusive does a disservice to both.

Reply on news.ycombinator.com