Hacker News Reader

Comment by _ea1k | Hacker News Reader

A lot of people will spend dozens of hours and tens of thousands of their company's money to avoid learning Java.

I'm not even sure if I'm joking. :)

This is data engineering, where people spend thousands of dollars of their company's money to avoid learning SQL. The place with no Java is across the street (old Soviet joke, originally for meat/fish stores).

Sad but true. Or they learn "something" about SQL but not about indexes, data types, joins, or even aggregate functions. I've seen some python horror shows that would select * entire tables into lists of dicts, only to do the equivalent of a where clause and a couple of sums.

Ouch. I was really hoping this wasn't so common, but I guess it is. I'm sure they heard somewhere that "joins are expensive" or something along those lines, so they solved it!

There are also other "lets pretend we have big data" patterns I've seen. Running something like AWS Athena or Hadoop over and over, for less than a couple gigs of data that would fit into memory on a laptop from 15 years ago.

Prisma built a whole ORM around that, complete with the client-side joins.

Classic, I remember seeing something like this

SELECT * FROM X, into a C# list, Filter it with LINQ, and then use C# to do calculations

"Why not EF?" "Someone told me it was slower"

I mean RisingWave, the solution mentioned in the article, is a complete startup rewriting things in Rust mostly to avoid the larger Java solutions like Flink and Spark...

Flink and Spark are painful for streaming.

They do work, but they have some sharp and rough edges.

I think 90% of the use cases for streaming data is because managers have nothing better to do than to reload their profits dashboard.

They are rarely supported by real business value.

Save the data as soon as it comes in and transform it in batch mode.

Reply on news.ycombinator.com