Hacker News Reader

Comment by the-alchemist | Hacker News Reader

I don't quite understand the YJIT excitement. A few benchmarks show that TruffleRuby generally beats YJIT, sometimes by a 10x margin.

[0]: https://eregon.me/blog/2022/01/06/benchmarking-cruby-mjit-yj... says

View on HN

And by many accounts Oracle makes the fastest database yet everyone still uses Postgres.

Also our experience with truffleruby is death by a thousand subtle differences. It might be 97% compatible but chasing down that 3% undocumented behavior difference on every minor version update for every gem got exhausting fast. Everyone uses and tests against cruby.

I have only seen one set of benchmarks, apparently posted by someone willing to tempt fate by violating the benchmark ban of the license, which compared them and in those PostgreSQL crushed Oracle. Of course those could have been misleading or straight up lies but it is not at all a known fact that Oracle has better performance than PostgreSQL. And the database consultants I know who have worked on both seem to think performance is roughly equal.

Yea, I’ve always seen Oracle’s big selling point is the admin tooling that they provide. At least that’s what I’ve been told.

I've always seen their big selling point as it being required by other giant software packages. At least that's the way it threatened to enter burrow into various places I've worked.

> by many accounts Oracle makes the fastest database

I've never seen that credibly claimed. Why would their license ban benchmarking if their performance was any good?

Their licensing does not ban benchmarking; it only bans publishing the results. A subtle but important point.

By all means, run benchmarks on your workload profile and choose the technology that works best for you - that is perfectly OK. But not make claim that xxx beats oracle in yyy.

Oracle goal here is to make a good enterprise ready DB solution; not to fight the open-source community on hundreds of different performance claims, half of which have been specifically engineered to put the competing, leading product in a bad light.

You might not like what oracle does or how they do it, but it’s clear that they have a target on their back and sometimes the best way to play the game is not to play at all.

Just look at the « independent and fair review » that Top Gear made about Tesla.

> Oracle goal here is to make a good enterprise ready DB solution

Maybe. Or maybe it's to soak naive clients for all they're worth. The remedy for bad benchmarks is good benchmarks, not no benchmarks.

> Just look at the « independent and fair review » that Top Gear made about Tesla.

What about it? It was accurate and informative (unless you think Tesla lied about their range stats). The good points about electric cars are real but they don't mean we should ignore the downsides.

It was later shown that the whole part were the Tesla goes out of power was staged.

The car did inform them it would run out of battery on the way. They made a segment about it and failed to inform the viewer that they had ignored several warnings given by the car before reaching the « oh no we are out of battery in the middle of nowhere » point.

This is not what I would qualify as a « fair review ». There are no « false claims » (they did run out of battery, after all !) but the information are given and presented in a way that paints the reviewed product in a specific light.

The way I remember it it was the other way around. They had the voiceover saying "and after x miles..." with a video of their crew pushing the Tesla into their pit lane. Tesla's argument in the lawsuit was that based on their telemetry they hadn't actually had zero battery when filming this, which seems beside the point when they quoted the mileage number from Tesla's own materials.

> The remedy for bad benchmarks is good benchmarks, not no benchmarks.

The comment you replied to just said Oracle allows running benchmarks internally, though.

Sure. How likely is it that someone running bechmarks internally - who is unlikely to be a benchmarking expert - is going to do a better job than a dedicated benchmarking project like TechEmpower that has a whole lot of people looking and contributing?

TechEmpower accepts a wide variety of contributions without much vetting. It allows quite a lot of benchmark gaming. The mantra of every good engineer about benchmarks is, 'Don't trust others' benchmarks, do your own with realistic workloads'.

Not everyone, other than for a specific customer project, I have mostly worked in SQL Server or Oracle, and slowly approaching 50 now.

There are still quite a few enterprise workloads that only the likes of stuff like DB2 are a match for them.

What? When was oracle free or opensource?

and when was oracle the fastest database? You are not allowed to even publish that metric....

I do have some sympathy for companies who don’t want to have to deal with a constant barrage of benchmarks that are egregiously wrong.

HN has been host to quite a few of these spats over the years and I wouldn’t want to deal with it, there are pretty much only downsides to letting people poorly benchmark your product and have every incentive to get it wrong.

well to be fair, most benchmarks are of course not comparable with the real world. People should always use what works best for them.

TruffleRuby can’t run production Rails traffic today, and YJIT can.

The README states "TruffleRuby runs Rails". Anything specific?

It passes most (maybe all?) of the Rails test suite. But it can't run a randomly-selected production Rails application.

This was based on my conversations with Chris Seaton before his passing, about 6 months ago.

That doesn’t sound like a very good test suite then.

i'm pretty sure eregon is biased...

Reply on news.ycombinator.com