Hacker News Reader

Comment by threeseed | Hacker News Reader

a) It's been widely acknowledged that we are approaching a limit on useful datasets.

b) Synthetic data sets have been shown to not be a substitute.

c) I have no idea why you are linking Moore's Law with AI. Especially when it has never applied to GPUs and we are in a situation where we have a single vendor not subject to normal competition.

View on HN

Synthetic data absolutely does work well for code.

While Moore's Law probably doesn't strictly apply to GPUs, it's not far off. See [1] where they find "We find that FLOP/s per dollar for ML GPUs double every 2.07 years (95% CI: 1.54 to 3.13 years) compared to 2.46 years for all GPUs." (Moore's law predicts doubling every 2 years)

https://epochai.org/blog/trends-in-gpu-price-performance#tre...

It’d be really nice to see research in this area from somewhere without a financial interest in hyping AI.

That incentive doesn’t invalidate research, but AI results are so easy to nudge in any direction that it’s hard to ignore.

I wonder when people mention Moores law do they use that vernacular literally or figuratively. IE literal as having to do with shrinking of the transistors, figuratively with any and all efforts to increase overall computational speed up.

In this context it’s the latter, but practically speaking they’re the same thing.

b is made up. They have absolutely not been shown to not be a substitute. It's just a big flood of bad research which people treat as summing up to a good argument.

Reply on news.ycombinator.com