Only a few sites support these options. Storage Review was an early leader but hasn't much moved much, Notebookcheck is another, and of course Phoronix.
Only a few sites support these options. Storage Review was an early leader but hasn't much moved much, Notebookcheck is another, and of course Phoronix.
You don't get that excuse from AnandTech. We do our best to keep a long history of benchmark data for users to peruse: https://www.anandtech.com/bench/CPU-2019/2224
The main limiting factor on how far back our benchmark database goes is software updates. When we have to update the OS or CPU microcode for Spectre, Meltdown, etc., or update GPU drivers, that invalidates results, and re-testing a large pile of older hardware takes a long time. Historically this has mostly been a problem for GPUs since their drivers are such a moving target, but the past two years of CPU vulnerabilities have been a hassle.
Personally, all I want from Anandtech is better editing and more articles.
Aside from the quality of their content, their format is something I really like about Anandtech. Is it really stagnation when you have figured out a format which works and stick to it?
There's been a lot of debate in the enthusiasts community, but reviewers believe that user benchmarks don't have much value. There's so much variation in software, cooling, RAM speed, GPU speed, etc. Even misconfigurations like different running background apps can skew the results.
However, for a processor that's been in the market for a while, I think userbenchmarks is a good site to look at the aggregate data. Their rankings were recently updated to disfavor AMD chips, so don't take those too seriously. But for head-to-head comparisons of processors with a lot of users, you can get a good idea of how much faster a processor is.
However, I disagree that review sites should consider "user data" because 1) these are new processors and people who read these reviews are usually early adopters who want to make a buying decision and 2) the testing setup and methodology is a time consuming and scientific process and shouldn't be discounted by just asking random people to run an app.
If you think that it’s be better - why don’t you make it?
I suspect the current format is the way it is because it’s easier and “good enough”. I feel if you reach out to Phoronix you might get a response.
Because I am already quite busy on other projects.
0. CPUBoss 1. CPUBenchmark 2. UserBenchmark
And if you’re considering an upgrade, it’s almost always the case that you should be comparing latest generation offerings to arrive at a purchase decision.
Don’t call the web stagnant because they decide not to flood the page with JS libraries.
The role of Anandtech/TomsHardware and the like is not to do this. It’s to give news about new CPUs and where the fit in current offerings. They are news sites after all.
case in point: https://www.computerbase.de/2019-11/amd-ryzen-3950x-test/3/#...
you can press the button on the top right of the graph to show/hide other, less relevant entries. click an entry to lock it in for relative comparison. the top dropdown menu lets you choose between multicore, singlecore and application specific results.
There are a very large number of review sites, Anandtech being one of the first, presumably all following the same sustainability formula, but despite all that effort, consistency, thoroughness, built in tools, and building on aggregated output are the exception.
The lack of innovation on the web can be directly attributed to the lack of advertising money on the web - outside of Google and FB, of course.
One of the sad realities is that even in the enthusiast market a lot of people don’t know/don’t care to tune and test their system, and there’s a lot of traps that would make the results useless.
There’s some of it that you can check and verify, but things like watercooling or chosing the appropriate RAM configuration and timing will have a big impact.
"Objective" benchmarks were an almost tractable problem 15-20 years ago, but with the way modern OSes run background tasks, access network resources, and perform self-maintenance, it's even more difficult to bridge the synthetic-to-real world divide.
So, you select a point of reference (e.g. - new GPU), you choose the most common system components and a selection of components that tell a story (usually "if you're building a new PC now, here are the options"), you assemble the current versions of all software/drivers, and run your tests x times. The good sites will have "sub-stories," like specific workflows or new/changed features, but once you get to around 4 of these, you start hitting too many permutations to clearly communicate the significance of your choices.
Even if you crowdsourced it, it's A) extremely difficult to verify the integrity of results and prevent manipulation from marketing departments or brigading, and B) an unrewarding, tedious process where the best practice is to leave the machine untouched for hours. Most of the crowdsourced benchmarking sites are set up to be little competitions and sanity checks when overclocking. For a pure review of shipping hardware/software, you run the benchmark and you're done. It isn't particularly fun, requires invasive cataloguing of system specs, and isn't very new-user/first time builder friendly.
Dynamic graphs would be nice (I love them), but I suspect many review sites have run the numbers and found they reduce overall web metrics. Many (I would argue most) people don't notice interactive page components, aren't interested enough to turn their quick article skim into a deep dive, or are reading in less optimal settings (e.g. - on phone on the toilet). Instead of getting 7 page views for 1 minute each with separate graphs, you're getting 1 page view for either 1 minute (probably 70%) or 10 minutes (probably 30%) with a dynamic graph.
I'm a total data junkie and I WISH there was a good solution to these problems, but there isn't a practical implementation that I've seen or dreamt up that doesn't have enough variance to render the point of having such fine-grained data moot.