Providers of RISC-V HW or Designs, like Alibaba, SiFive or Western Digital, do lots of testing too.
It would be like saying that Linux does not provides the quality of commercial closed OS (kernel), this is just wrong in multiple ways:
1. Linux is also sold as base of some commercial OS (e.g., RHEL, SUSE)
2. Lots of commercial companies revenue depends directly or indirectly on it, so there's a massive testing effort. In it's sum it so massive that I cannot believe that it's matched by any closed commercial kernel (no actual numbers here, sorry, this is really just personal ball parking).
The same can happen to RISC-V, the combined testing effort can easily outnumber the one of ARM in the future.
So while yes, testing and verification is not free, it is just not an argument at all against any open CPU design, be it openPOWER, RISC-V, open MIPS, ... Rather, it speaks again for openness as combined test effort will be hard to be matched by a single closed player.
Open always wins....given enough time. No single company can compete with a collective open ecosystem at scale.
Open source CPU design is fundamentally different than open source software design. In the latter costs are extremely low - just the cost of a computer per developer. That developer's computer need not be replaced for years. There is no significant incremental cost for a software bug - just recompile and in a few minutes you're off to the races with a new executable which can be distributed over the internet for next to nothing. Contrast that with CPU design - every time a hardware bug is found you'd have to fix the design, verify it in software simulations, fabricate a new wafer, package it, install it in test hardware, and then perform hardware verification. This is 5 to 6 orders of magnitude slower and more expensive than software. Sure, corporations can perform this open CPU design, verification and manufacturing function. But in the end for a CPU to have a certain level of speed and reliability, you'd have to spend at least the same amount of money as the commercial CPU makers. Companies that produce an open source CPU chip are incurring huge monetary risks - and would have to be compensated for this risk if their chips have bugs and cannot be sold.
The only way for an open source chip design to be remotely competitive would be if they were to embrace FPGA technology. But FPGAs run 4 times slower than purpose built ASICs and are at least 10 times more expensive per unit in volume.
This also means you'll see more and more vendors of RISC-V cores and RISC-V verification suits. Chip designers are just as willing to pay a license and support fee to these as to ARM, as long as the cost is lower. So I'm absolutely certain that you'll eventually see RISC-V cores verified to the same level as ARM.
Also remember that open source RISC-V cores is more likely to get free verification efforts from universities, students and hobbyists.
> The only way for an open source chip design to be remotely competitive would be if they were to embrace FPGA technology. But FPGAs run 4 times slower than purpose built ASICs and are at least 10 times more expensive per unit in volume.
Ah, yes, but you can verify on FPGA and ship on ASIC. That's why even hobbyists can now work on developing and verifying RISC-V cores.
These days it's really not that hard to verify a microcontroller-level CPU core. It might take a long while until you get a state-of-the-art superscalar multicore RISC-V CPU for servers and desktops, but I think it'll eventually happen for RISC-V the same way it did for ARM. Hell, the open-source designs are already there, and they're pretty well verified, which is way further than ARM was at the same stage (they had zero incentives to do anything that wasn't commercially viable after all, unlike the research communities developing RISC-V cores), you just need further optimization and verification.