948 karma · joined August 23, 2010
I think you might be two orders of magnitude off here. A class leading general purpose compute chip on a cutting edge process? I don’t think $5 billion to produce working chips is unreasonable. If it were only $50M to produce A14/M1 level chips, Qualcomm would already have done so.
Cerebras is cool, but they are not on TSMC 5nm and don’t have to design general purpose silicon, where you have pesky things like huge reorder buffers or 8-wide decode blocks, which are not seen in any other production CPU right now.
Will they? That didn’t happen with the UAC dialogs in Windows.
> If the workers who were recorded as employed but absent from work due to “other reasons” (over and above the number absent for other reasons in a typical May) had been classified as unemployed on temporary layoff, the overall unemployment rate would have been about 3 percentage points higher than reported (on a not seasonally adjusted basis). However, according to usual practice, the data from the household survey are accepted as recorded. To maintain data integrity, no ad hoc actions are taken to reclassify survey responses.
If you apply the correction, the rate would be 16.3%.
> If the workers who were recorded as employed but absent from work due to “other reasons” (over and above the number absent for other reasons in a typical May) had been classified as unemployed on temporary layoff, the overall unemployment rate would have been about 3 percentage points higher than reported (on a not seasonally adjusted basis). However, according to usual practice, the data from the household survey are accepted as recorded. To maintain data integrity, no ad hoc actions are taken to reclassify survey responses.
If you apply the correction, the rate would be 16.3%.
Considering how big the die is (826mm^2 @ TSMC 7nm) and how many transistors there are, they really must have beefed up the TensorCores much more than the general compute units.
The 20x 32-bit floating point improvement is probably achieved by comparing doing full FP32 calculations on the previous generation vs doing TF32 calculations on Ampere. This would not be an apple-to-apple comparison as the TF32 result is less precise.
That said, it is probably not terribly important for deep learning at least, given the success of BF16.
Full adders in FP units scale with the square of the mantissa bits, so if the number of mantissa bits stay the same, they can reuse the existing units.
Since Nvidia already has existing FP16 units on die, using those units for TF32 calculations probably doesn't cost too much additional die area.
Thus, traders who still have contracts open but cannot take physical delivery will have to close their positions out today or tomorrow, thus causing the dip we are seeing.
The June contract, where most of the trading is happening, is at around $22, and the July contract is $28.
Maybe something like Intel's EMIB technology where they have small interposers along edges of chips rather than having a giant interposer might help here.
Yields are probably fairly good if they design for manufacturing by placing extra cores / wires to route around failures as I am sure they are.
For instance, the DGX-2 has an MSRP of $399,000 and consumes 10 kW of power [1]. The average commercial electricity cost across the US is about $0.11/kWh [2], so a DGX-2 running at full tilt costs $1.10 an hour in electricity. Thus, a DGX-2 running at 100% utilization for 3 years costs $9,636 in electricity, which is ~2.5% of the cost of the box itself.
Of course, you probably could get DGX-2s for a lot cheaper if you are buying 100 of them, but the acquisition costs are still going to be significant vis-a-vis power costs.
[1]: https://www.anandtech.com/show/12587/nvidias-dgx2-sixteen-v1...
I wonder if Tesla should not have entered the mass market with the 3 and stuck with the S and X. Higher margins and lower CapEx there.