Worst case I've found was off by 15 orders of magnitude.
15,198 karma · joined June 8, 2017
Worst case I've found was off by 15 orders of magnitude.
I don't know if they're actually building AI DCs out there, but it doesn't seem completely unreasonable like you're suggesting.
Rather than hamstring all software by requiring DOT testing before firmware updates are published...
I don't know how the rules work in the UK or Ireland where the author is, but the US has no such mandatory testing. Also, all manufacturers have telemetry these days and the ones I'm familiar with all do gradual rollouts (to varying levels of competence). You basically can't do immediate rollout given the scales involved.Please don't take this as suggesting any of them are good at software, mind you.
I think it's pretty extreme to imagine their nth descendants caring about the goals of the parent civilization.
It's an example to demonstrate the concept in familiar terms, not a psychohistorical prediction.The results speak for themselves. Those early printed works are beautiful to a degree few other books have managed since.
The equivalence with Einstein summation was noted in this paper [0]. Sandia labs wrote an SQL database based around graphBLAS called TenSQL a few years back. And something similar to your idea of relational algebra as the basis for AI had a paper published earlier this year [1], naming it Tensor Logic.
Are all of these common operations secretly relational, just with the wrong data model?
Sadly not, but you can get a long way before you find the limits. Modern databases are very well optimized for their use cases, but there's many other possible points in the design space to explore. If you want to really make efficient use of hardware, there are fundamental scaling problems with natural joins because of the combinatorial explosion that gives us WCO though. If you accept an inability to express triangular queries, you can get something that's embarrassingly parallel and scales really flipping well.If you want to stick to the full relational model, you might be interested in Differential Dataflow [2].
[0] https://arxiv.org/html/2510.12269v3
In the US market, I think Hyundai (and to a much lesser extent Kia) is probably the most interesting OEM. EV sales that aren't declining, the highest American-made percentage vehicles (relevant given recent policy changes), a manufacturing partnership with Waymo, widespread consumer awareness, and relatively lower prices than their competitors. It'd be nice if they provided a better repair situation than Tesla, but that's the current industry I guess.
If you stick to the safe bits I've been discussing elsewhere in this thread and your platforms implement IEEE floats, float math will also be commutative + associative and you won't have to deal with precision loss. That means your usable range will be narrower than the same size fixed type (because it's limited by the mantissa) but it's large enough to still be useful.
You're right about that particular constraint, though I'd question why the achievable 2-4mm precision at 32 or 64km are meaningfully different. Covering up unstable collision code and adaptive methods would be a rewrite?
- Transcendental functions (exp, sin, etc.) are really hard (multiple literal PhDs) to implement according to the rules I’ve just described (“correct rounding”), so you’ve only been able to get such implementations in the last few years, and I believe no stock libm has completely switched so bring your own if you need them.
This is right, but it's useful to point out how important the "correct rounding" qualification is to the difficulty of the problem. Writing a "good enough" function with floats is easy even for non-experts. exp can be efficiently implemented in hardware with a 4 element lookup table and polynomial interpolation. Sin/Cos are range reduction + a minimax polynomial from sollya. But the standard [rightly] doesn't prescribe specific implementations, so you need fully correct rounding to have cross-platform determinism.Correct transcendentals are also difficult in fixed point. When pigweed was implementing them for their fixed impl, they got help from from an ex. Mathworks floating point expert on the LLVM team with a relevant PhD to do it correctly [0].
First, a point I didn't make, is that if you have 32 bits of fixed, you get way more precision than with a 32 bit float.
That's true, but I already responded to it. If you step up to the next size of float (e.g. f64), you have more precision than the fixed32. You can do exactly the same computation in f64 with equivalent inputs, and you'll get better precision than doing it in fixed32. Or you can round at every step like fixed does and get a bit-equivalent value if you don't want the precision. It's less memory efficient, but my point is that the remaining use cases for fixed point are exceptional/situational and getting increasingly niche.Maybe using a bigger float type is cheating, but it's basically free because of the ubiquitous support for floats.
In contrast, with floats, you can lose precision.
This only happens with values outside the range of your fixed point type if you use larger floats as mentioned above. I consider that a different argument. You can alternatively view this as the float handling a situation more gracefully than fixed point would have. Afaik finance people don't even use floats for things like account balances, because you can't represent something like 0.1$ exactly.
Finance types typically use decimal types from what I understand. This is really just the result of using a decimal syntax to initialize/output a binary representation. Fixed point has exactly the same problem. Decimals have an analogous issue with the value 1/3. It has other advantages, like your calculation running the exact same regardless of CPU and/or compiler, which I'm sure a lot of analysts care about.
I wrote a library that makes floats more practically deterministic across platforms for very little cost (linked at [0] so you can see the limitations and numbers), and the underlying problem is [maybe] getting a standardized solution in C++29. You can get the same thing today just by changing compiler flags. If you need the special, non-reproducible float functions, your options are mainly to import a library or implement it yourself, same as fixed point. Not trying to be antagonistic, just figure out where you're coming form.
I work in safety-critical automotive/robotics, used to do audio DSP, contributed a bit to the aforementioned standardization, etc. I also have a talk on this topic I've been working on for the last few weeks. It's a bit of a pet subject. Fixed point has basically no language support, and is very hard to get right, but sometimes you need to do that.
There are absolutely situations for it, but that's exactly it: it's situational. And those situations are increasingly uncommon these days, now that hardware with good IEEE support is essentially ubiquitous and compilers/standard libraries are improving their implementations.Floating point is generally deterministic in practice with a fairly minor amount of effort, the major remaining issue being library rounding. I actually wrote a library that guarantees this for arbitrary code, with some small, obvious caveats like standard library precision. And the conference talks linked above note, the standard library issues are an increasingly solved problem for modern toolchains. The remaining cases are mostly things you won't do in fixed point. Let me know if you're aware of anyone computing erfc in fixed point for determinism though.
I'm not saying there aren't any situations where other systems are justified, but you probably won't know if you fall into any of them without the kind of numerical analysis that most codebases will never receive.
Yes, fixed point can use simpler hardware. That's also a completely irrelevant consideration for software. The vast majority of processors are optimized for floats now and some operations (e.g. division) are actually faster.
The precision argument also falls apart. Any float with mantissa >= X+Y can get exactly the same results as a QX.Y fixed point. The float will actually perform better across the same range because you have to round it to perform like the fixed point. That means more precision, lower error, automatic normalization, better overflow behavior, a larger working range, etc. And it'll probably be just as fast, unless you're bottlenecked on memory bandwidth of inputs (unlikely). When you inevitably want an exp() or another special function, it's a heck of a lot easier to call libm than implement your own and it will perform better.
Floats are also much easier to get right for your coworkers that aren't numerical analysts.
"What’s up Outragers, just dialing in from my NBA team’s headquarters to yell at the people whose careers I’ve just pummeled"
In a water cooler channel, while the CEO was calling in from his NBA team headquarters to announce layoffs...It's mildly dramatized, bordering on milquetoast satire of exactly what he was doing.
Don't bet the company on no one ever making a mistake. Set up a system where the unintentional risks are minimized and the consequences of inevitable failures are mitigated by seatbelts and insurance policies.