This textbook covers CS theory using real numbers instead of integers.
https://www.amazon.com/Complexity-Real-Computation-Lenore-Bl...
This textbook covers CS theory using real numbers instead of integers.
https://www.amazon.com/Complexity-Real-Computation-Lenore-Bl...
Anecdotal consequence: 16bits floating point numbers have so much non-linearity in their round-off error that you can use them to build neural network with no activation functions (which are traditionally needed to introduces non-linearity).
I imagine that how well this works also strongly depends on the kinds of rounding. I imagine that stochastic rounding, or the rounding used in Google's bfloat16 are different in this regard in comparison with standard IEEE floating point rounding.
I believe it is not used because it requires 16bits precision which, nowadays, you only get on GPU. People usually train on GPU but then evaluate on CPU (in production) where the discontinuity would be much smaller (as you would use 32 bits precision).
Furthermore I don't know if, in practice, that type of discontinuity trains as well as a classical activation function (the gradient propagation might be hindered by the limited precision).
AFAIK analog computers are still the standard in radars for example and it sound like neural networks would benefit from similar hardware.
I would go one step further and argue that we shouldn't teach kids discrete math first, but rather continuous math instead.
Sure, you have discrete digits and toys, but Piaget (and his student Papert) observe that kids begin pouring water between different containers in the bath before they can do integer counting and from that develop understanding that objects of different shape can have the same volume and concepts of partial filling, ratios etc.
The human scale world is continuous more than it is discrete.
(Euclidean) geometry is an interesting case. It has discrete arrangements that you can vary continuously.
Of course, there's also areas of math without anything resembling numbers or the discrete vs continuous distinction in it.
"A textbook that is widely used in computer-science departments as a substantive but light-hearted treatment of the analysis of algorithms" --Wikipedia