Hacker News Reader

Comment by godmodus | Hacker News Reader

I love this answer.

For some reason, I've been dreaming about negative numbers lately. I think they deserve their own number set notation.

the standard IEEE floating point uses a 'sign and magnitude' system. Negative numbers are basically made by flipping a single bit. I've been playing around with a number system that expresses real numbers as two's complement (like how integers work) - xor all your bits and then add one. In this system, 0 is -0 and the number that is weird for signed integers: 0b100000...0000 is infinity.

Now, there's an exponent and a fraction, too. I've been playing around with how to do these numbers with logic gates (and verilog!) and you can either two's complement the whole thing and work with absolute values, or you can keep the fraction part as a two's complement...

So I just redid multiplication using two's complemented fractions! And addition/subtraction too, which for floating points in general is significantly harder than multiplication. The nice thing about two's complemented floating points is you don't need separate algorithms for addition and subtraction; you can just do everything with one algorithm.

Have you seen "Stanford Seminar: Beyond Floating Point: Next Generation Computer Arithmetic"?

https://www.youtube.com/watch?v=aP0Y1uAA-2Y

Well, I'm the guy in the black shirt who did the demo.

If you liked that lecture, I'm already starting on some verilog implementations.

This is an example multiplication 8 bit * 8 bit -> 16 bit unpacked (20 bits). It differs from standard floating point in that the fractions are stored as two's complement. It takes a little bit of wrapping your head around, but the hidden bit for negative numbers is actually -2 ! Moment of zen.

https://github.com/interplanetary-robot/mullinengine/blob/ma...

Way cool -- I realized that when I checked out your profile just after posting the comment -- what timing -- I discovered the video a few days ago when looking for precise/compact interval representations. Interesting work indeed.

Has there been much traction for getting major chip manufacturers to implement this? I know they're all looking for the next big thing and Intel is working on specialized neuromorphic chips. A general "drop-in" replacement for floating point seems like an opportunity for a general-purpose win from low-hanging fruit:

Intel Gets Serious About Neuromorphic, Cognitive Computing Future https://news.ycombinator.com/item?id=13623846

well seeing as how John invented these numbers literally two months ago, I haven't seen any traction yet! But I am persuing fundraising opportunities. In the demo I showed how you can effectively reduce the bitwidth to 8 bits and still train in a very trivial machine learning exercise. I'm currently enrolled in the udacity machine learning class and implementing everything in parallel in julia so that I can try more complicated architectures using posits.

I do have a hardware architecture in mind for how to very effectively and efficiently execute machine learning calculations using posits.

I’m interested in representing angles / points on the circle; 3d unit vectors / points on the sphere; unit quaternions / points on the 3-sphere using 1, 2, or 3 relatively low-resolution posits, under stereographic projection.

How efficient do you think regular C or GPU code (on existing hardware) can be made for compressing a 32-bit float to e.g. a 16-bit posit, and for expanding the posit back into a 32-bit float?

I don't think it can be made that efficient in software. Is there a particular reason why you need 16 bits? A 32-bit float is going to be better than a 16-bit posit almost always (posits are better their equivalently sized float, but they're not that good) and once it's in posit representation do you have a way of doing mathematical operations on them?

This is just for data compression, not for computation directly. 32 bits is often overkill for transmission/storage of rotations, unit vectors, geographical locations, unit quaternions, and the like. Depending on the use case 8 bits might be enough, or 12, or 16.

To actually do computation I would convert the posits back into 32-bit floats (or e.g. in the Javascript case, 64-bit floats), and then take the inverse stereographic projection.

[Stereographic projection is extremely cheap; for each data point only requires one division and some additions and multiplications.]

I’ll do some experimenting at some point.

Posits look too good to be true! Is there a reason why regime bits do not include the sign bit? Then both "0 0001" and "1 1110" could be interpreted as 4 regime bits. Even better we could include the last flipped bit as well and we would have 5 bits.

Edit: Well. I see it would result in losing the values 0 and 1. Another question: Since it is fixed length of 4 bits (for N=32) why don't we just extract the 4 bit value, then we could represent 2^4 regimes this time without losing 0 and 1.

I wouldn't screw around too much with the sign bit. The way it's laid out is really kind of cool... Negation is simple two's complement.

In my software posit library (which is intentionally strictly binary and not backended by IEEE floats), (https://github.com/interplanetary-robot/SigmoidNumbers) I did everything by first inverting negative numbers and doing decode in the positive domain.

As I design the hardware, it's actually better to NOT do a two's complement inversion to do the decode, and keep the fraction as two's complement!

Also the 4 bit posit was just a simplification to help you understand the structure from a constructive point of view. posits can be of arbitrary length; they have a property I call isomorphic - so appending zeros exactly preserves the value of a short posit when increased in length; conversely, rounding a long posit to a shorter one reports the "nearest representable value".

You need to be a lot more explicit about what you’re asking. Look at the slides, searching down for “At nbits = 5, fraction bits appear”. Notice that every possible bit pattern is used and meaningful.

http://web.stanford.edu/class/ee380/Abstracts/170201-slides....

@espeed didn't ask if you contributed significantly to the lecture, they asked if you'd seen it. Please stick to the question!

If you are unsure why you were downvoted, I strongly suspect it's because politely giving and receiving due credit are important, knowing that one's interlocutor was involved in specific research is very useful information (to know what questions to ask), and it's probably kind of rude to call someone to account in this manner. Hope this helps :)

I was being sarcastic on a whim, fully aware that it would attract downvotes.

Speaking of number sets.

I never understood why complex numbers are considered one. I mean, yes they "are" a set, but besides that they are completely different.

All number sets I learned about did fill some gaps in one dimension, but complex numbers somehow added a new dimension.

Like real numbers stood in an entirely different context to rational numbers than complex numbers stood to real numbers.

You are mixing up a couple of things. This "dimension" you're talking about is probably the dimension of a vector space. The real numbers, as a vector space over the field of real numbers (recall that vector spaces are defined over fields of numbers), have dimension 1. However, you can consider the vector space of real numbers over the field of rational numbers. What is the dimension of this vector space? Well, the dimension of a (finite-dimensional) vector space is given by the number of elements in a basis for it. So how do we go about finding a basis for R over Q? A basis for R over R consists solely of the number "1", since given a real number "x" there exists an element from the field (in this case R), namely x itself, such that multiplying it by 1 will give you the number x. Yes, this sounds obvious, but that's how you prove that R over R has dimension 1. So, going back to R over Q, we see that "1" cannot be a basis, since if we pick, say, pi, there is no rational that we multiply 1 by to give pi. Moreover, there cannot be any finite number of real numbers that would make up a basis for R over Q, since then R would be countable (look this up if you don't know what it is). So the dimensionality of a vector space depends on which field you're considering. By the way, the vector space of complex numbers over the reals has dimension 2 (a basis is {1, i}), but over the complex numbers it has dimension 1. So, there's an "extra" dimension only if you consider it over the reals.

C (complex numbers) is a field, just like R (real numbers). It's an algebraically closed field, unlike R, so in some sense it's actually the most natural set to call "numbers".

https://en.wikipedia.org/wiki/Algebraically_closed_field

I'm not trying to argue for R being more a set than C.

In my head a one dimensional thing like R is fundamentally different from a multidimensional thing like C.

Well I was answering the question about why is C considered numbers - it's a field, hence elements of C behave exactly like numbers. Moreover, it's an 'algebraic closure' of R, hence a more natural choice for "all numbers", and it's a maximal one at that (i.e. quaternions and such are no longer fields and don't really behave like numbers, while C still does).

Edit: FWIW, I consider the terminology of 'real' vs 'imaginary' completely stupid and misleading. This terminology didn't really make it to other languages, e.g. in Russian it's 'material' vs 'complex' numbers, but they don't use the term 'imaginary'.

We do: вещественная часть и мнимая часть.

Oh yeah, forgot about that. Well, then Russian terms are as stupid.

What do you mean by “number set notation”?

Reply on news.ycombinator.com