Technically, [digital] computers only natively "store" high and low voltages. The interpretation of a sequence of those signals as "integers" is entirely up to you.
In all seriousness, representing integers is not a given either (e.g. two's complement vs one's complement, BCD, bignum, sparse integers, etc.). If you are going down in abstraction levels to describe floating point representation, you should not just call integers "native" blindly.
Furthermore, floating point is not the only way to represent computation involving real values. It is indeed possible to represent computation on real values symbolically in many cases and lazily compute arbitrary precision results for output-printing purposes.