The specific kind of chaos here implies (roughly) that a system starting in state X and a system starting arbitrarily close to X will at some point in the future behave totally differently. Not different as in "they'll drift further apart," but different as in "this one rolls off the cliff and this doesn't."

It's a fascinating field of dynamical systems, which is one of the coolest subfields of math (since the world is dynamic). Here's the wikipedia page https://en.wikipedia.org/wiki/Chaos_theory

No. Chaotic dynamical system can be completely deterministic.

Not only can chaos be exhibited in fully deterministic systems, it can be exhibited in absurdly simple systems. The logistic map is an incredibly simple iterated function that exhibits highly chaotic behavior.

It remains one of most mysterious things in mathematics, as far as I can tell. Studying it has consumed a lot of very smart people's careers.

The fact that the answer to this is "no" has fed entire subfields of mathematics for a generation.

The double pendulum is a nice example https://en.wikipedia.org/wiki/Double_pendulum