As another person noted Rancher sits at a level of abstraction ABOVE kubernetes -- you can set up Rancher to work with Docker Swarm, Mesos, Kubernetes, and Rancher's own libraries. Since this was how I understood it, I went with just vanilla Kubernetes first, and figured that once I got really comfortable with that, I could throw the Rancher UI on top of it (the install documentation is actually super duper short, it's just another container) when I got tired of typing things in at the console.
I didn't opt to go with RancherOS because CoreOS was already a step away from what I knew -- most of my machines are either arch, alpine (inside a container) or debian/ubuntu. RancherOS takes the stripped-down OS paradigm even further by dockerizing a bunch of system services -- I wasn't quite ready for that. CoreOS not coming with a package manager is already jarring enough for me (I'm used to it a little bit more now) -- I wanted to take a small step rather than a huge one.
They support Cattle (Rancher's own), Kubernetes, or Swarm for orchestration.
I talked with some of the folks that work at Rancher at the last Dockercon and IIRC they even had some rudimentary Mesos support.