I've been running between 300-500 Apache Solr search cores continuously for almost 5 years, starting with 1.4, migrating up to now 4.10.4 (5 and 6 are in the works), all with around 99.98% uptime (averaged on all the servers over time).
I have detailed logs and per-search-core stat tracking (queries, index size, query time) for the past 3 months and archives back further, and I have only had to remove noisy neighbors a few times (and did so quickly, by first isolating them on their own VM, then helping them move off to dedicated resources if needed).
Also, I explicitly state I offer no SLA in the support docs—some people are okay with that.
But yes, I'm always on call, technically. I've only had to fix 'emergency' scenarios about 3 times in the past 5 years though. Even security updates are automated via Ansible/Jenkins. I just need to log in and click a button and things are updated, or a new server is built.
I highly recommend Google's new book on SREs; whether a one man shop or a multi billion corp, the learnings are exactly the same!
Do you have a partner who can do support when you need time away?
I run a small game website. A couple of years ago it experienced an outage while I was away from home on a 2-day climbing trip. I didn't learn about it until I checked email the first evening. It was a terrible feeling to realize I'd been out having fun, and the site had been down for most of the day. I had to drive home to get the site up again. It was miserable.
That goes along with the 'keep it simple' philosophy; since the service has a very small public API and surface area, it's easy enough to diagnose issues quickly with limited analysis of log data and simple monitoring.
But no, I don't have any partner actively, but I do have a contingency/succession plan in case anything would happen to me (for the sake of my customers).