Hacker News Reader

Comment by neild | Hacker News Reader

In my experience, the Apple Watch blood oxygen monitoring was horribly inaccurate. It would report wildly variable results, often telling me that I had a blood oxygen level of 80% (which, if true, would indicate that I should be getting myself to an emergency room ASAP).

Regular pulse oxygen meters are cheap and reliable.

View on HN

On their best days, they're accurate to within 2-4%. But so many things can trip up the reading, like melanin:

  As a result, for darker-skinned patients, oxygen saturation readings can read as normal when they are, in fact, dangerously low.

https://publichealth.jhu.edu/2024/pulse-oximeters-racial-bia...

When everyone starting looking at every percentage point of their SpO2 during COVID as if it were life or death, the FDA had to remind people of this:

https://www.fda.gov/news-events/fda-brief/fda-brief-fda-warn...

You would be unable to read an accurate pulse oximeter at 80% because you would have lost consciousness. Doctors have to worry about false negatives just as much as false positives with those things.

80% at sea level is very bad. 80% while asleep at 11,500 ft is not unusual. Ref: https://pmc.ncbi.nlm.nih.gov/articles/PMC10157825/

There’s a chart somewhere in there on mean sleep so2 by elevation

Big news for the 0.18% of the world’s population that lives above 11,500 feet, I guess.

Or sleeps on a plane

The FDA standard for blood oxygen sensing is within 6% absolute, 95% of the time.

So variability in the sensing is pretty normal, and you want to look at long-term trends rather than individual measurements.

The problem with consumer health sensors is they have both high random error and inconsistent systematic error. When your SPO2 sensor gives you 92% one minute and 98% the next while you're sitting still and it is almost always 2% under, you're not getting "noisy but usable" data - you're getting garbage.

They should hide the live reading and give you daily / overnight insights.

That caused me nightmares when I was first diagnosed with sleep apnea. I would check my oxygen levels during the sleep to see if my treatment is effective. Even though the CPAP machine would show a few short events Apple Watch would show levels as low as 75%. Thankfully in my next sleep study I learned that my oxygen levels were consistently above 95% and the watch is indeed very unreliable (how snug it is, which direction it is facing etc highly affect the results).

I’ve always felt the sport loops (soft w/ velcro) provide the best contact with wrist while not being too cumbersome. Very easy to tighten just before a workout or loosen before bed. All the while it stays planted on my wrist. Unlock the rubbery band it normally comes with, which is prone to sliding around and less easy to adjust.

Out of curiosity, which band do you use?

I also switched to the sport one and I like it because I sweat a lot and use it while swimming and it dries quickly. But if I don’t wear it uncomfortably tight while sleeping it gets looser probably because I move a lot while sleeping. One thing I noticed is that the biggest drops in measured o2 levels happen while I wake up to go to the bathroom. Normally it only measures while your wrist is flat and the watch is facing up but it is probably not able to detect it that quickly.

It may depend on skin type, body composition and wrist hair - perhaps the validation work used a skewed sample?

I’ve found the sensor to give stable results, with repeated measurements always within 2 percentage points.

And the results give qualitatively very reasonable data when I sleep at high altitude. The readings have a clear dependence on the elevation.

I haven’t cross checked against other meters, but my Apple Watch 9 sensor gives stable and reasonable results that match expected altitude trends. So yeah it may not be tuned to a wide enough variety of wrist types.

In contrast my Garmin and finger pulseox match exactly.

Yep, my Garmin also has matched the doctors office instrument to the 1% every time.

Which model do you have?

Epix Gen2 in 51mm

I really use the hell out of it. Yeah I can't play solitaire like an iWatch, but the battery lasts 7 days in the backcountry, the flashlight is unbelievably handy while hiking/camping/boondocking, and it helps me be healthy with all of the data. Being able to trigger my inReach is also a nice touch. It's definitely a tool rather than a fashion piece.

I don't know what Garmin you have, but I'm about half convinced that my Instinct's heart rate measurement is implemented by a PRNG. It's frequently off by 50% from a count/time cross-check.

It does not inspire me to move up their range when this watch eventually dies: if they can't get the basic feature working, I have a hard time seeing how they're going to manage anything trickier.

Heart rate measurement on my Garmin (fenix 7 pro range) is great, the pulse ox measurements are shit though, and absolutely rinse the battery life.

https://www.youtube.com/@TheQuantifiedScientist

That guy is a great reference, and through his videos you can find various measures where he compares devices against reference devices (e.g. the Polar H10 for heart rate for instance). A lot of the reliability of these devices relies upon a tight fit as well.

Accuracy varies wildly with each model. Obviously the more expensive ($400+) ones are better, but Garmin devices are generally good with heart rate tracking. Same for Apple watch, Pixel watch, and a few cheaper options from Huawei and Xiaomi.

Heartrate is generally very good but only as long as the fit is tight. Blood oxygen on the other hand is a joke.

I've never had any trouble with it on my series 9 (purchased Dec 2023 just before the feature was disabled). It's always closely matched the fingertip meter that I have. Which is to say they both always read >= 95% for the most part.

Do you have tattoos? I find most sensors cannot read my vitals through my tattoos.

Indeed, just generally this is a silly feature that was used to sell updated devices, but has almost no value to end users. There is shockingly little diagnostic value of the reading unless you are in such a critical state that you likely want something better than an incredibly unreliable and inaccurate smartwatch feature cram.

For anyone remotely healthy, 100% of the time your real value will be between 95% and 99%, and there is almost no diagnostic value to it. Heart rate is actually interesting and is something you can learn from and work towards. SpO2 is just "eh...neat".

as some one whose family passed away due to pneumonia, spo2 is a life saving feature if we had that back then. probably 99.9% of the time spo2 number is good enough. but the value is really about the left 0.1% . of course the false positive rate should be low enough.

> For anyone remotely healthy, 100% of the time your real value will be between 95% and 99%, and there is almost no diagnostic value to it.

Sure, but if the value is less than 95, that does have diagnostic value (if it's accurate)

Sure, but unlike heart conditions where people often have no idea (about afib, or even abnormally high or low heart rates), people generally know when they have respiratory difficulties. Like the other comment noted something about family having pneumonia, and I cannot understand how the watch would have made their situation better. If someone in that state wasn't already seeking medical advice, it's hugely unlikely a watch saying "yo it's bad bro" is going to help.

It's like heralding a G-sensor in your watch telling you that you're falling. It's likely pretty obvious already.

Seems to me, it has some value (again, if it's accurate) for letting people know about sleep apnea; especially as part of an overall sleep tracking dodad.

I've got enough mild asthma around me that we have a finger pulseox (or two cause we "lost" one and found it later) and I've started yelling at sick people to check it once in a while. Cause they don't usually think to, but sometimes it lingers and by the time they decide to go into an office, the numbers are pretty low.

Of course, we're not on the Apple bandwagon and stopped wearing watches once we got used to having pocket watches again.

Wouldn't you already be super dead with a true reading of 80? Or at least unable to cognitively interpret the reading?

That's definitely a danger zone for healthy people but interestingly enough people with things like COPD may have a blood oxygen level in the 80s and while that is indicative of the disease, they may be totally stable and may not even need oxygen [1].

[1] https://www.drugs.com/medical-answers/normal-oxygen-level-so...

My grandmother's heart was completely fucked, so they'd have to adjust the alarms on the hospital monitors after checking their files when she went in. It's like "OK, well that's the problem... consults notes... Nope, apparently that is normal for her, now lets figure out what's actually wrong". It wasn't keeping regular time and it would sometimes skip, but apparently it was pumping well enough to keep her alive for several years.

Normal in humans is definitely relative and medicine has tended to assume that if we average 1000 humans (in too many cases, 1000 white college age men) that's what human normal is, which is crazy even beyond obvious problems like " people normally have 1.999 legs apparently".

Bodies are generally pretty amazing in that sense. As long as things go out of spec _slowly_, we will often adapt quite well. In the short term, we will tend to balance even fairly extreme changes out through various chemical processes and in the long term people can even develop heritable genetic changes. (E.g., how people acclimatize and have in some cases adapted to living at higher altitudes[0])

[0]: https://en.wikipedia.org/wiki/Effects_of_high_altitude_on_hu...

I had some momentary readings lower than 80 during a sleep study prior to going on CPAP. I didn't snore, or choke, or anything. Just ... didn't breathe. With CPAP, 98% all the time.

Obviously not. I did the experiment with a finger pulse ox and a Garmin device to check. You just hold your breath. My Apple watch was pretty good at it too. It's very uncomfortable and you'll get visual snow but I'm not dead, super or otherwise. Use your hand to clamp over your mouth and shut your nostrils if you want to try.

Reply on news.ycombinator.com