Create your own smart baby monitor with a RaspberryPi and TensorFlow | Hacker News Reader

Create your own smart baby monitor with a RaspberryPi and TensorFlow

towardsdatascience.com

179 pointssebg154 comments

Please, please, do not use something like this as the only method of monitoring a child’s safety. This sort of thing has given me honest to god nightmares.

My day job is managing development and infrastructure of internet connected cameras, with motion detection, notifications, and all the rest of it. Every now and again someone new will start on the marketing team and put together something about using them as baby monitors, swiftly followed by the engineering team descending on them to explain there’s no way we can guarantee sufficient reliability for something where the worst case failure mode is someone’s baby dying.

If you’re going to insist on something like this you need to be absolutely sure that you have sufficient monitoring in place. If it loses video from the camera, it needs to alert. If it loses its network connection, it needs to alert. If whatever is meant to be receiving alerts doesn’t receive a heartbeat from the camera, it needs to alert. In all those cases that alert needs to be obnoxious as hell, and guaranteed to fire.

And the machine learning just needs to go away, unless somehow you’ve obtained video of every way in which a child can end up seriously injured, from multiple angles, with which to train it.

I've been an SRE before, and have kids. I don't think there's any real expectation that a baby monitor is a reliable alerting system that can be depended on. There are definitely people in the thread who believe that's the case, and it also looks like some products are built to those high standards, it's not legally required, and I don't think it's even desirable.

I don't hold the people who care for my children to the standards that you would hold a video camera (IE, they're allowed to go to the bathroom with the door closed without alerting) so I can't see why I would expect it of the technology that approximated it.

I agree with the sentiment here; there seems to be an impression that a baby monitor is a safety device. I'm not sure it is.

We use a baby monitor during naps and overnight. It's a good way to relay any noises and reassure us he's ok, but you can't really see enough detail on an overhead camera to see they're breathing or anything like that. It failed a few nights ago, and we woke up when we heard the crying through the couple of doors. (If we were awake, downstairs and it failed, we would have tried to fix it pretty quick, but we have the fairly basic backup strategy of going to sit upstairs with him.)

Technology cant prevent disasters, but we still use Tesla autopilot, I suppose some even with babies in the back.

A baby monitor is to detect if a baby is crying not if a baby is dying. It seems like your exaggerating the user need (system engineer in HW+ SW and father of 3 here). Most use cases for baby monitors are “im relatively nearby, but want to hear if the kiddo’s waking up”.

This solution is great.

This exactly. My baby bro was a preemie with potential heart issues (he grew up fine), but he had a proper heart monitor connected with sticky leads 24x7 for the couple of months. It had a really loud siren and decent battery built in.

When you actually need health monitoring, you don't trust a baby monitor.

I don't even know anyone who uses a baby monitor. I think we can get by. If someone relies on one to the exclusion of actually checking on their child from time to time, the failure of the monitor is probably the least of risks.

I know we were gifted one, and it didn't ever get used.

When the baby was brought home he shared the bedroom with the pair of us, and it was immediately obvious if he was crying.

Later when he moved to sleeping in his own room we could still easily hear the noise of crying, as the room was immediately adjacent to our own, and we left the doors open.

I remember we setup a routine at some point where one of us would sleep in his room with him, leaving the other one free to sleep through the whole night.

Maybe if you live in a mansion, and you can't hear a baby crying from the other wing it might be necessary. But most of the parents I know have always been close enough to hear crying just with their ears. (Although I know there certainly are days, especially early on, where you'll hear but be half-asleep and not really do anything!)

Around here we have a custom of letting your kid have a nap in the baby wagon outdoors, even in winter. I guess people started using baby monitors for that by now. When I was small, mum just left a window a bit open (1970-ish Sweden).

This is even done at daycare down to -15 degrees C. However they do have someone present next to the kids at most times.

There are lots of stories of swedish people going to USA and getting arrested for leaving their kid without supervision outside while being 5 meters away inside a store for five minutes. We are used to a bit more relaxed way of parenting I guess :-)

I'm in Finland (although not Finnish), and we did the same thing. Putting the pram on the balcony, while the child was asleep. We'd usually leave the door partway open so we could hear if he cried.

But to be honest once he was having a nap he was generally predictable, and quiet. Especially the lunchtime naps after food - we knew he'd be absolutely fine asleep for 90 minutes and then he'd probably wake up. On a long day he'd sleep for two hours, but that didn't happen so often.

Interesting reading for people in other parts of the world:

https://www.bbc.com/news/magazine-21537988

Same here in Norway.

Its not necessarily about missing out the crying or not, its about hearing it sooner rather than later. You speak of "the baby". If you got two children, you don't want the youngest crying to wake up the oldest (or vice versa). You really don't want a toddler with a lack of sleep. Its causing temper tantrums extraordinaire. Thus far, I got lucky with ours. When she sleeps, she's gone in la-la land, her brother crying be damned. Or so it seems.

True. As my coworker, who has 3 kids of her own, always says, if they're going to stop breathing, its not like its something youre going to hear. They could be in a crib in your bedroom at night, and you'd never know. The monitor is just to see whats going on, are they even sleeping yet, who threw their stuffed animal out, etc...

Add O2 sensors to it?

The possibility of failure is maybe a good reason to consider not selling this as a product for fear of lawsuits because if you have millions of these out there what happens when one fails and a baby dies of SIDS. However I suspect baby monitor companies have figured out a trivial legal or technical solution to this issue and I really doubt baby monitors are engineered to the standards of a life saving medical device.

If you’re using a baby monitor to observe a child in safety-critical situation, you’re doing something wrong.

https://www.nhs.uk/conditions/sudden-infant-death-syndrome-s...

Even if you get the perfect training data, the state-of-the art neural networks will not be 100% reliable. Convolutional neural networks work by recursively recognizing shapes. In theory, the first layer should match the picture on something like this and conclude "it's probably an eye":

  o o X o o
  o X X X o
  o X X X o
  o o X o o

(above, "X" denotes low relative brightness, "o" denotes high relative brightness).

The next convolutional layer is supposed to see something like this and conclude it's a face:

  (eye)        (not a bumper)    (eye)
  (not a door)    (nose)     (not a wheel)
  (not a shovel) (mouth)   (not a window)

That's in theory. In practice, each input to the second layer will be a vector of probabilities. Like 90% eye, 80% wheel, 1% door. The output will also look like a vector of probabilities, and the layer is supposed to figure out a linear formula matching inputs to outputs that results in the best statistical outcome. So it can very easily come up with kernels like this:

* A: (an eye or a wheel) and maybe a bumper to the right of it

* B: (an eye or a wheel) and maybe a bumper to the left of it

* C: (two wheels) side-by-side

In this case, the baby face will be encoded as A together with B and definitely not C. Because each next level produces a linear combination of the previous level's outputs, the "or a wheel" parts will get nicely canceled out as long as the "2 wheels" kernel input is a low value, concluding that it's a face with a confidence of 99%. Until the light falls from a different angle and the baby holds a round lego brick near their face, suddenly yielding 80% confidence that it's a car.

Drawing parallels to organic life, neural networks are good at replicating inborn instincts. Like animals instantly recognizing predators, food or poisonous plants. But they are still far away from having a situational awareness needed to make judgement calls in most real-world situations.

> Convolutional neural networks work by recursively recognizing shapes.

Bold claim. CNNs work by jumbling an image through a series of linear transformations and non-linearities

You are... aggressively anthromorphizing

I agree. I feel like the other people replying to this comment is misunderstanding the intention of having a baby monitor and the state of a mind a new parent is in.

I still remember the panic that set in when the cheaper camera loss signal and didn't notify us. The good monitor made a loud beeping sound if it's losing connection or is low on battery. It's a peace of mind knowing there are safe guards when the technology fails you.

I don't expect a baby monitor to keep track of the child's health status, but I do expect it to work reliably as digital link to the room.

Lol... I don't think you understand how baby monitors are used.

> If it loses video from the camera, it needs to alert. If it loses its network connection, it needs to alert. If whatever is meant to be receiving alerts doesn’t receive a heartbeat from the camera, it needs to alert. In all those cases that alert needs to be obnoxious as hell, and guaranteed to fire.

An exercise to think about this is the baby monitor should DEFAULT to the alarm state. Now, what conditions have to be met to turn the damn alarm off.

What?? What do you imagine a baby monitor is for?? How good do you think the ones on the market are??

I guess the disconnect is between "what is the monitor used for" vs "what can a baby monitor producer be sued for". Their worst case is the worst possible litigious customer.

Although I don't get why the engineers argue that one rather than lawyers. It's a disclaimer / liability issue, not a technical one. We know the tech will fail at some point.

So what you're saying is, to save a baby we gotta hurt a baby

I know this was intended to be a joke, but it didn't really land with me.

No thank you. Baby monitors should not be connected to the internet, rely on machine learning to make decisions for you, or rely on push notifications to gain your attention. Keep them simple, local, reliable, and dumb AF. In this use case, less is most definitely more.

I ended up getting a simple video monitor from BT that is an "appliance" - it's not got a wifi connection or an app.

It works great! And it'll keep working until we don't need it. No software updates, no discontinued support, no questionable data storage policies on chinese servers. Just a monitor.

And it turns out that once the baby's passed ~4 months, the video isn't too important anyway. It's mostly "Is he crying or not".

Exactly, my #1 criteria for a baby monitor was that it does not connect to the internet! Maybe not enough parents have heard how people hack home networks and terrorize children through cameras.

Thanks. This is just a hard no immediately. Not only for the health of the child, but does ML have to be the solution for everything? I think a Rube Goldberg machine would be more reliable than this.

I received a hand me down Motorola baby monitor + camera. It doesn't connect to the Internet. It'll beep if the camera is disconnected. The monitor has power save capability (video feed off, sound on), let me know if power is low....It has everything I need. I just needed to replace the battery for a very cheap price because the charge didn't hold as long.

> Baby monitors should not be connected to the internet

This one connects to the network, not necessarily to the internet. (the latter is up to you)

There's a lot of negativity in these comments, but I'm excited because I'm working on exactly the same thing. I have a raspberry pi 4 hooked up to a PS3 eye camera/mic, and have been tweaking it to only forward audio when my child cries.

I wasn't going to start with ML - just send the audio if it was over a certain threshold for a certain amount of time (say, 5 or 10 seconds of noise that's above the normal ambient background for that room).

For everyone chiming in about how this is terrible parenting, I know it comes from a place of concern, but I personally am not worried about using this, and I can imagine that the author of the article is probably thinking similarly to me.

I would not use this monitor in a situation where the health of my baby was in any way suspect. My baby tends to make lots of noise at night - shifting around, sighing to herself, and occasionally uttering a single piercing wail for absolutely no reason for just one second (sounds like an audio hardware test, just the brain exercising the circuitry during sleep). While none of this bothers the baby, I am a very sensitive sleeper and having her in my room is completely destroying my sleep.

I'm going to use this to be able to move her to another room. It's a room adjacent to mine. It's close enough that if she works herself up to her full volume I'm sure I would still wake up and hear her. This is so I can filter out her background noise, but still respond to her night wakeup crying without having her reach her full protesting volume. I think it's nicer for her that way. It is not dangerous; just a means for me to get some more sleep while still being able to respond promptly when she wakes up.

> I wasn't going to start with ML - just send the audio if it was over a certain threshold for a certain amount of time (say, 5 or 10 seconds of noise that's above the normal ambient background for that room).

In my experience this might very well be sufficient. I have a baby monitor (eufy spaceview) that does just that, and as long as you can set a volume threshold to account for expected noise (e.g. a white noise device) I doubt an ML-powered version will add much value for this use case.

I also have an eufy Indoor Security Cam, which does some more intelligent audio processing, such as detecting when someone is crying, and is able to send notifications based on those alerts. It has worked flawlessly in my experience, but also hasn't provided additional value over the baby monitor that only detects noise over a certain volume.

In the end it doesn't really matter what kind of excessive noise you're hearing from your baby's room. If its loud enough to trigger our baby monitor I'd like an alert even if it doesn't sound like crying.

The security camera we have also doesn't trigger an alarm (on Apple devices, besides from the notification sound), so that's another thing to take in to consideration.

Other comments notwithstanding: how does a new parent find time for this???

You have less time, but you learn to organize and structure your time better. You also realize that if all you do is carry the baby, feed the baby and change diaper you will lose your mind, so you begin to find ways to push out work in 15-30mins increments. At least that's my story. :-D

Lucky you. Baby #1 required beyond constant attention, literally like 30hr per day (healthy just needy). Baby #2 is sweet and easygoing, but then there’s 2 of them...

Agreed. We found Baby #1 to at times be a full-time job for two people in the first couple of months.

Down-time is precious precious precious. Either catch up on sleep, or relax as best you can. For me personally, consumption has largely taken over from creation which I am not sure how I feel about but its working so far :-)

You feel OK for now.

There might be depression and increasing anger, once you realise the kid is still needy and your partner is now occupied with #2.

On a more serious note, for me consumption at this point in my life is a vicious cycle that does not make me a happier or better person - yet the circumstances keep pushing me towards it.

I have compassion for your experience having two of my own where one still is difficult after seven years, but I'm here to nitpick and win the pedantry award on your addition of "literally" when it's impossible to take 30 hours per day, as a day is only 24.

Do I get to steal that pedantry award if I point out that two parents can take 48 person-hours a day?

But the baby itself can only get attention for 24 hours per day. I think you need to give that pedantry award back. :P

No. Because why the child can only get attention for 24 hours per day, other tasks primarily focused on the child (buying nappies, cooking baby-friendly meals, washing bottles, etc.) are what Parent #2 is doing while Parent #1 looks after child.

I read it as “required beyond constant attention, literally like 30hr [of attention] per day”, not “required attention beyond constantly, literally for like 30hr per day.” ¯\_(ツ)_/¯

Not a fan of pedantry. Would like to see less of it in the world.

Although I meant my comment figuratively and in abuse of the word “literally”, I now feel compelled to defend it with lame grammar Nazi tactics:

I said the baby requires attention for 30hrs a day. This does not mean it receives that much attention, rather that a prerequisite to it making it through the day is 30 [person-]hours a day, indeed split between a few people. One person entertains, the other washes bottles, etc.

Tbh as a parent of 2 kids I barely have the time to write HN comments, I’m positively surprised how coherent it is in the first place...

You will not win that award. From Merriam Webster:

> 2: in effect : VIRTUALLY —used in an exaggerated way to emphasize a statement or description that is not literally true or possible

Harder still is getting it past the spouse regulatory approval process.

My wife has been pretty accommodating for any nerdery I want to engage in for taking care of the kids. The only project she really put her foot down on (and probably correctly so) was about building a crib myself. She figured there were a lot of safety standards about materials, size, bar spacing, etc. that were best left to the professionals.

I don't know your woodworking experience, but I don't think building a crib is dangerous at all. The safety standards are pretty straightforward and they are very clearly spelled out. And for me at least, no mass manufactured piece of furniture is even close to a minimum quality that I'd build myself.

On the other hand, a crib is a particularly annoying piece of furniture to build. First, it has to be able to break down, so that is an engineering challenge if you don't have experience with that. Second, it is a particularly monotonous thing to build, especially if you go with spindles all the way around. Cutting, sanding, assembling and finishing ~50-60 spindles is a lot of very boring work. So you may have dodged a bullet on that one.

Where my partner usually steps in is not safety, but time.

It is great if you're able to build some things yourself. But the kid and in particular your partner need your hands on deck, not on the workbench for days on, just to end up with an unfinished crib. And there are so many other, more important things to do that cannot be outsourced that easily.

And once the first kid is in the house, expect interruptions. Oh, kid has a "mild reaction" to a certain vaccination? Forget sleep, forget "stuff that needs to be done", survive while keeping the basics in order. And with more than one kid expect to be on call for every minute you are home. Nothing can go by uninterrupted if you have small kid and a toddler and only four hands.

I envy everyone that has a different experience. Grandparents/relatives help a lot. COVID-19 does not.

Projects suddenly need to be small or non-critical.

Yes, I fully understand the time problem.

I put off building the crib until my son was 4 months old expecting to be able to finish it in 2-3 weeks. It took over 8 weeks and by the time it was done it was at least 1-2 weeks overdue. And a lot of the work was done between 10pm - 2am when everybody was asleep, which took away from my sleep!

I'm still glad I built it (now that it's done!), but yes, outsourcing it certainly would have been the correct choice.

Got a picture of it?

Here you go: https://danbirken.com/crib.jpg

Wow, very neat!

Exactly.

One fun exception I had was building a climbing frame in our garden, with a swing and slide. Took literally days, had to reorder parts, go shopping for tools.

But, it was just before lockdown, and even playgrounds were closed where we live, so in the long run it was a life-saver.

It helped that I did it when the granny came to visit...

If you are building the crib after the baby is born, you are doing it wrong.

I generally agree, but only because there’s no time for a big project like that with an infant around

But not everyone uses a crib right away — we do a combination of bassinet and co-sleeping with our ~4mo. Probably time to still build that crib if woodworking is your down time

Edit: though my wife would never have gone for an idea like that, she got the nesting bug pretty hard

I use the just-in-time method of building things. I was spending the time before he was born building the changing tables and dressers for all his clothing.

My wife built a beautiful wooden crib (despite no previous experience with finer carpentry, mortise and tenon joinery, etc.) and it's worked out wonderfully. I think she based it off of this: https://mattcremona.com/uncategorized/walnut-and-maple-crib

Not if your s/o is either a nerd herself or hasn't major issues with weird hacks. I got approval for converting our living room into something that could pass for a crossover between pub and disco - a wide assortment of liquors and actual stage worthy, DMX controlled lighting and effects. So much power that I had to run a new 32A circuit and re-wire the entire electrical distribution...

You get a few months of lead time to get the house ready. Don’t waste it.

100% agree with this.

If there are any pending parents reading this, get all those outstanding things done now.

"We'll do that once baby is born" - nope! Get the nursery set up as best you can, get the buggy/stroller unpacked and assembled, get a stockpile of nappies/diapers, fix that broken light switch or dripping tap etc, get the car serviced and the tyres changed etc.

If you have any side-projects (or even games you are playing through) on the go, get them to a good place where you can stop and leave it for a while. I had grand ideas of continuing to hack away at some projects - even if it was just 15-30 minutes at a time - but of course like everyone says there is no time for that. I should have tied things off before baby arrived so things were neat and tidy (i.e. not in the middle or some epic refactor or juggling a bunch of PRs that need to be merged etc), left myself some notes, and then come back to it in 6-9 months when you get a bit more time again.

There wont be time when baby arrives. Good luck! :-)

Time can be found as a new parent, but it's really tough. My biggest question is, should a new parent spend time implementing this solution?

More of a philosophical question, but from my POV if you're spending time implementing this, you're not valuing your time enough. For the creator (Fabio), it's arguably time well spent: define a problem, use your creative muscles to a functioning solution, document it and show the world your creation. All well and good.

Tinkering to implement Fabio's solution, now that's where I see it being low-value usage of your own time. There's an opportunity cost to it; you could be spending time finding your own problem and creating a solution you could display.

>how does a new parent find time for this? Speaking as a new parent, we have a knack for lighting our time (and sanity) on fire in order to provide things of marginal value to the baby.

I'm talking things like boiling water before making formula, sterilising bottles and dummies, installing and then looking at baby monitors, feeding baby at regular intervals (even in the middle of the night), making the latest "Montessori" toys and cloth nappies, etc. etc.

Anything that helps us fruitlessly burn ourselves out to the point that quality time with our newborn is seen as yet another problem will be welcomed with open arms.

ha. I think it's like when I ride my bike. I have idle brain cycles and I start thinking about improvements I could make to the bike-riding and that's how I got a bluetooth media button to skip around in podcasts while I ride (one ear only).

this is for someone that has idle brain cycles while babysitting. (and possibly can't go anywhere and has a pi nearby)

2 parent family. Sometimes with grandparents or aunts and uncles.

This other parent do the work, 60% of the time for the first child, then 15% more for each and every subsequent new born.

To be clear, it’s a statistical statement not what I vouch for.

This is great and all, but it has to be 100% utterly completely reliable with no false-negatives.

If it doesn't tell you (i.e. false negative) even once, then you can guarantee that it will never be trusted ever again, especially by the birth parent.

We've gone through a range of frankly awful monitors and cameras that simply were not reliable. E.g. cameras that would regularly go offline for maintenance between 3am & 4am (precisely when you need it...), or cameras that would freeze and show you a sweetly sleeping silent baby that turned out to be simply a frozen frame from 45 minutes previously etc and the live stream had failed, cameras that would lose wifi for no reason that required intricate sequences of button-holding-down-for-x-seconds-then-tapping-then-releasing-etc (not great when you've had about 30 minutes of sleep over the past 36 hours), app-connected things with awful apps that crash or don't quite work right unless you do things in the exact right order etc. We even ended up with a digital clock in-frame for a while so when we looked at the video we could tell if it was "live" or not... this is how unreliable a lot of baby monitors you find on amazon or high street retail.

Once these things fail you once, they cannot be trusted again. Ever.

Perhaps me and my wife are just ultra neurotic, but there you go.

We ended up with a fairly reliable set up for multiple redundancy that seems to work:

- a DECT-based monitor from BT that actually does detect cries (you can entirely mute the intercom but it will audibly-beep and flash a warning if it hears a cry - also does temperature read outs and alerts). 100% ultra bullet-proof reliability so far.

- a Google Nest camera (only camera we found that has been acceptable in terms of reliability & non-shonky app - it does mysteriously "go offline" from time to time, but it sends a notification & and email as well as the app being clear it is offline, and usually recovers on its own fairly quickly). Expensive compared to other options (and not specifically a "baby monitor") but worth it I think (we do not pay for the subscription though)

- an "Owlet" heart-rate and oxygen sensor (although this has poor hardware reliability - the charging port physically crumbles and the "sock" that the sensor goes into is easily moved to a bad position by the baby squirming around.... although the fail-safe there is that it will alert if this happens).

Did you know that billions of babies were reared before video monitors were even invented?

Yes, and I also know that the parents of those billions of babies knew there wasn't a monitoring system.

We used a Summer Infant video monitor for a while, but eventually the receiver broke.

Then we used a Yi Dome camera for a while, but it craps out of a few seconds occasionally and when it does the app mutes the feed when it resumes, making it useless at being a baby monitor. Also, privacy concerns since you had to access it via the internet.

We ended up going with a generic IP camera - Amcrest's 1080p wifi camera, to be specific (would have to go look up the model number if people want to know EDIT: IP2M-841B). You can change just about anything about the settings you could want to (such as adding a timestamp so you can verify it hasn't crashed), including disabling access from the internet so you can only access the monitor from the LAN. It's been rock solid for ages - there is a setting that will reboot the camera once a week, but you can set the time, so I set mine to reboot about an hour after the kids normally get up so it never bothers us.

We just have the feed pulled up on an Amazon Fire HD8 tablet that we got for $60, lets us get a 1080p video stream with an ONVIF-capable camera. The Amcrest app isn't terrible either, only downside is that there's an option for 1, 4, 9, or 16 cameras - but I only have 2, so I have to waste half my screen (and the app will only play audio from one stream at a time). And if you don't want to use their app, once it is set up and connected to the network (which you probably don't even need the app for if you are hardwiring with ethernet) you can connect to it via IP address with any RTSP/ONVIF app or even via VLC, or once the firmware is upgraded you can view the stream in the browser.

It has motion sensor capabilities but I haven't experimented with them.

I think you put too much trust in technology. I have used baby monitor for a short time, but I have never ever walked out of the house with one - if my baby cries I want to be able to go to it as fast as I can - so even if my baby monitor is not working I will hear the baby, just a bit later maybe.

So the devices described in the article is great, it can show me that baby is crying, not only using sound, but also I can use it to ping me in a visual way - which I always missed from any baby monitor.

Have you considered therapy? Not just to provide support for your mental health, but also the (future) mental health of your child.

I mean this from a place of kindness and compassion. As a father of two, I can't imagine going through multiple baby monitors and eventually settling on a three-device redundant sensor array that sounds like it belongs more in a NICU rather than in a home.

This is with therapy! :)

Everyone's scenario is different. We have our (private & personal) reasons why this level of monitoring is something we need for our baby.

I hope all is well. I did not consider the possibility of pre-existing conditions. I'm sorry.

The V3 Owlet has magnetic charging instead of the port, which will hopefully be more durable.

So, don't get me wrong. This is a fun project and a neat little device. I don't mean to take away from it in any way.

However it is really important to consider why baby monitors are so primitive: because the cost of a false negative is huge. I didn't see any mention of this in the author's experiments (only a '>98% accuracy' note). So let's talk about this a little bit: is "accuracy" what we want? Probably not---I don't care if I get accidentally notified, but I care very much if I don't get notified when the baby is crying. So you want to weight your classifier's predictions heavily against false negatives (at the price of false positives). It would be good to make an ROC curve to characterize this behavior. More importantly though, any predictive model assumes a stationary distribution; i.e., training conditions accurately reflect test conditions. But will they in real life? What about when your neighbor's house is under construction? Can interference from chainsaws cause the model to fail to detect the baby crying? What about the dude down the street with his super loud motorcycle? What happens then? I bet the training set doesn't have situations like this.

I really, really don't want to come off like a wet blanket here. But I feel obligated to, because this is a model that directly impacts the welfare of a human, and so we should at least talk about or discuss potential drawbacks. (Again, cool weekend project, just, we need to be clear about the implications of outsourcing the decision of whether the baby is crying to a black-box model where we can't interpret what it's doing.)

> (only a '>98% accuracy' note). So let's talk about this a little bit: is "accuracy" what we want? Probably not---I don't care if I get accidentally notified, but I care very much if I don't get notified when the baby is crying

Presumably this is 98% accuracy per sample or something, not per session of crying. I wouldn't want to leave my kid completely unanswered 1/50th of the time if he actually needed me, but I think with the model as trained, regular crying would eventually get noticed and that's fine with me.

Somewhere else on the thread though somebody else called out the real downside of this model: it overfits on regular crying. Kids make all kinds of noises. Mine made a completely different, eldritch noise one time when he noticed a spider, for instance.

> the real downside of this model: it overfits on regular crying

I'm with you there, that's a big downside too, but that's not the "real" downside---there are like seven different downsides present with the data science going on here and it's hard for me to say which is the biggest issue, because they're all issues. Data science is not trivial!

This reminds me of the home-built garage door opener that popped up here a few months ago. When it comes to safety devices, there are literally man-decades of expertise that has gone into these devices. It is never a good idea to rely on something you whip up yourself over the weekend or convince anyone else that they could.

Was going to come here to say something very similar. I'm no luddite, but some things really shouldn't be left to tech like this...

Maybe not this specifically but a similar tech should be considered to detect babies in hot cars.

Automobile manufacturers are doing this now, IIRC. With sensors on the seats to remind you that you put something heavy into the car.

Hmm, I don't think so - seat sensors are used to enable / disable passenger airbags, my memory is that they only register weights over 40lb / 18kg.

> In 2016, General Motors took the lead among automotive manufacturers by introducing the Rear Seat Reminder, a technology designed to nudge drivers to check their back seats as they exit their vehicles. It uses an audible alert and a front panel message to tell drivers to check the rear of their vehicle for occupants.

> Rear Seat Reminder technology became standard on all new Chevrolet, Buick, GMC, and Cadillac four-door sedans, SUVs, and crossovers starting with the 2019 model year, and also will be standard on all 2020 model year GM pickup trucks, said GM spokesperson Phil Lienert.

> Kia, Nissan, and Subaru offer rear-seat alert systems in many of their models, according to Car and Driver, and Hyundai announced on July 31 – National Heatstroke Day – that it planned to incorporate the technology across all models by 2022. One of Hyundai’s newest innovations is the Ultrasonic Rear Occupant Alert, in which a sensor can detect the presence of a child (or pet) and activates a loud horn if the driver leaves with the child inside.

https://mashable.com/article/car-seat-alarms-prevent-hot-car...

Interesting, though hard to see how that prevents people from leaving children and pets in the front seats...

I'm not sure where you live but in the United States, in some states, children are required by law to ride in the back seat of an automobile until they are, I believe, 8 years of age. I believe that various state and federal agency recommendations are for children to ride in the back seat of an automobile until they are 13 years of age.

Presumably, then, parents are not putting their young children who cannot operate a car door into the front seat of a car and leaving them.

Pets? Sure, that's a different story.

Fearing the worst outcome ought to inspire you to make the tech better, not avoid it altogether.

But the tech exists. It's not like we're using tin cans and string and trying to replace that with a gizmo; we have the gizmo, and now we're trying to make an open-source version of it. While the goal is admirable there's no real benefit from using a less-tested Raspberry Pi project. There are less risky ways to learn the same lessons - a video walky-talky maybe.

As a father, I generally agree with what you both are saying regarding being conservative with tech choices on delicate areas such as babies, but I‘d also add that 99% of situations of my baby making random noises (that also trigger a baby monitor) are not something that are going yo have an impact on the baby long term. Most times my baby has lost the pacifier in the dark, and finds it before I reach upstairs (or annoyingly, between me waking up, and me reaching my bedroom door).

Also there’s the scary converse: some important things do not make a noise, such as a baby suffocating in her sleep.

> So you want to weight your classifier's predictions heavily against false negatives (at the price of false positives).

Which makes me think that a simple trigger based on ambient sound level probably does the job... I suspect that many baby monitors work that way.

This also actually makes sense because I likely want to be alerted in case of noise in general rather than just cries, just in case.

I have to agree - I like tinkering and hacking as much as the next dev, and I don't like to be negative, but... what is the point of AI/ML here, where a simple sounds trigger works just fine? What problem is it actually solving - I only see this approach causing problems.

Forgive me for bringing it up, but this is also the reason why smart gun technology is a complete dead end.

Cars are every bit as dangerous as guns. But no one here seems to be in such a panic with Tesla or self-driving cars.

I feel there is a bit of undercurrent of Luddite sentiment going around here. Lots of people mentioning how "primitive" a baby monitor is or should be. But is it? It's an LCD video screen powered by modern advancements in battery tech connected wirelessly to a CCD video camera, with sensing technology to detect when a baby is crying and even provide other information such as room temp. None of this existed 20 years ago. Not to mention that between now and back then, baby monitors went through a long phase of being hot garbage. Many still are garbage.

> I feel there is a bit of undercurrent of Luddite sentiment going around here.

Oh, I'm no Luddite, don't get me wrong---I'm a machine learning researcher. I have no problem with data science. The difference between all of the complex technology you pointed out inside the baby monitor and what we're talking about here is that all of that complex technology is robust and, for the most part, well-designed! The data science work here has tons of issues, and I think the vast majority of reactions are reacting to that: machine learning and data science really can be useful... but not if you apply it really badly. Someone else commented elsewhere in the thread that a simple thresholding algorithm would be just as effective---and not suffer from the myriad potential problems present with blindly applying TensorFlow because it's cool.

I wonder if there are huge advances to be made in suppressor technology? Other than the noise, guns are basically perfect in their intended function. I guess you could try to further reduce recoil.

There have been some pretty fascinating experimental weapons over the the years. For example, the H&K G11 [0] used caseless ammunition. The potential advantages of that are no need to eject a spent casing and a soldier can carry more of it since it weighs less.

[0] https://en.wikipedia.org/wiki/Heckler_%26_Koch_G11

That would be a game changer for sure.

If we could get a caseless ammo system to work I think that'd be pretty game changing. Lots of factors going against that though.

Yes, that's a great point.

Oh, there's a lot more stuff that can be done to guns:

1) improve aim: embed wind and angle sensors, possibly even battlefield intel (position, weather, land layout) to account for any drift that might impact the bullet. Also, auto-fire if a designated target is in the crosshair (train the AI on human faces, combine with the previous sensoring). I would not be surprised if this technology will be developed rather sooner than later with armed robots, and then made smaller until it fits in a gun or at least a rifle.

2) improve/rethink propulsion. Right now almost all guns operate by some form of bullet in a casing with explosive propellant (excluding the rare caseless guns and co2/pressured gas sports guns). Railguns are already a thing at "ship scale", it will be only a matter of time until it gets scaled down to hand-held guns.

3) improve projectiles. Right now bullets are dumb pieces of metal. Why not have active bullets (e.g. subminiature rockets) or bullets laced with poisons so that even a scrape kills in the end?

4) improve... guns themselves, as a concept - think laser guns a la Star Trek, highly focused microwave, sound or other energy.

In the end humanity will always improve ways to kill each other, and all the concepts are already there in sci-fi (and in the case of poison bullets, the Russians made it a reality with the Markov murder).

I think those are all really great ideas for new inventions, some of which might eventually replace firearms. But I still think firearms are basically topped out. Anything else done to them complicates them more than it improves them.

Wonderful analysis.

I don't get it. We bought a baby monitor in 2012 that was probably $80 (Philips brand I think). It was based off the DECT standard that was developed for wireless landline phones, so range was around 1200ft line-of-sight, in practice you could go two stories down in our apartment building to visit friends and it would stay connected just fine.

The baby unit was quite bulky and ran off wall power, but the parent unit was significantly smaller than a modern phone and had a loud speaker plus a really strong vibration mode. And if it lost the signal from the baby unit, it made a hell of a racket to let you know you were out of range. It also transmitted temperature. Can't say we ever had trouble with it running out of battery either.

I mean, I get the appeal of hacking just as much as the next guy. But IMO it makes no sense to hack on something like a baby monitor, where there is very little to be gained over off-the-shelf products, and where the consequences of a hardware or software failure can be that your baby is left crying unattended for a long time.

Go grab an ESP32 and some WS2813s to pimp your stroller instead, much more fun.

I gotta say, you guys must own some giant houses if the monitor is the only way to tell the baby is up and crying.

In my experience it’s not so much about hearing the full blown crying, it’s about hearing the fussing that indicates that crying is imminent.

> your baby is left crying unattended for a long time.

How is it possible? Are people leaving babies in far away corners of the house and walk outside with baby left alone? For me baby monitor is to inform me sooner when baby is starting to cry so I can comfort it faster and make it sleep again.

I have never ever been 100% dependent on baby monitors. I will always eventual hear when my baby is crying, I do it right now (I don't use baby monitors anymore because they were useless and loud and didn't have visual feedbacks).

If your partner isn't at home, and you're doing laundry in the basement while the baby sleeps on the 2nd floor, you definitely need it.

Also we let the babies sleep outside, it is a tradition around here [1]. Then you need a baby monitor.

And when our first kid got to the age where she slept fairly reliably (close to a year), we did go over to visit friends next door. The baby monitor was still in range there, basically twenty seconds to walk from door to door. But I'll concede that it's fairly unusual to have good friends living that close.

[1] https://www.treehugger.com/let-sleeping-babies-lie-chilly-te...

In our case we have a non verbal autistic son and a video baby monitor is really useful since it's not like he can call out for us or tell us if he has a problem. I am busy replacing his with a raspberry pi based ones because I've had enough trouble with Nest having too much control over the device and changing the functionality without warning.

You can also get a standalone monitor for $30.

"Lavish attention on me and entertain me."

― Maggie Simpson via the baby translator

https://simpsonswiki.com/wiki/Baby_translator

That being said, every parent knows the "he/she has never cried like this" moment where your blood freezes and you rush to the child's bed. Seems hard to add this to the model in a reliable manner.

Please fill me in as a non-parent : what was actually up when the baby cried like that?

There's a difference in many of the cries: Boredom, hunger, pain, cold, heat, exhaustion, loneliness, no reason at all, fear...and, once, a hair tourniquet.

A hair tourniquet is formed when a strand of hair lost in a onesie or sock wraps itself around a toe, penis, or finger and the appendage swells, cutting off its own circulation.

It's terrifying to blearily try to diagnose the issue and they just don't stop. Eventually we undid the swaddling and took off the sock and found the purple toe, removed the hair, and then there was a change to the screaming as skin pinch went away and the feeling returned to it, but that was good news.

Not the OP but you get tuned into this pretty quick going through the normal gamut of kid issues. The _worst_ cry I've heard our son do was when he had a double ear infection, it was immediately obvious from the shrill tone that he was in pain and needed to go the doctor. There is also a slightly less desperate one where he's stuck in something (usually leg/arm in the crib, not really stuck but can't yet reason out how to fix the problem).

Double. Oof. That’s no fun.

Very early on we switched from an audio monitor to a video monitor specifically so we could answer this question without panicked running across the house.

It’s really hard to generalize what an alarm cry could mean. In my experience they were never anything outwardly physical. Both of our kids had alarm cries that we associated with waking up from dreams, for example.

For us, our daughter got sick and vomited everywhere once. Of course, the baby has no idea what's happening, so it's scary as well as uncomfortable.

That was a cry I remember.

Our son at ~8 months had managed to get his leg in between the slats in the crib past his knee. I agree with OP -- this was instantly distinguishable from a normal "pick me up" cry.

It's so weird, with COVID time at home, it's almost like I've picked up on my 18 month old's language. All his little noises and cries absolutely have meaning, they're just not in a language I understand. I can't wait until he is able to use words though, it will make it a lot easier!

Maybe he thinks, “I wish this large baby would speak my language.”?

I have a 6 month old going to sleep in my arms... worst cry was probably yesterday with his biggest ever, smelliest poo. The joys of weining.

It's usually either outright pain (examples from our child: wasp sting, finger stuck between the valve handle spring of a garden hose, stomach pain, ...), nightmares or the feeling of being stuck somewhere (examples from our child: arm / leg stuck in the lattice of the baby bed, leg being caught in a shoe rack after climbing on it, ...).

Does anyone else just, you know, watch the baby or stay near enough to hear them when they go off? We didn’t have any clever stuff for our kids and it was tiring but ok (both full time jobs, no family to help us)

Cosleeping is what we did for our twins until they were about 1y old. They've turned out fine. I mean, what did pre-tech societies do?

Yep, I think sometimes working in tech can bite back along the “medium is the message” lines. In engineering sure tech is the answer. With kids it’s not always so.

I don't have a baby, but I might have the need for a baby monitor. A sick family member who needs help sometimes in the middle of the night. Training TensorFlow to hear her yell "Help!" isn't going to do the job. For now, she reaches over to her phone, presses the home button, and says "Hey Google, call $familymember". It works, unless she can't reach her phone for any reason.

I had looked at this article wondering to see if it could be a valid replacement, but instead it's a very hopeful hobby project that I wouldn't trust anyone's safety to. It's definitely cool, but I think it's more suited for identifying a barking dog or a meowing kitten than a Human In Need, baby or otherwise.

Why don't you put a shortcut the home screen to call you directly?

It's a good idea. The main problem however is touch. That's why we went to voice activation. If she could unlock it, she could call and wouldn't need voice activation.

An old Android tablet or phone with the app Dormi does great, too. The best part is that it doesn't make any noise until there's sustained noise detected in the baby room. It also alerts if it loses connectivity and can be snoozed at 10 minute intervals.

Does the battery hold up to the level of abuse?

One of the extremely important things with any DIY project related to your baby is KEEP THE WIRES SECURED AND AWAY FROM THE CRIB.

Or just don't.

If you're not certain about your setup, yes, avoid doing anything. I have a Bluetooth speaker in the crib to play white noise for our baby. It's a ruggedized speaker secured to the crib with 150-pound zip ties, and we charge it when the baby's not in it and remove the charging cord entirely while the baby's in there.

As a father with a toddler at home, we use the baby monitor as a dumb radio in case we can't hear his cries through the walls. (We usually can, he's loud when he wants to be heard.) I wouldn't want any other features because I'm the parent, not the dumb radio.

Also, the line about notifying on your phone if your baby is crying really, really struck me the wrong way. Like, you need your phone to tell you if your own flesh and blood is in distress? Come on man, be a better parent than that.

Came here to post basically the same comment as everyone else, so I’ll save my words.

My child was born in June and we have been using a WYze camera with RTSP enabled + a kindle tablet with TinyCam installed and it has worked flawlessly. Highly recommend for a budget setup.

I recommend a simple audio one. Video is unnecessary.

It’s easier to hand off to babysitters, “just works”, and honestly your kids not gonna die because you dont have a 24/7 video feed of them. Just make sure the bed they’re in is the right kind (nothing to get tangled or entrapped in).

This camera stuff scares me

Actually this would be useful for something I've been wanting to set up. I want a white noise generator in my older kid's room to fade in and out when the baby is crying, since the baby's crying keeps the other kid awake.

I see a lot of potential of this for other fields. For example coughing detection in pig husbandry. So while I am a little wary about the baby monitoring, I am very grateful for the project and its documentation. Thank you!

One could use the free software Subtitle Edit [1] for the labeling of the audio. Output would be in SRT format, but a conversion to the required JSON should be trivial.

[1] https://www.nikse.dk/subtitleedit/

This is clearly an awesome project... for those with slightly less time and/or don't want to label their baby's cries, Motorola produce a range of WiFi connected baby monitors that have a lot (but admittedly not all) the required features.

This is exactly the article I come to Hacker News for.

I recently started thinking of learning more about ML and here it is, exactly what I needed (I have a baby), or maybe not what I need, but what I can use for fun (I mostly hear when my baby is crying).

Awesome post. If you’re feeling too lazy to build this and are willing to throw money at the problem, check out Nanit. It’s an amazing baby cam that can detect sleeping and breathing. It works like a charm. Our family loves it.

Sorry, but some things simply shouldn't have technology applied to them at this level. If author is so worried about trying to "automate" away parenting duties, I think they should take a good hard look at themselves in the mirror and seriously reconsider their approach to parenting.

There's no amount of RaspberryPi or Tensorflow to justify reducing parenting time so you can do more RaspberryPi or Tensorflow-ing.

The tone isn't great, but I think it's in part like that because they're trying to pitch their project as automation. To me it seems more replacing/extending something (analog baby monitor), but that's not exactly automation.

It’s up to the parent to move the speaker as they move to different rooms, as they can’t play the sound on any other existing audio infrastructure

game changer if it could also smell poop!

I actually wondered about this. Could any of the various "VOC Sensors" that you can get from sparkfun/adafruit etc actually be used to detect the smell?

We have an Arlo baby monitor with a VOC sensor and I'd say the probability of detection is ~80% but the false alarm rate is ~50%

I can say from personal experience that they can detect farts, but only really stinky ones.

Ding Ding! All aboard the HN negativity train!

Seriously, I don't remember HN being this bad before. Is it that the pandemic has everyone in a perpetually bad mood? It would be interesting to do a sentiment analysis over the past couple of years to find out!

all these comments.

bah.

Sounds like you need 3 devices. One windows, one linux and one possibly off-the-shelf version that is UL listed.

They should each have cellular, radio and wifi connectivity, and battery backup.

Also:

https://amzn.com/B00NL5JPZC and maybe https://amzn.com/B07CRHWBSJ

Reply on news.ycombinator.com