Hacker News Reader

Comment by jc4p | Hacker News Reader

I found your blog post (the learn more button) but I'd love more info on this if it's available somewhere.

How does it handle differentiating different types of bread, which have differing carbs?

How does it handle a thick layer of butter or another fat put on the sandwich in the Avocado Toast example, which would presumably be below the visible avocado?

A long time ago my friends and I offered a manual version of this as a service via sending pics / emails to us and us then manually going through and guessing. It worked well enough, so I have high hopes for a ML version!

My biggest pain point doing it manually came from pics of things like pasta where I couldn't really guess how much oil was in the sauce.

You can definitely get far with just estimating the macronutrients from a photo, and the absolute accuracy matters less than consistency in measurements over time.

View on HN

> How does it handle differentiating different types of bread, which have differing carbs?

We don’t nail everything yet but we allow users to refine the predictions. So in your example we might predict bread and let the user pick the type.

> How does it handle a thick layer of butter or another fat put on the sandwich in the Avocado Toast example, which would presumably be below the visible avocado?

We don’t predict portion sizes yet. At the moment we give a sane default and ask the users to adjust it. The next time you eat the dish we bring back the past meal so you don't have to specify the details again. We’re hoping to start predicting some of those details once we get enough data from our users.

Thank you for trying it out and the feedback.

> We don’t predict portion sizes yet. At the moment we give a sane default and ask the users to adjust it.

This is a serious problem. Research suggests that one of the main causes of obesity in children is lack of ability to identify portion sizes or understand how much to eat.

Obviously there is a market of people who understand this well and want to track what they eat, but you are very likely going to be misleading a very significant amount of your userbase into making worse decisions for themselves.

Predicting portion size is something we’re actively looking into. One of the reasons for getting Bitesnap out to a larger audience now is to be able to collect more training data for doing this. We’re also experimenting with allowing users to specify portion sizes in more natural units -- for example by comparing a serving to the size of their fist -- and we’ll automatically convert these to conventional units. Finally, we’re also building more tutorials and help content into the app to educate people on better estimating portion size (among other things).

so you want early users to be a guinea pig for potential future users, and receive inaccurate counts? i dont think there will be any future users if accuracy is an issue in the beginning.

this is one of those things that i believe you should have right from the beginning.

You might look into a custom cutting board or plate/container that has a fiducial on it so you can measure each item. That's also another revenue stream.

I spent a lot of time looking for a small bluetooth food scale that I could put under my plate but wasn't able to find anything. If I ever find some free time I might try to make one.

Would be cool if I could pull one out from my pocket, stick it under my plate, get a measurement then subtract whatever is left after I'm done.

With regard to portion sizes, you could have users put a "standard" object in the photo, such as a dollar bill, credit card, teaspoon, or quarter, iPhone, etc. into the photo as reference sizes. Eggs are also reasonably consistent, so in photos with eggs, that could work also. Love the idea (I had it myself a long time ago, but too many things to do and not enough time) and hope this takes off!

Reply on news.ycombinator.com