Robot uses machine learning to harvest lettuce | Hacker News Reader

Robot uses machine learning to harvest lettuce

cam.ac.uk

81 pointshhs71 comments

I'm sure there are complexities here that I'm not seeing, but...neat rows, uniform textures, little noise, no external interference, physical feedback, low risk.

Relatively to say a self driving car that needs to deal with dogs running into the road in questionable weather at decent speed this seems comparatively easy.

That's not meant to be condescending - I know for a fact I can't program a lettuce harvester at all let alone a better one. Just seems like there is a weird difficulty differential at play here.

This feeling will only become more common as we start exhausting all possible applications of machine learning.

There are quite a few human jobs that deal with "neat rows, uniform textures, little noise, no external interference, physical feedback, low risk." Think: gift wrapping, cookie cutting, sewing, etc. As it currently stands, people still work these jobs in large quantities even though we have the technology to automate all of them out.

It is certainly comparatively easy to self driving cars. But that's the point: while we haven't solved level 5 autonomy yet, we do seem to be solving these lower level problems at or above the same skill level as trained professionals, and the amount of problems we solve with machine learning technology appears to be increasing at an exponential rate. It's logical to extrapolate from here that AI will be able to solve even harder problems in the future, including those our best researchers still haven't figured out.

A job at its core is applied problem solving. When AI services and physical robots can solve every problem in a better way than humans against every metric possible, there won't really be a need for humans in the workforce.

> When AI services and physical robots can solve every problem in a better way than humans against every metric possible, there won't really be a need for humans in the workforce.

That logic assumes humans want to share equitably the ownership of robots and their benefits.

I often see this pessimistic prediction, and I don't get it.

What benefit would someone have to not pass on cost savings to consumers if they had a super effective robot to do something? And regardless of their motives, how would they stop it from happening? Patents have limited lifetimes and and techniques developed will be quickly replicated/democratized.

Perhaps someone could corner the market for a while, but not indefinitely.

In the end, the result of automated production will be downward pressure on prices, with consumables trending towards $0.

With this in mind, it strikes me that the pessimistic predictions stem from the idea that there is a secret cabal of the "wealthy" who take great pleasure* in keeping their fellowman down. Today, by many measurements people living in wealthy countries have a standard of living so far superior to that of kings of old. Has the secret cabal been just lying in wait until now to strike?

* sure, some percentage of billionaires are probably sociopaths, but I'm sure most are not.

This is exactly the fear of many at the moment. A solution being talked is to tax robots (or automation to be more specific). Expect more research to come out from Academia over next few years. Even Bill Gates has been advocating its consideration:

https://qz.com/911968/bill-gates-the-robot-that-takes-your-j...

1) Even if the cost of products approaches 0, consumers will still need some kind of income to attain them. UBI could help in that regard, but it's still a far way off.

2) The transition period might not be that pleasant. A lot of people will be left without any means to provide for themselves while the rest of society struggles to adapt.

3) Removing the need for labour will be a major game changer to our society. A lot of people will be left with a feeling of having no purpose and nothing to do. Fostering a culture of creativity to fill people's waken hours sounds like a good thing, but I fear there will be a lot of social unrest if people don't have anything to do.

In the very limit, at full, 100% automation, this might hold true. But right up until that point, there will still be some jobs that need to be done by humans. And if we don't fundamentally change our understanding/social contract of what it means to be a Valuable (TM) human being and member of a society, there will be huge conflicts arising from this.

Think of all the anti-welfare rhetoric around people just being "too lazy to work." If this continues to be a socially-accepted way of thinking about people without jobs, then automating away significant portions of the required workforce will be very painful for the victims of the process.

Interestingly the three responses to my comment are quite similar, which makes me wonder that I misunderstood what the grandparent was writing.

> That logic assumes humans want to share equitably the ownership of robots and their benefits.

I read that as the robot ownders not wanting to share, but now I see that perhaps the grandparent was talking about society as a whole.

I suspect the difference here is one of context. I live in a country that has a fairly strong social safety net when compared to the popular conception of the safety net in the United States of America.

> What benefit would someone have to not pass on cost savings to consumers if they had a super effective robot to do something?

Literal monetary benefit.

We're seeing a new wave of low hanging fruit getting pruned, but complex problems still require complex solutions and no amount of gradient descents will change that any time soon.

I would think the lettuce classification is not the only hard part.

Having worked on fruit sorting machines where each fruit is rotated under a vision system and various measurements and classification are made, in a completely controlled lighting environment.... it's not trivial, especially getting it to the level of commercial viability. Making the robotic cutting mechanism / vision / control system would also be challenging.

The real killer is reliability, not so much in terms of percentage of produce spoiled but in terms of how often it gums itself up and needs manual intervention. And in a production environment, the numbers add up really quickly and you need a lot of nines.

A human doing the job takes 2 seconds per lettuce and can probably work a full 12-hour shift without intervention. Adding some leeway for pauses and breaks, call it 10 hours of uninterrupted production, that's 18,000 lettuces. If your failure rate is only 0.005% that's still one failure per robot per day.

exactly. In large fruit packing lines if the plant came to a stop, it would cost a lot of $s per minute of downtime (some lines involved 100s of people, incoming trucks, out going trucks etc that all get frozen). Some crops are very time sensitive and delays in the whole supply chain are problematic. Though the good thing with free form robots like that is you can buy a certain amount of redundancy. Some problems though are disastrous, like if the robot starts automatically destroying the lettuces for some reason (unforeseen mechanical failure which is undetected). I have watched this happen a number of times with the automated machines I worked on..... nothing quite like debugging problems with that kind of pressure :)

It does seem like that, doesn't it? And there's a lot of money and potential for social good in agriculture.

Makes you wonder why there aren't already loads of commercial solutions like this, especially since self-driving cars are right around the corner.

I guess it could be because (iirc) there are products that already fill the niche, like combines that can navigate along GPS waypoints. But I bet it is also extremely difficult to get the last couple of 9s that you need for commercially-viable accuracy and precision in an ML model, even with ideal conditions.

> combines that can navigate along GPS waypoints

The easy things to mechanize and automate have already been done.

> I bet it is also extremely difficult to get the last couple of 9s that you need for commercially-viable accuracy and precision in an ML model, even with ideal conditions.

With how much food never makes it to people, I bet 95% would be good enough for a lettuce picking robot. The problem is it's $50,000-$500,000, while the humans are $8 per hour.

> The problem is it's $50,000-$500,000, while the humans are $8 per hour.

It depends how active you can keep it.

£500k financed over say 10 years at 5% is about £5k/mo, a 25y+ human in the UK working 40h workweeks costs about £1600/mo (bare minimum, assumes no overheads or illnesses just the legal minimum + taxes/etc).

Another thing to consider is specialization and seasonality. If the robot is over-specialized in something seasonal, it'll sit idle most of the year. Laborers, on the other hand, move from farm to farm and crop to crop.

There are a lot of companies working on robotic farming; two that I'm familiar with from the ROS community are Iron Ox and Deepfield:

http://ironox.com/

https://www.deepfield-robotics.com/en/

Deepfield gave a great talk at ROSCon a few years ago about their data management platform for monitoring plants over time, see: https://vimeo.com/187696092#t=1025s

This sounds cool. I've been spending some spare time designing an outdoor solar pumped hydroponic waterflume system intended to connect to the top and bottom panes of a residential window, that uses robotic gates between looping sections for growing leafy greens suspended in the flowing water by floating planters that progress slowly from the inlet at the top to the outlet at the bottom of the window as they progress from seedling to full plant. The design can also be disguised as a 'garden water-feature' in a range of styles from '70s sci-fi to minature Roman aquaduct.

edit - am intending to publish an open source git repo of the thing, should I get any further than my current mass of sketches.

So, like an extremely slow Lettuce printer?

It depends. How fast are lettuce printers, generally?

What's your price range?

200,000 - 250,000 kg, of 6 month lettuce futures.

Very neat idea, I'd love to hear more about this.

How would you control the position of plants within the flume?

Personally, I've been pondering a solar-pumped hydroponic system which uses bell siphons under a big 'deer scarer' like off-balance tipping vessel to periodically flood-then-flush a number of sequential gravity-fed zones.

>How would you control the position of plants within the flume?

Turnstile gates that allow a plant to pass through with the flow of water into the next section. So the main power requirement is a pump to raise the water back to the top, the gates are very minor and the whole thing is achievable easily with a fairly modest solar panel and battery setup.

Cool, thanks.

I recently found cheap (and weak) solar pumps on aliexpress. They can only pump up about 3ft when lit, would prefer something with better performance, will probably try some others. Been playing with the idea of making a passive solar system for filling a water tower from a rain barrel on the ground. Might need to find a bit beefier one. But maybe you'd be interested in these - no battery so it only runs when the sun is out.

ex. https://www.aliexpress.com/item/32893336462.html

Thanks :) Sounds ideal for my purposes, I don't need more than three foot.

Look, more simple but labor intensive jobs being replaced by automation. Our society will definitely need to solve this problem sooner than later. Soon we won't have any jobs left that can be done without an education.

That is something I thing about frequently. On one hand, I think it's great to have self-sustaining homes/neighborhoods that require minimal manual repetitive work. Then I think that there are lots of people who know nothing but that. knowledge". If you lose the ability to adapt and learn (something we all probably do as we age), then you are left out.

This is why retraining programs are often inneffective and why i support ubi and andrew yang

There is another side of the coin - to make universal basic income work cost of foodstuff, healthcare, and construction (and thus housing) needs to be pushed extremely low. This kind of automation enables it.

There is no argument that automating these jobs will incur tremendous human suffering by taking away jobs. However, in the long term I don't want anyone to be 'harvesting lettuce'.

Or we could expand wage subsidies like the earned income tax credit. That has the advantage of not causing labour supply to drop precipitously and would cost much less than UBI so it could be implemented far sooner. And if the math on UBI comes anywhere near working out you passed the point of being able to afford wage subsidies long ago.

But harvesting lettuce can be fun - in your own garden on your own time of course. Not back destroying wage labour...

Yep. Logic next step is UBI. What of all the poor countries with millions of uneducated that need UBI but the country doesn't have a upper end to carry the UBI?

I don't see how that's gonna work :(

Globalization has been bringing up living conditions across the entire planet for 50+ years now. Presumably that trend would continue such that they could get the eduction they need, possibly through foreign aid or through sharing the bounties of the natural resources in those countries.

But you're right, it will be a hard problem to solve if we continue to separate wealth by country. At the end of the day we'd need a planetary economy to really deal with it.

It depends on the adoption curve. Right now farmers are having a hard time finding summer hands to harvest some types of produce. Fewer teenagers and adults want to work as farmhands, so this is a net benefit in this industry.

If we completely automate food production and house construction perhaps they'll become cheap enough that it doesn't matter

Food production is already plenty cheap in the developed world. In the 50s in the US middle class people still worried about the price of milk and butter and the working class did not have the option of eating out for every meal. Even Mexico, a middle income country, has an obesity problem rather than a malnutrition one.

In areas with high demand the constraint on building housing is zoning, not cost of construction. You can build in masonry up to eight stories high as long as you have elevators. With steel frame much higher obviously. Everywhere the cost of housing is substantially above construction plus land costs the reason is zoning.

> Food production is already plenty cheap in the developed world.

Even a lot of that is hugely inflated at the various steps to get to the consumer, in developing countries basic staples can be astonishingly cheap by western standards. Even semi-developed India you can have a decent diet for roughly $4USD a day: https://www.numbeo.com/food-prices/country_result.jsp?countr...

Definitely for far less. In one of the most expensive private universities in Pakistan, in the cafeteria you can eat three "home-cooked meals" for $4USD/day. I.e. on top of the ingredients you are paying for cooking/cleaning labor and space. You can eat for maybe half that at home.

This is a good thing: we are already significantly over the carrying capacity of the planet. A managed decline of population down to a couple of billion engineers, and robots for everything else, would mean a much better quality of life for the average human, and much better outcomes for the environment. Then, when we expand beyond Earth, we can grow the population again.

Technological developments have always driven the human species forward - all of this is nothing new. The difference now is the extremely high populations (enabled by technologies invented elsewhere in the world) in some countries with excessive unemployment levels.

Genetic engineering and molecular biology will also enable us to identify the genes which contribute the most to useful traits, and to expand access to those genes to a greater portion of humanity

"A managed decline of population"

So. What do you propose?

Global access to education, contraception, and abortion for women, for starters. Redirect foreign aid away from food aid and towards this purpose. If you feel that your family cannot support a child, it should be within your power to control that. Many women simply do not have that choice.

How did we do it so far? It wasn't long ago that most of us didn't have an education.

This stuff has been said for years, but it's almost 2020, when do you consider soon? If anything, more jobs will be created as we can mass produce goods at home without going through China et al. In fact in my country, they can't find anyone to pick the fruit even for very decent wages (even if the picker sucks).

On the issue of skilled work, if i was a male with no skills or family, I'd be becoming a truck driver right now due to all the doom and gloom which already has led to shortages. The truth is, a lot of these automation problems are very hard to solve.

There are lots of experimental picking robots. Apples, especially. None are yet commercial technology, but several prototyepe apple-picking robots are getting reasonably close. The machinery is still too slow and too fragile for routine use.

Machine learning for weeding is in production. "Recognize weed, kill weed" targeted weeding is available from John Deere and others now. Picking, not yet.

Not sure if ML is the answer given there are much more efficient machines that already do this at scale [0]

[0] https://www.youtube.com/watch?v=VjAXX-PCwjI

Nice video. The article somewhat addresses this: "These previous approaches include using a belt‐driven band saw‐type mechanisms or water jet cutting. These approaches have limitations, most notably that the outer leaves of the lettuce can be easily damaged when harvesting and there is a lack of reliability in stem cutting height and quality."

Seems like the solution discussed in the article does not really improve upon this significantly: "As the project stands, the damage rate, caused by cutting the lettuce stem too short, is too high for supermarket standards, although the harvested vegetables were perfectly edible. The most recent sample size of 69 lettuces was enough to confirm this as the next problem to address (hundreds of lettuces had been harvested over previous iterations). Future versions of Vegebot will need to address and improve the damage rate, perhaps with visual feedback from the harvested lettuces dynamically adjusting the force threshold at which the cut is made. In parallel, the end effector needs to be made lighter to achieve a human‐level cycle time, possibly by manufacturing with carbon fiber, or by using an alternative, stronger cartesian arm design."

A similar machine exists for white asparagus. I believe it was custom made by the people who own the family farm.

The link below doesn't give much info. I think it uses electrical resistance in the soil to determine if the asparagus spear is ready to be picked.

https://www.futurefarming.com/Machinery/Articles/2018/4/Spar...

I think this works extremely well in custom applications, small family farms etc. - here's another beautiful example from a small Japanese cucumber farm

https://cloud.google.com/blog/products/gcp/how-a-japanese-cu...

>> Although the prototype is nowhere near as fast or efficient as a human worker, it demonstrates how the use of robotics in agriculture might be expanded, even for crops like iceberg lettuce which are particularly challenging to harvest mechanically.

I'll be the sour one again, but we've had robots who are nowhere near as fast or efficient as humans for at least 60 years now and they keep getting better. And they're still nowhere near as fast or efficient as humans.

Robots keep on expanding in the range of areas where they are as fast and efficient as humans, or better. Industrial automation is why the value of US manufacturing keeps growing while employment has dropped or held steady.

Google has been collaborating with Japanese farmers in this field as early as 2016[1], and put up a case study to demonstrate their GCP AI offering.

[1] https://cloud.google.com/blog/products/gcp/how-a-japanese-cu...

Collaborating seems like a strong word to use here. Google wrote about a farmer using Tensorflow to identify good cucumbers.

somewhat related in 2016

https://futurism.com/world-first-robot-run-farm-harvest-3000...

It interesting to think what this could do for farming. Imagine one of these to a field continuously driving around. First weeding, killing pests, targeted watering, then when you get to harvesting you can be so much more targeted. If you have an order for 100 lettuce of x weight, the robot can remember the 100 that most closely match that weight, and go an harvest that 100 only, replanting a seed in the space. Or you could plant at double density and harvest every other one young.

It could measure nutrients and moisture in every square metre of soil, recording what grows best where in each patch of a field.

The amount of precision it could bring is amazing, potentially organic, and at least much improved efficiencies and carbon emissions.

On a related note—YC funded a few years ago a cool-looking startup working on a robotic greenhouse, Iron Ox [0].

[0] http://ironox.com/

Not to be flippant but isn't it machine learning using the robot, not the other way around?

Not really, the robot uses it in the sense that the machine learning algorithm will detect cabbages and tell if it's ready, and that information will be used by the robot to perform harvesting.

Of course, you can have end-to-end control of the robot via machine learning (they seem, for example, to use another camera for blade control), and at this stage you could be right but in the robotics community we always say that a robot uses machine learning, not the other way around.

If you think of machine learning as the mind and the robot as the body...does the body use the mind or does the mind use the body?

Why does it have to be a one-way use?

It was more of a comment on how we use language about a machine and its intelligence compared to how we use language about a human and a human's intelligence. To be clear, my comment was not taking a position on whether the dichotomy of mind and body, machine and algorithm actually exists or is meaningful, but of perhaps a bias that we have in how we describes machine that exhibit some intelligence.

I think it is more common to speak of our minds as controlling our bodies than vice versa, but we don't seem to naturally extend that to machines with intelligence.

Even if that was true, the « mind » of most robots consist of mamy algorithms, some of which are not based on machine learning i.e. SLAM and path planning.

I think of a robot as a complete system which was created by humans to do some task. This includes actuators, software, and sensors.

With this in mind, the robot uses a machine learning system to help perform the task of picking lettuce.

obligatory: my cabbages!!!

But no this is an extremely cool application of machine learning. It's interesting that many of the low-paying professions are actually extremely difficult to automate -- a robot that can harvest lettuce is much scarcer than a person who can harvest lettuce.

Why do you think a robot capable of harvesting lettuce is more valuable than a person?

You are confusing monetary value with intrinsic value -- I have edited my post to be more clear.

The robots could probably work almost 24/7

This actually has a chance to make lettuce safer by cutting down on how much lettuce is handled by humans.

https://www.cnn.com/2018/05/02/health/e-coli-lettuce-explain...

It is does that, this could be a case of machine learning literally saving human lives.

Contamination of produce has been due to using contaminated water for irrigation or washing, or infested storage. It has nothing to do with there being a human picker or not. The article you cite essentially points this out.

Reply on news.ycombinator.com