How a Japanese cucumber farmer is using deep learning and TensorFlow
cloud.google.com
cloud.google.com
Fruit and vegetable sorting using computer vision is routine. That's why commercial produce is so consistent - it's been pre-sorted. The existing technologies are so fast that it's routine to inspect and sort peas, and possible to sort grains of rice.
Most of the cost and complexity in the commercial machines is in the material handling. There are conveyors and devices to get the individual items lined up. Cleaning and washing stations are usually part of the line. Commercial machines have to be built sturdily so they survive continuous use, and have to be cleanable with high-pressure soapy water. The little unit that does machine vision is a minor part of the processing line.
AI techniques are routinely used. Support vector machines are popular. One Indian system uses fuzzy logic. ANNs have been tried but are not yet popular.
The systems generally measure user-defined features - length, width, curvature, color, flaws - which then go into a simple grading algorithm. They're not trying to match human decisions.
[1] https://www.youtube.com/watch?v=5DCpOx-q_yo [2] http://www.aweta.nl/en/produce/cucumber.html
I think OP in a way demonstrates something characteristic of Japanese agriculture:
"There are also some automatic sorters on the market, but they have limitations in terms of performance and cost, and small farms don't tend to use them."
Japanese agriculture is notoriously unproductive compared to other countries' yields and inputs, and a major reason is the lack of scale and mom-and-pop farms. If this farm were bigger or needed to save labor costs more, it could afford the already existing solutions and would sort cucumbers into more standard grades than their own ad hoc system. But they're not, so instead they do it by hand.
As an aside, my dad swore the produce was notedly better than what we have locally when he was travelling there. He on multiple occasions also discussed their dedication to quality.
Maybe they're onto something given this related discussion was on HN earlier this year...
In fact, even given their weird terrain features, they're pretty close to the top in cerial yield for example:
http://data.worldbank.org/indicator/AG.YLD.CREL.KG?end=2014&...
In some cases the shovel is better, but you can't effectively compare the two of them.
Too late: the thread you are participating in has already made the comparison from the very beginning. It's like an HN article titled "Local Man uses nuclear-heated shovel to clear snow off driveway" where the first comment is about how JCB and CAT offer superior options...
I'd argue that industrial application is much, much more advanced that current automotive AI application (because plants don't have tho choose the absolutely worst and cheapest viable solution for cost reasons as it is the case in automotive applications).
Do you have any information regarding what kind of accuracy such SVM have in practice? What do they do with the false positives and false negatives, do they reinsert back in the line ?
TensorFlow is, in no way shape or form, ready to be used by anyone but very technical people with a solid knowledge of both software engineering and machine learning. In fact, it's probably unfair to single out TensorFlow because every deep learning toolkit I have seen is still in the same "extremely early adopter" phase. Yet, this article makes it seem like a line of business user (farmer) with some background in technology (former embedded systems engineer) can use deep learning to transform his/her business -when, in reality, much expert help is needed to make it work. In other words, it would have been a lot more plausible/genuine if the article read like:
1. A progressive farmer got in touch with Google.
2. Google dispatched their solutions architect to work with them.
3. Hey, a cool, early prototype is working!
I am not trying to be a hater. Deep learning has huge potential, and Google, among others, is doing a lot to make it accessible. That said, this type of over-promising ahead of market reality is what gives cutting edge technology a bad name.
"2. Google dispatched their solutions architect to work with them."
That's not what happened at all. He built the system entirely himself before getting in contact with us.
I think that while very complex machine learning tasks do require specific knowledge, creating a neural network that can do real work is within the abilities a single individual.
Good to know. While what I said was meant to be hypothetical, I am happy to stand corrected.
>creating a neural network that can do real work is within the abilities a single individual.
This really depends on said "individual." My biggest issue was the tacit conflation between two themes, one valid and another reeking of "feel good" marketing:
1. That a reasonably technical person can use neutral network to do useful things, no small part thanks to something like TensorFlow (valid)
2. That such individuals are prevalent in agriculture (???)
I am all for hero-making: it's at the heart of marketing and customer advocacy. However, as someone who has been doing technical marketing for awhile, I just find this story exceptional in both senses of the word (as others commented, perhaps I turn out to be wrong!)
Advocacy is partly about inspiring others and I think this does that while presenting an accurate representation of what the farmer was able to accomplish.
Actually, this type of marketing is _exactly_ what is needed to get people interested in applications for this technology.
This particular farmer, perhaps, is an unusual case in that they went straight to machine learning with sexy new tools. However, machine vision has been used for applications in agriculture even as long as 20 years ago (I know this personally :-) )
It isn't at all surprising that a farmer decided to go "cutting edge" with a little help from Google. In fact the theory for the "diffusion of innovation" was based largely on observations of FARMERS!!(https://en.wikipedia.org/wiki/Diffusion_of_innovations). What I am saying is that although farmers seem to have an undeserved reputation for being low-tech, some are _very_ adept at technology especially when it has the potential to give them an advantage in the market.
Just the other day on HN: people use excel because they don't have the elite mental reasoning ability that programmers have. Programmers are so awesome, it's cute that factory foreman use excel when they should be using my stellar programming skills for some hand wavy reason like "it's hard to tell what you did and reproduce it."
OK. Point taken.
Now here the sentiment shifts to the other extreme. People shouldn't go off and try to use cutting edge technology without the skilled supervision of an expert. Really, I would take this story better if Google were involved because they have a monopoly on the world's brain power.
---
I get that the common thread in both of them is that a programmer should be involved, and I get that I was overtly sarcastic.
But can't we just judge based on merit? Why not critique his approach, why not suggest what you would have done differently, or something to that effect.
Why do we immediately turn it into smarter than everyone programmer vs the idiot common folk.
Similar to programming. Programming languages are difficult to get and learn to use. See e.g. my test here on how difficult it is for nontechnical peopel to even install python: https://news.ycombinator.com/item?id=11453086 Whereas Excel is widely available and taught in schools.
I totally agree. Having been taking classes and read ML/CNN articles online, this Google Cloud blog post made this project seem to be significantly easier than the actual effort that went into it. In particular, the blog simply covered the secret sauce to make it work in one line:
> Makoto used the sample TensorFlow code Deep MNIST for Experts with > minor modifications to the convolution, pooling and last layers, > changing the network design to adapt to the pixel format of cucumber > images and the number of cucumber classes.
I doubt this is a one man operation in just a few months of time, considering the computing power that's required to accomplish such feat (which it was said to take 2-3 days for each complete training set). Let alone the fact that they admit 7000 cucumber images are NOT enough to train the model to achieve the typical 95% accuracy that the original MNIST model it was based off.
Although promising, this blog post reads more like a PR campaign for GCP's ML offering (with the custom hardware devices to do the compute, perhaps TPUs?)
If you look at: https://github.com/tensorflow/tensorflow/blob/master/tensorf...
you could actually manage it just by changing some of the image size and classes constants. In earlier versions of that code, you'd have had to manually change a few constants that weren't extracted well. This is approximately easy, once you know your way around the python side of TensorFlow. (Which itself takes some work, but it's not rocket science.)
A better approach would probably add a third convolutional layer to continue to shrink the 80x80 input down to something easier to handle in the fully connected layer.
What's surprised me is actually how slow it is, knowing nothing about their model. My guess is they could speed it up by adding that extra convolutional layer to reduce the amount of work done in the final FC layers. The fact that they're that slow suggests the opposite of what you're noting -- I doubt they had serious assistance on the machine learning side. I'm more impressed by the amount of time it must have taken to get the physical parts working and to collect 7,000 training images!
Training MNIST itself is trivial. You can get > 99% accuracy in only a few minutes of training using a single GPU. This model should be at most 10-20x slower than MNIST (but, again, that FC layer would scale badly).
If you look at the stackoverflow tags for tensorflow, you'll see a lot of very similar problems being solved. ("I want to reuse model X on problem Y, where such and such is different").
TPUs are awesome, but you don't need them for this problem. You could run the inference at 1 cucumber per second on a laptop. And you could train it on a Titan X in a matter of hours at most.
(Source: I was and am still slightly on the Google Brain team, as a non-machine-learning person.)
They didn't approve my request for alpha access though, so no real insight on my end.
I remember when the deep dream thing came out, there was a huge interest by people wanting to experiment with it on their own, or just use the existing code on their own computer. But installing these libraries is very difficult or impossible, particularly on Windows. People were trying to get it to run on virtual machines, which adds complexity and slows it down, and can't take advantage of GPUs which is necessary. They don't work on non-nvidia GPUs, etc.
The best example I found was Brain Simulator (http://www.goodai.com/#!brain-simulator/c81c). Which ran on Windows, had a GUI, and was intended to eventually be used in the game Space Engineers so nontechnical users could experiment with AI algorithms and build robots for the game.
Do you have any suggestions for the API?
If you wrote some simple code that would work if there was a functioning learning module, what would it look like (if you feel like contributing to this)?
Change "deep learning" to "expert systems" and "TensorFlow" to "decision trees" and that article could have been written in the 1980s: https://www.youtube.com/watch?v=6xpcES-ueKw
Why do you say that?
(Also how do you know or why would you expect that it's categorizing by shape? Neural networks are infamous for finding "irrelevant" features in training data).
Of course it's reductionist. Generalizing an abstraction across many things that superficially look different is what programmers do. Reducing problem complexity to make something easier to automate is what programming is for.
Saying cucumber grading is like letter recognition is a true generalization that is also useful. Saying that they're both "adding a bunch of numbers" is a true generalization that is less useful. Programming, and (way more) broadly, science and engineering itself, is about finding useful generalizations.
So most farmers won't be tweaking TensorFlow any time soon even once the product becomes mature - well, they might wind-up using a high quality UI for customizing said existing solution.
Also, the guy is getting 70% accuracy. I doubt google helped him :p
That's interesting. Is Google going into the consulting business? They might. That's much of what IBM and HP do now. If you're selling a new, complex B2B tool, you almost have to provide consulting services.
Is it? From what I've heard farming is pretty high tech already.
If one were to leverage the power of a supercomputer trained with tens of thousands more observations - all meticulously hand categorized - it would probably get better.
Truly, we are only limited by our imaginations.
It's not like conveying knowledge in the sense of information transfer (where by definition a student can't know more than their teacher -- without external help).
It's like teaching a skill.
In human terms, a person can get to play better piano than their piano teacher, better poker than the person who taught them poker etc.
As the "training set" she just gives the basic skills for the recognition. The algorithm could discover further patterns she can't see beyond what she knows (e.g. she knows just that greener ones are usually better, but the algorithm notices that greener and taller are usually also better), do the calculations much faster and consider many patterns while the woman does a more shallow examination, etc.
Alternatively, you could track each cucumber with an Independent ID to see when it is sold, relative to other cucumbers - on the assumption that people are selecting the best from any given batch? But then you'd need to control for things like the position of the cucumbers in the display. Now you're in pretty deep into someone else's business, because you're a wholesaler, and the data will still be insanely noisey.
Merely having an experienced human score images for them can be enough for the machine to self-discover new criteria based on the pictures and their scores.
> "In Japan, each farm has its own classification standard and there's no industry standard."
Right now the only question is "how closely can the machine approximate human sorting."
The machine sucks at it right now.
However, as soon as they described the actual solution I felt somewhat let down.
1. Reducing the input to 80x80 images is likely unnecessary; it removes some of the most important features (number of thorns, blemishes) from contention. If the issue is computational cost (and therefore training time) you might consider using cloud GPUs. However, a network with multiple "towers" (like Google's Inception model) could conceivably allow for evaluating the sorts of aspects that require higher resolution relatively cheaply (one-or-two convolutional layers with pooling and then a fully connected layer) along one pathway, while a different tower (with more layers or more units per layer) processes the resized images.
2. If the network used was just a modification of the MNIST one, then it seems like it ignores the ordinal nature of the problem. If cucumber quality can be ordered, why not use a regression model instead (or maybe regression along multiple criteria, like straightness, bumpiness, thickness, etc.) instead of treating each class as if they were completely separate?
3. Why not release the dataset? I have GPUs, the necessary background, and a few hours to try out some models. I also have a heretofore unrecognized yen to use my models to generate images of the ideal Japanese cucumber, so I'd be willing to do it for free.
I personally think it's exciting to see average-person real world use cases for these new deep learning tools. I wasn't going in expecting a fully-fleshed out system. It's just one guy helping out his parents with a summer project.
E.g.:
https://www.sandisk.com/oem-design/industrial/industrial-car...
http://www.cactus-tech.com/en/products/industrial-grade/sd-c...
The electronic/mechanical engineering of the sorter is actually much more demanding than training tensorflow on a custom data set - which is a fairly trivial task.
Anyway, cool work!
But what seems different with the AI "revolution" is that we may be seeing a hockey-stick rate to these changes that simply has no historical precedent. In which the "progress" (read: forced obsolescence) that might hitherto have happened over the course of 1-2 generations may soon be happening in the space of 5-10 years, for large categories of job descriptions, planet-wide.
So guess it's not that scary.
Unless you're the one whose job is threatened -- or as is apparently the case of this "success story", eliminated already. Do you really think all these displaced cucumber sorters can find work as Machine Learning engineers at some hot startup?
Or in the near term, any kind of job at all, without significant difficulty?
Cars took at least a generation to become mainstream. This time around it seems technology will destroy jobs almost at the flick of a switch.
Unless you're arguing that the adoption rate will not be significantly quick, which I think most would disagree with.
> that might hitherto have happened over the course of 1-2 generations may soon be happening in the space of 5-10 years
It's been happening in that shorter timespan for a while now, so really, what have those people who have already lost their jobs done?
1) Either technology is truly revolutionary, which means that it has a revolutionary change on society by producing an entirely different dimension of techniques and ideas that make everything in the past irrelevant.
or
2) The technology is incremental and will only make tasks more efficient, and only has a marginal change on society by producing newer linear techniques.
If it's (1) then in the long term there is definitely a huge benefit, but in the short term there's bound to be a great social upheaval. Ones that change the shape of governments, and throw society in chaos. Just look at the industrial revolution, or agriculture.If it's (2) then really it's merely a gradual process that almost goes unnoticed, save for in retrospect. Which makes it very straightforward to change and redirect people through education.
So pick either case. You can't have both revolutionary technology, and gradual social change. Either the tech is truly revolutionary which in that case will result in social revolution, or it's incremental and will result in merely incremental changes.
And besides, no one was arguing that the technology was "truly revolutionary", so I'm not sure what point you're making?
I personally don't think it's incremental and that we're at a point where we just had the idea of collecting seeds and planting them in the ground. The resulting aftermath of that simple idea produced civilization. The resulting aftermath of computation is yet to be seen.
But the parent wasn't actually suggesting that progress be stopped, merely observing that it was going to cause social problems.
And not everybody would need to have a horse, the same way not everybody needs to drive (and most don't) in cities designed for humans.
If the solar installer can't get to the customer's house because he's traveling by horse...
Or the biofuel engineer spends an extra 2 hours a day commuting on his horse, rather than developing new technologies in his lab...
Then it's not better for the environment for us to all move to horse and cart.
Not really. I would the opposite is true. Most people don't like change and appreciate stability as they get older. You cannot eliminate people's jobs, tell them to adapt and expect them to take that. And when a lot of people use their jobs during a short period of time, guess what? Extremist government/ideology will emerge, and we are already seeing some signs of it.
Would that just meant they have to be cut off before you can eat it?
Because I was not a Cucumber ... "
-Song of those Tensorflow'd out of their jobs.
[Edit: Changed Cucumber sorters to cucumbers to clarify the sarcasm]
But the tone of the article is ridiculous. For any reasonable definition of "it works", the system doesn't work yet.
No one's saying it's a great job. But the point for many of these people is that it's a job. Which they won't easily be able to find a replacement for.
Let me guess: shallowly and for marketing purposes...
I won't say this kind of innovation is bad, per se, or should be stopped. But the blitheness with which the article sidesteps the collateral damage that will necessarily come of this brave new era of efficiency and progress -- as if there was nothing more to this story than an unemployed engineer helping out Mom and Pop -- really is quite breathtaking.
"Democratized unemployment, and for many older blue-collar workers in years to come, a nearly inescapable sense of hopelessness and despair" is what we might want to call it.
Regardless of the non-relevant nature of the context, the HN crowd (the target audience) is well aware of the side effects of technological progress.
Similarly, we don't expect every blog post by security researchers highlighting the latest software exploit to include blurbs about the societal implications of vulnerabilities on privacy. There's a time and place for that.