PlantNet – App that helps identify plants from pictures | Hacker News Reader

PlantNet – App that helps identify plants from pictures

plantnet.org

394 pointsshrikant71 comments

I've tried Seek by iNaturalist [1] to identify some plants in real time (the app can also identify fungi and wildlife). It's been a bit hit or miss sometimes. I like that it doesn't require any registration and doesn't collect any data. I'd like to know how this compares (at least for plants).

[1]: https://www.inaturalist.org/pages/seek_app

I haven't tried Seek, but I've had better luck with animals than with plants on iNaturalist, personally. It's useful to algorithmically ID the plant family.

Neat :) I just had a little play with that in our garden it got things like himalayan clematis, forget-me-nots, dafodils and chives right. Will have to try on mushrooms etc!

Sometimes when it didn't seem to give a high accurate match it says 'dicots' for the family.

It said hemp family for my hops, I guess it's hard to get an exact match of hops, unless they're flowering I guess. But impressed it got that, it seems a hard problem!

Edit: it did get hops eventually after trying a different angle :)

I heard about this app on the radio yesterday and I literally just got back inside from using it to identify a plant outside my house. It worked ok. when I went to install it, there were a number of other apps that came up in the app store and I downloaded picturethis which identified the plant instantly whereas inaturalist gave me a list of possible plants to pick from. picturethis has some in subscription stuff going on that I didn't look into tough.

Myco-nerd here: absolutely never rely on the image recognition apps for fungi ID.

There are fungi in entire different genera that are VERY morphologically similar, and the apps are just not there yet — likely won't be for a few dozen years.

An app can make a lethal mistake much easier than a human.

I’ve used Google Lens as “if it says unsafe, definitely unsafe”. I would not rely on it for “oh yeah, I can eat this” :).

I doubt your “few dozen years” though. Humans are only so good at it themselves. Computing has improved a lot since 1984 (3 dozen years ago), and so I’d wager that by 2050 we can be better than human at “Eat or not?” for fungi. Up for a longbets.org wager? :)

I mean the thing about fungi is that the tops can look the same and you just need to do a spore print to positivity distinguish one from another. There may be some mushrooms which are simply impossible to tell apart by outward appearance. In one of the fast.ai lectures Jeremy shows how to distinguish different breeds of cats. Then he shows how to look at the confusion matrix, and he found one pair of breeds the network really struggled with. It turns out they look really similar to him too, and when he researched further he found they’re simply hard to tell apart. Perhaps with an enormous data set there might be small differences a network could detect, but the confidence might still be low.

And given that mushrooms can kill you, it may simply never be advisable to rely on any photo based identification.

I don’t consider it against the rules of the bet to allow multiple pictures, including the underside and perhaps even “here, smush the mushroom on a piece of paper and take a picture of that”. My question is can a vision-based AI thing outperform humans within another thirty years, not if it can do it via a mechanism that isn’t discriminating.

For all the myco folks here: Do you have a sense of whether or not the multiple hours mentioned is “required” or “just” makes it easier to get a strong signal? (That is, how much is the signal boost due to our inability to see well as humans)

[1] https://en.m.wikipedia.org/wiki/Spore_print

We foragers and amateur mycologists use smell, touch (slimy, dry, etc,) sometimes taste (bitter, acrid,...), habitat (on wood, ground, type of wood, is there a bulb below ground or a root-like structure, time of year, spore prints, sometimes color change due to drops of chemicals (especially on boletes), sometimes even microscopes to view spores, and more.

all of these variables could of course be coded for a good classification algorithm.

just saying, it's often more than simply visual.

It's mostly visual. I would add location and latest weather to any software trying to recognize the mushroom. The mushrooms that people commonly forage for are not that numerous, so the algorithm needs to know about a dozen or two varieties. Chanterelles are easy to tell from images. You can probably have an AI chanterelle identifier coded right now and for most edible mushrooms very soon if not now.

Other sensors combined with photo would likely be the solution and the results might not be instant for some samples.

Looks great. The app doesn't appear to be open source, just the API docs: https://github.com/plantnet Given the public funding of the project, wouldn't it make sense for the code to be open to welcome contributions/improvements from the community/users?

iNaturalist is publicly funded, a wide collaboration, and IIRC mainly FOSS and Open Data. They're using machine learning and knowledge of users to identify flora and fauna; I've found it excellent at doing automatic ID at the genus level (but not really species).

https://www.inaturalist.org/pages/developers

Depending on what you are interested in (and where) there will be plenty of enthusiastic users willing to identify your observations. In my experience, birds will always get identified, while insects and plants will only sometimes register interest from others. I think it's because bird watching is a popular hobby, while entomologists tend to be a rarer breed.

There are also at least 2 orders of magnitude more species of insects on Earth than of birds (>1 million compared with ~10,000). An amateur birder can know a significant fraction of all birds, and all of them commonly found in their location, while no entomologist is anywhere close to knowing all insects.

Absolutely, a good point. I would think for the particularly avid birder that you could probably memorize all of them. We certainly have larger vocabularies than that!

Cool. It is a pity that Seek contains 5 trackers and is not available on F-Droid, though.

https://reports.exodus-privacy.eu.org/en/reports/org.inatura...

PlantNet latest only has Google Firebase Analytics as its single tracker.

Plantnet makes requests to gravatar too (a tracker in my book)

Yes, quite right.. in mine too.

The fundation asking for donations is claiming it will be free and open-source

> Résultats > Des logiciels gratuits et sous licence Open Source.

[French only] https://www.agropolis-fondation.fr/Pl-ntNet

Google Lens does this. But not only for plants. And it is quite good. I could identify most of the plants in the park. Cool to know there is some edible stuff in the park like ramson

I was going to say the same thing, I would like to see a comparison of the two

Ramson smells like garlic, anything herbaceous that smells like garlic when cut fresh is Allium. Is a pretty unmistakable genus.

Something that does not smell exactly like garlic can be Allium also, like Onions.

Convallaria has runners also so it forms thick mat roots. Leaves stand in a different angle. In Bear's garlic each plant is individual.

Thanks, I meant of Google Lens. I am going to have to try ramsom though

Start with the flowers to avoid any risk. Are edible also. Then you will recognize easily the same flavour in leaves.

Were those ramsons flowering? Lily of the valley leaves could easily trick Google Lens, they would be more likely to find in a park, and they are deadly.

Been using PlantNet for a few years, rather surprised to see it up here.

It's probably the best of its kind, as long as your photos are sharp and well lit.

For scientific identification you still absolutely want to verify with a dichotomous key, but it's really good for quickly getting genus.

Out in nature, you can cross-check with something like https://wildflowersearch.org/ to help verify that you have the correct species.

For french speakers, you can follow the Tela Botanica Mooc on botanic here : https://mooc.tela-botanica.org/course/view.php?id=12

Thanks!

Semi related: goes well with https://birdnet.cornell.edu/ for bird song identification.

I've used this for a while but I'm never really sure about the results. Honestly I had no idea that identifying trees accurately could be so difficult - and I mean as a human with access to photographs and Wikipedia and books and other people.

Yeah, it's wonderful how complicated the natural world is.

We're still discovering stuff every data. Recently it was discovered that one of the most unique looking critters in the world (the mata-mata, a turtle) was actually two species.

In even grander confusion, a friend of mine was working her way through her masters in mycology. She focused on the fungus Phytophthora ramorum. In the end it turned out that the Phytopthera aren't even fungi. They're algae, an entirely separate branch of the eukaryotes. It's like discovering that your cat is a house plant.

> In the end it turned out that the Phytopthera aren't even fungi.

What? those %&$#!!! things are classified as public enemy!

Thirty years ago I was walking farmers soybeans fields, making weed maps and then writing prescriptions of what to spray. One feverish afternoon with the temperature hovering in the nineties I had an idea. What if we had a plane flyover, grab images and have a software program identify the weeds and spit out a prescription?

I talked to some professors at Michigan State and other software developers I knew. The consensus was it was a great idea but the technology simply was not there or even remotely close.

Little did I know several years later Monsanto would come out with genetically engineered soybeans that you could simply spray Roundup over. Walking fields, weed maps and prescriptions of chemicals became a thing of the past.

To me it was an excellent example of how technology can blind side you at times. Now of course weeds are becoming resistant to Roundup and prescriptions might make a comeback!

Prescription maps and Precision Agriculture are most certainly alive and well.

I tried something similar with Google auto ml a while back. Although I'm no ml expert, I think it's quite difficult to identify plants. There are many different parameters like distance, flowering or not or even time of the year. In some cases it can also be quite dangerous to mix up plants, like wild garlic and lilly of the valley ( I hope I translated those right)

I've had this app a while and just used it to identify Eastern Skunk Cabbage growing near me. I didn't know that some plants could be thermogenic.

This app seems to work pretty well. I tried this identification first on a single leaf and only got results for plantains. But using a pic of the whole plant got an accurate ID.

I’ve been trying to determine whether the oak in my front yard is a scarlet oak or a pin oak. The two varieties have very similar leaves.

According to the app rankings, based on acorn, leaf, and flower pics, it is a downy oak. Not perfect, but I am impressed that it picked up that it was an oak at all.

oak genus is easy to recognize. The species level can be a nigthmare.

I've been using https://plant.id/ for this. It's also the first and maybe only time I've seen my camera viewfinder embedded in a mobile browser!

What about mushrooms? What about slime molds? What about insects? Google Lens does all of the above as well as plants and flowers and fruit and shoes. I think PlantNet looks nice, and I appreciate the web-only version! But Lens is built into my camera, it's hard to beat.

What I think would be useful is an app that could diagnose problems with plants. Some get yellow when dry, some get yellow when too wet. You could tell it the species and it could compare it to training data of sick plants of that species.

(it seems that replacement of taxonomists with robots is going full steam ahead...)

How does it compare to other plant-identifying apps?

(PlantSnap, PictureThis, etc)

I've tried a bunch and plantnet is the best so far

Agree, PlantSnap is not bad either but I still prefer PlantNet

I've just tried PlantNet for the first time. In the one case I tried it on, it seemed better than PlantSnap, which I installed a couple of weeks ago. There's a flower I've been wondering about for years. It looks very like a Forget-me-not, which grows in a number of places nearby and flowers at the same time, but has wider leaves. After quite a few attempts with PlantSnap, it came up with Chinese Hounds Tongue Forget-me-not[0], which sounded plausible. However, after a much smaller number of attempts with PlantNet, it suggested Green Alkanet[1], which I'm starting to think is more likely, e.g. due it flowering earlier and being more weed like. Not an expert, and I know this is only one test case, but based on this I'm favouring PlantNet. Looks like there will be fewer notifications too.

[0] https://www.rhs.org.uk/Plants/5194/Cynoglossum-amabile/Detai...

[1] https://www.rhs.org.uk/advice/profile?PID=1001

I can already see how future ML platforms going to gain a lot from crowdsourced data.

Imagine a platform that can publish an app for classification of any particular use case, and the dataset is contributed and vetted by the community of users

For example: 1. App for identifying animals ( like inaturalist ) 2. App for identifying a car/bike model versions 3. App for identifying languages and translate 4. App for identifying different kinds of dogs

All these apps need to do is come up with a CAPTCHA like implementation. If you hated "Find all crosswalks", just wait for "Find all Lilium longiflorum".

The website is missing an aboutus page, what is this a company, a nonprofit? a project of a research institute? it's not clearly identified.

Agree. They act like they're doing it for public benefit and connected with academia but without being clear if they're a commercial offshoot or charitable or something else. They are accepting donations in a way that implies they're charitable but isn't backed up by clear details (although maybe it's buried somewhere in the site?)

Whether they're planning to be open source with their data would be good to clarify too.

I once wrote an identification app for a some trees with a decision tree.

Then it would ask a handful of questions like, are the leaves jagged or round.

Where did you get the data?

From my biology class in middle school. There was a flowchart, perhaps from a textbook.

Then I had pictures for each case. Do not remember where I got them from

A decision tree app for car problems would be really cool. What is the problem? The car does not start. Etc.

Reminds me of a similar project - Diatoms[1].

[1] https://diatoms.org/

For people interested in identifying plants, the PlantCLEF dataset is a great start. Here's the link: https://www.imageclef.org/PlantCLEF2019

Implement a simple image classifier built with fast.ai and you can go quite far!

I believe it is to be used for research only.

I've used PictureThis with great success.

Is there a good plant ID app that's not based on my camera? I was thinking something that will ask me increasingly specific questions. "Where are you?" "What kind of bark does is have?" "How large are the leaves?" Etc

In book form that'd be called a "plant key" or "identification key". I can't recommend anything specifically, but maybe that helps with the search?

The Virginia Tech Tree ID app is quite good, although it only covers North American species.

https://dendro.cnre.vt.edu/dendrology/idit.htm

Probably a book rather than an app. Search for something like "Field Guide to Plants in <location>".

"Nozha Boujemaa is the scientific co-leader with Daniel Barthelemy (CIRAD) of Pl@ntNet project"

https://project.inria.fr/nozhaboujemaa/

I tried their plant identification game. I think they could really benefit from hiring a game designer.

It would be possible to make something pretty cool out of the database I think.

Google Lens has done this decently for me in the past.

Is there a training dataset for this kind of thing (with nice license)?

Android app not available in the Australian play store?

Please forgive me. A Shazam for plants?

Yes, exactly. :-)

@shrikant @dang

Kindly edit the current title, "Pl NtNet." Change it to PlaNtNet if Pl@NtNet wouldn't work.

Current title looks like it's about Raspberry PI.

I posted it in a bit of a hurry, and couldn't think of a description that didn't sound like shilling the app. Looks like dang (or sctb?) has put in a perfect descriptor!

I misread it as "identify planets from your pictures" :)

Reply on news.ycombinator.com