I built a DIY license plate reader with a Raspberry Pi and machine learning
towardsdatascience.com
towardsdatascience.com
Tesseract OCR can do this, using only the Raspberry Pi, at a "good enough" framerate for any real driving situation.
Machine learning is a fine tool for this job. The specific machine learning setup is an overkill, though.
Reflective helps you only if you actually shine light from the direction of the camera. which you can't practically do from within a car (the setup described in the post), and in most jurisdictions cannot legally do at all.
If their car was electric, this project would be increasing the power consumption of the vehicle by around 10%.
My view is, if your car uses electrons, and some of those are for compute, and you just offload the compute to the cloud, you haven't actually reduced the total electricity consumption.
Similarly here, the total "footprint" of the car is increased by adding this feature.
mentions that they used 20 K80 instances to get real time inferencing. That seems a bit excessive IMO for an object detector.
Once cortex started supporting gunicorn (still yet unreleased but present on their master branch), I was able to reduce the number of GPUs significantly. Finally, the detector only needs 2 T4 GPUs and the identification part is the most expensive one (10 T4s GPUs) for a grand total of 12.
Converting the models to use single-precision could further reduce the need to about 1.5 GPUs for T4s or just 1.2 GPUs for V100 - which in both cases it would still mean using 2 GPUs.
You can also customize the network to your use case, e.g. you don't need YOLO's default 5 anchor box sizes if you know the thing you're detecting is a license plate.
Also, profile your code and see where your bottleneck is. If your bottleneck is at NMS for example there are things you can do to speed it up. I've seen a lot of cases where the neural network runs fast but there's a lot of Python bloat for pre/post-processing -- not sure about yours without seeing code.
You really should be able to run a license plate detector/reader on something a lot smaller than a V100. A Xavier or quite possibly even a Jetson Nano would very likely be good enough if you use it well.
Object tracking is yet a very good idea. I will consider it. Anchor-box tuning is another very good idea.
Also, the CRAFT text detector that I'm using should IMHO be removed. Instead just use a very well trained text recognizer (like the CRNN I'm using). The text detector is expensive computationally since it's based on the VGG-16 model.
Then convert the models to use mixed-precision.
All in all, I think the performance improvements can be anywhere between 1 and 2 orders of magnitude.
Either the Neural Compute Stick or the Google Coral both have more than enough grunt to run real-time object detection models. Both will run on USB2 power. I don't know the overhead of good OCR, but license plates are a very standard format so perhaps you could train a second detector to extract the letters?
Even if you do OCR in the cloud, local bounding box extraction would save a huge amount of bandwidth.
If you’re interested in trying it out: https://siftrics.com/
Detecting the license plates is really cheap computationally speaking, but not on the RPi. The most expensive part computationally was identifying the words (letters) - that's because detecting the text within the bounding boxes obtained from YOLOv3 is based on a VGG-16 model. Running that multiple times in a single frame (for multiple license plates) is expensive.
Surprisingly, the bandwidth was the least of my concerns. I was very surprised to see I didn't need much at all. For YOLOv3@ 416p and @30FPS I need about ~3Mbps. I wouldn't consider that much.
Now, this is a demo of what a production system could theoretically look like. I know it could be much better optimized.
3Mbps doesn't sound much, but that's constant bandwidth. I have a 15 minute commute, which would make each journey about 400MB of data. That adds up quite fast especially on a mobile contract.
You can do real-time number plate recognition on a Pi3/4 with a Coral TPU or Movidius 2. Both of which costs ~US80. It's probably got a lot to do with hammers and nails. Not everything needs to be 'web-centric' or 'cloud based'
A+, would hire.
For trying out something in a few hours, of course you don't want to spend hundreds of hours setting it up, by definition. Yes, the result is "just about works, but doesn't scale" - but that's the point of experimenting. Sure, this is a LEGO-style experiment in reinventing the wheel, but exactly for that, an excellent way to start learning about this problem domain: power consumption? Latency? ML basics? Sure. That's hacking at its core - even though the project is rudimentary.
If I'm not mistaken Larry Page used to be praised some time ago for building a printer out of LEGO pieces (that may just be an urban legend, I admit I never verified this information).
https://www.geek.com/gadgets/a-32-hacking-tool-can-open-your...
(Perhaps recognizing the car's license plate plus MAC address would be more secure...? Many cars use BT/WiFi nowadays.)
Thus the license plates can easily be argued they are PII.
Take this in the meantime: https://outline.com/qsv7ab
So try deleting your Medium cookies as a workaround.
It is easy to imagine someone with these glasses from Bosch, walking around in a bar. Personal information about other people would projected right to the retina.
What FPS you getting out of it?
It's really buttery smooth if I disable the recognition part and just leave in the detection. Since it's demo project (something I just wanted to experiment with), my focus hasn't been on optimizing it. Lots of improvements could be brought to it.