App scans your pile of Lego and uses ML to suggest projects | Hacker News Reader

App scans your pile of Lego and uses ML to suggest projects

boingboing.net

63 pointsskanderbm32 comments

I don't recall exactly when but i tried this same app at least a year or two ago. I was super excited but results it gave was kinda underwhelming. But again this was a while back. Hopefully it improved quite a bit because i really like this idea.

Last night we tried ChatGPT to see what we could change the “Welcome {MyFamily} Name” at our AirBnB to say.

ChatGPT 3.5 gave suggestions where most of the options needed letters we didn’t have.

ChatGPT 4 took a very long time and then apologized and said it was trying to loop over the letters but it couldn’t verify if they were real words because it didn’t have a dictionary.

Multiple tries returned the exact same results each time.

I’d hate to do this with Legos and then be missing parts part way through!

Counter intuitively, language models aren't good with letters and string manipulation. It probably has to do with tokens being a few letters and the lack of those tasks in their dataset.

https://twitter.com/LEGO_Group/status/842115345280294912?lan...

Brickit is many years old at this point, and many newer competitors are available. Here's a megalist of Lego classifier Computer Vision apps and models https://github.com/360er0/awesome-lego-machine-learning And here's photos of legos sorted by value https://www.webrick.com/all-bricks-pieces.html?p=2&product_l...

Last time I tried it, it was ios only and the projects it suggested were pretty simple. I found that just building something was more enjoyable every time.

It's a neat idea though

Same here. I've paid for the quite expensive (relative to mobile apps market) one-off license only to discover it has a very high error ratio (sees blocks that aren't there) and extremely simple projects to build (in the range of 10-30 blocks total). I wouldn't recommend it to anyone barring the free use for simplest of builds.

I need it for sorting. I literally spend 20 days fulltime finding all the pieces belonging to sets. Yes it was a big box full of lego.

I was just wondering this earlier today actually, then came across this thread. Is there a vision/ML project for identifying lego bricks? I have 6-7 large tubs of Lego and we're a very active Lego family, and I would absolutely love an easier way to sort bricks than bribing the kids and getting "ok" results. :P

This is a great app and inspires our kids to use the lego more. It also, like any manual, does not need to be followed exactly. It really is a great use of computer vision and I personally have used lego more since we got this app.

Replacing one's imagination with ML does not seem like a good thing.

Compared to buying the set of LEGO with a 100 page manual of step by step instructions on how to build something?

I wonder how the training dataset was built

This is the sort of thing where artificially generated samples work well. Spawn a random set of blocks, run a physics engine for a bit to have them pile up, add random lights and render to an image. The renderer can also give you a ground truth annotation for each pixel to the corresponding block type.

Does not work. I tried it. The error rate is too high and kept suggesting builds for which I have no bricks for.

And its never going to work, at least in its current form. The problem is that some bricks will look the same from top while being different on the bottom. That produces wrong suggestions and you can only discover it when you checked all pieces that are needed for the suggested model. Perhaps it can work with basic collection of pieces, like Lego classic but it can't work with random heap I throw at it.

How about 2 or 3 passes, where are you shuffle the pieces around and take another scan or two, to resolve ambiguity.

It will resolve some but it will probably add additional errors like confuse bricks from previous photo with each other. You need to separate pieces and take multiple shots like in that Lego-sorter video. You can probably get away with filming the bricks from different angles, preprocessing video and using that to feed into nn.

We are getting close to an automated Lego sorting machine.

It was already done: https://www.youtube.com/watch?v=04JkdHEX3Yk

Lego is so expensive, just give me a VR app where I can play with (free) Lego.

Basically Minecraft

Good to see BoingBoing still have their finger on the pulse of two years ago.

The article literally acknowledges it in the first sentence.

Yes it does.

This is the opposite of what lego is good for.

Why scan a pile of Lego and suggest projects, when you can also have a robot build those projects for you?

In fact, why not 3D print a finished piece designed by AI?

You still get the fun of cleaning up mold lines for the AI, unless they build a robot for that.

If that's truly opposing what Lego is good for, then why does anyone buy Lego sets to build specific things with instructions?

This is just generating different sets of instructions; it's no different conceptually than downloading a PDF of instructions from someone online and building that.

People build sets all the time, that's not odd. What's likely is that my son would like to build some set, where we've never bought it but already have the pieces.

Let’s not gatekeep. People can enjoy their harmless toys in whatever way they want.

I understand what you're saying, but as someone who spends most of the undirected Lego time with my daughter struggling to figure out what to build, I'm going to suggest that you have fallen into the trap of assuming that the way you do things is "correct" and not simply "the way I do things."

I was turned off by the subscription model. No way I would use this frequently enough to pay for a subscription. And I have a 9 yo who plays with legos every day.

Yes, a subscription model is a big turnoff.

Reply on news.ycombinator.com