MIT, Autodesk develop AI that can figure out Lego instructions | Hacker News Reader

MIT, Autodesk develop AI that can figure out Lego instructions

theregister.com

138 pointssamizdis51 comments

This is entirely beside the point, but I think Lego instructions were my first real foray into technical documentation. It's surprising how well they worked. Their ability to boil down a complicated process into something a 10 year old could follow was impressive. As I got older I sort of dismissed them as antithetical to creativity, but after doing Lego robotics with some kids I began to realize the value I got from them.

An enormous amount of effort has gone into making Lego instructions as clear as they are, regardless of what age you are and what language you speak. They really show what is possible.

The new ones are much clearer than they used to be in the 80s and 90s.

I'm convinced the old-style ones were the reason I buried the needle the one time I was given a cognitive test that specifically tested spatial reasoning. The old sets' instructions were basically 3d spot-the-difference puzzles, in which orientation might even change between states, so they were pretty much training for that kind of test.

The new ones have a little of that, but a lot more arrows pointing things out, little insets highlighting important parts, et c. Plus they put the pieces in numbered bags so you don't have them all out at once, for medium-sized or larger sets! Much, much easier than they used to be.

Flip side is the old ones used larger pieces, on average. The modern sets are all fiddly, tiny bricks, which does make them a little harder to assemble (and much harder—nearly impossible—to repair from memory if they're damaged during play)

Now if they could print them in higher quality so I’m not constantky guessing if that’s a black piece or dark blue …

Maybe I’m older and my color vision isn’t as good as it used to be, but damn it’s frustrating.

I don’t think it’s the printing. The downloadable pdfs have the same issues

> As I got older I sort of dismissed them as antithetical to creativity

Yah, there's a lot of criticism of the "directions".

My kids build big elaborate modular buildings from the directions... and then leave them as-is...

And then try to build equivalently complicated things from loose bricks (sometimes copying a little bit of a mechanism or a trick they've seen elsewhere).

I think the instructions are great. They just shouldn't be the be-all, end-all.

yup. the blocks themselves give you the first-order building components, and then the instructions give you an increasingly library of higher-order building components to work with... like reading code

AS AN ASIDE; When I was a young'n I had a lot of legos - but my granfather was a nuclear engineer with GE... (helped build Hanford)

There was a technical design/drafting book in his book case I picked up where I was around ~6 or so...

I drew every single diagram/ortho/iso whatever in that book. Multiple times.

When I got into middle school, I was opted to elect a highschool class, and I chose technical drafting...

When I went to class, the textbook was the same one from my grandfathers bookcase - I sailed through that, and then...

This was what got me into design and architecture and CAD... I then setup the first cad network in my highschool, while I was still in 7th grade.

I was in VOCAD by 9th, and came in second in the CAD Olympics in 1993 (I was the fastest ever to finish the test in 30 minutes, on a 3 hour deadline.)

THE ENTIRE POINT BEING ;

Get kids in front of technical design/docs/whatever is complex as EARLY as possible.

There is no loss in teaching a toddler calculus. They may wind up being a flamboyant broadway star, but at least they can maths.

> They may wind up being a flamboyant broadway star, but at least they can maths.

What fun is it to be a broadway star when you know maths?

You can calculate probability of success as a broadway star.

Understanding that the world and your interests may not be one dimensional... ;)

AAAAAND TWO AND THREE AND FOUR AND FIVE AND SIX AND SASHE AND TURN THAT NOMIAL INTO A PRIME AND FIVE SIX SEVEN EIGHT! HANDS LADIES!!! LET ME SEE YOUR JAZZ HANDS TIMES THE SQUARE ROOT OF SEVEN DIVIDED BY THE OUTPUT OF THE GLITTER FACTORY!!!

Sounds like it could be fun.

I know calculus and programming and all those neat things. Acting and performing are still great fun

More so that they do it with few words and mostly images and icons.

Brings back an old memory. I took CMU's AI class in 1986. Most CMU classes were really two classes - one about the lectures and one about learning to hack. I did my project for this class in Lisp and wrote an engine to solve Lego assembly problems. While testing, I gave it a problem to solve, and it came up with a better solution than mine. Was my first AI eureka moment.

That sounds really interesting. It was awhile ago so you may not remember, but how did you “operationalize” the problem? I wonder how that kind of problem / task is structured.

The problem was structured with two inputs. 1. the available Lego pieces. 2. the structure to be built. At least that's what I remember.

I developed an AI to do this- well, I guess it wasn't artificial. I had a kid, and after showing him a few times (and correcting some mistakes due to the instructions sometimes being... as sparse as technically possible....) he had no problem. Now the issue is getting rid of all the 5-year-old Hogwarts Castles, etc...

Feel free to break them down and ship them to me, or some other parent of a young kid. Those sets are expensive!

There is an app that's been linked on HN a few times called Brickit. It analyzes a picture you took of a pile of Legos, identifies pieces with object recognition, then uses AR to highlight exactly which pieces to grab to build a model. If there's such thing as a beautiful app, this is it.

https://brickit.app/

Don't people get their kids Lego to get them away from phones?

This is begging for someone to make a robot using MEPNet that can assemble any kit for which you show it the instructions.

Bonus points if it's made out of Lego and can build itself.

Interestingly, people were predicting[0] in 2018 that the robotic equivalent of this task would be solved by 2023:

> The team also predicts that by the year 2023, a machine will be able to “physically assemble any Lego set given the pieces and instructions, using non-specialized robotics hardware.” The paper cites University of California Berkeley research that showed a machine joining two Lego pieces together in 2016 through reinforcement learning.

[0] https://www.inverse.com/article/44660-lego-building-ai-robot...

Official Research Page:

https://cs.stanford.edu/~rcwang/projects/lego_manual/

- which includes the paper, code and video.

I wonder what they taught the AI first to make enough money to be able to afford legos?

I'd be more impressed if they can figure out Ikea directions :-P

I actually find ikea instructions quite clear and reasonable. I don't know why so many people have a hard time with it. Maybe spacial awareness is not something you can learn easily.

I get a real B&W experience with Ikea - it would be interesting to have a technical writing expert compare the Ikea instructions and the Lego instructions. Lego seems to generally be accepted as more universally simple, and yet Ikea products frequently have fewer parts and steps.

I find both quite clear. I think the main difference is that it's a lot more frustrating if you mess up a step in Ikea instructions than in a Lego build. So if you're bad at spatial reasoning or following instructions, you can still have fun with Lego, you'll just need to redo some steps sometimes and you might not even remember it, but the same kind of mistake with Ikea will throw you in a fit of rage.

Ehh. Ikea instructions have their issues:

- branching paths aren't well marked

- I often find typos (e.g. go to page 16 when it's really page 17)

- parts page sometimes doesn't match the supplied parts in terms of size and shape

- it's not always clear when parts are optional

- some things come as separate pieces and it's not clear how they go together as the instructions are for each piece and not the entire assembly

I hope Ikea is reading this!

I don't get it either. Every single non-Ikea flat-pack thing I've assembled has had much worse instructions than Ikea. Plus usually been more expensive and lower quality than the Ikea equivalent.

I mostly agree, but once in a while there are very subtle things, like 4 parts that at a glance look identical in both the diagrams and in reality but only if you're really paying attention do you realize they weren't actually the same. I've had to disassemble and reassemble things when I missed such a detail and then things didn't fit/work as expected.

Of course it's my fault for missing the tiny difference. On the other hand, they could also design better. Put post-it style (meaning easily removable) stickers on the different parts so the instructions can call out the step 5 needs the parts with the stickers, not the parts without. Etc...

I got cabinet from Wayfair, I don't remember the brand but the instructions were surprisingly better. As just one example they had put every set of parts it it's own numbered bag. The bags were all one long strip (kind of like the way zots candy's come) where each portion of the long crimped bag had different parts and a number printed on the bag. The instructions then referred to the bag. Vs Ikea where all the parts are in a single bag and it can be difficult to notice some of the bolts are 60mm and others are 64mm

I've never failed to correctly build Ikea furniture and I've probably built ~60+ pieces but there is arguably room for improvement.

Ikea instructions are mostly fine. They could do with what lego does if there's similar parts though. A lot of ikea furniture has chiral parts and it almost never gets called out in the manual.

Try assembling furniture from another maker (e.g. anything from Target, Home Depot, etc) and you'll be thirsting for the clarity, completeness, and the large, legible print of Ikea instructions.

Step 1: build half of the bed. Step 2: build the other half.

a bit off topic, but is there like a more affordable lego like alternative? it really disgusts me how expensive lego and litigous they are. so many Americans live in poverty who would love to get their kids to play with lego but can't because its ridiculously expensive.

I’m not sure how they compare in price but I really enjoyed erector set and k’nex when I was a kid 30 years ago. And now I’m a robotics engineer!

Also, an Ender 3 3D printer provides a lot of similar construction skills and learning (with the printed objects) for the price of one high end Lego set.

+1 to buying a child a 3d printer. There are okayish ones for about $140 now. About age 6-8 seems like a good time.

Set them up with an easy to use bit of cad software and show them how to 3d print a cube... Then let them 3d print whatever they want to make.

3d printers have hot parts - let them burn their fingers once or twice and they'll soon learn which bits not to touch. After that, a 3d printer is pretty safe. Test it first, but generally the motors aren't strong enough to crush a finger placed in the wrong spot.

The only downside is many 3d prints take 1 hour + to print, which is pretty frustrating for a child. You need another activity to entertain them during printing breaks.

I have no kids but I definitely found that waiting for the print was tough! Often what I do now is I get all my design work done at night. have fun getting design features sorted, or searching online libraries for printable parts, then get everything configured and start the print before bed. Watch the first layer or two to make sure it's gonna work, and then go to sleep while its printing. I wake up and the print is done!

idk if this would work for kids but it works for me! it also means that prints which are 8-10 hours don't feel long, because I start it an hour before bed time, then as I am waking up and doing my morning routine the print is on its last few bits!

Try going to <yourcity>.craigslist.org and searching for Lego in the For Sale section. You'll likely find bulk Lego for super cheap.

Toys—even Lego—are damn near free if you aren't picky and will accept whatever you can find used. Often they're literally free. Kids often outgrow toys before they're worn out or broken, and for some reason people hate giving used toys as gifts so there are just tons and tons of them always being dumped on the used market, with relatively little demand.

You could set out with a $20 bill and outfit a 6-year-old with a totally adequate set of toys, from scratch, by just walking a neighborhood on a yard sale day.

Who outgrows Lego? The only exception to that is my Lego went to my nephews, pretty much only because I don't have the space for it in my apartment.

Thrift stores have piles and piles of Legos. I've seen various small stores around that sell Legos in bulk. They'll have huge bins, and you can buy them by the pound. I used to buy a few official Lego sets back in the day, but also I would buy the big bucket of generic blocks. They were cheap and gave you a huge stockpile.

Stumbled over a (German speaking) youtuber, "Held der Steine" (Hero of the stones) https://www.youtube.com/watch?v=HVtbNWH_9So

Who reviews "Bricks" of all kinds including Lego. He consistently is annoyed by how expensive Lego is compared to everything else, with usual worse quality (in terms of design, fun to assemble, etc). If you can bear google translate or even speak German, he may be worth listening to.

Anyway, got sucked into that and got my first "brick" set since around 25 years ago and it felt like Lego (as far as I could remember). Instructions, presentation, the stones, everything.

I believe "Cobi" and "Bluebrixx" are often mentioned as good, affordable (seems about half price for similar set) and lego compatible. From what I understand the patent for the particular form factor of the bricks ran out, which is why there are a bunch of alternative now.

I got that one: https://cobitoys.de/small-army-ww2/panzer-und-fahrzeuge/panz...

I live in the UK, think I got it via amazon, so maybe more difficult for you Americans.

I have to ask whether or not you are referencing some specific sets, like high-end Porsche, Lambo ect. or are you talking about LEGO in general, as it seems you have an axe to grind.

If you are genuine about your inquiry, then at this very moment there are 3 supermarkets around me with a toy aisle and a discount basket which includes discounted LEGO sets (30-50%), either due to products running out or box being damaged. I'm assuming there is something similar near you.

Second, and more important, if you or someone close to you work with an organization that helps kids living in poverty, you are welcome and encouraged to contact LEGO directly and ask for help. They have several organizations and initiatives inside the group that help either by directly donating sets or by donating money for other needs.

If the 400 billion or so Lego pieces that have been produced were pulled out of the attics and garages of the world, the market would collapse over night. I have at least $2k worth of Lego in two giant tubs in my garage, collected over the years my now 20yo son was growing up. I honestly have no idea what to do with them - they're surprisingly hard to give away and I don't feel like dealing with eBay.

Sadly, I suspect most of the pieces out there are in landfills or the oceans.

Try Craigslist. That's how I sold a lot of them locally with little effort.

Keep it for future grandkids. Our parents both had sets of Duplo and Lego still in the attic. Now my daughter and her cousins are playing with it.

Side note about MITs marketing... The original paper [0] was published by 5 authors. 3 from Stanford, 1 from MIT, and 1 from Autodesk.

I find MITs aggressive marketing pretty off-putting. Any discovery related to MIT _always_ has MIT in the headline, even if their involvement was secondary. It's annoying and makes me feel like MIT cares more about optics than good research. If I'm not alone in this belief, MIT will erode their brand by doing so. They certainly have for me.

[0] https://cs.stanford.edu/~rcwang/projects/lego_manual/

> If I'm not alone in this belief, MIT will erode their brand by doing so. They certainly have for me.

They already eroded their brand when they provided cover for Jeffrey Epstein.

Reply on news.ycombinator.com