Brickit – scans your Lego bricks and helps you build new creations | Hacker News Reader

Brickit – scans your Lego bricks and helps you build new creations

twitter.com

646 pointsChrisArchitect119 comments

I just tried it on my kids' Legos. I made two scans, one per box: it found ~2500 bricks, which I think is a very high hit rate. However, the build suggestions are very, very lacking. For reference, my kids are 8 and 4. The builds proposed are very simple, small, and still mention missing pieces (probably related to colors, which honestly most kids don't care about). I think my 8yo would like it, she tends to build "worlds", assembling many related contraption that each are fairly simple and match the themes found on suggestions.

As for my 4yo, forget it. He is a builder, making very elaborate and complex constructions, the bigger the better. He would find suggestions absolutely uninteresting. I didn't try to scan Technic legos (which he is fond of and make astonishingly complex contraption given his age). But it's an even more complex problem to solve.

I suspect work on the app was focused on recognition, which honestly is impressive. The second part, finding models, maybe need a bit of work ? I would recommend at least having an option to "ignore color": it's nice to have a build that looks good, but overall most kids - including myself - like to build first.

We ignore color by default now.

MOCs library is not very big at the moment, but we did not intend to go viral either :)

The library of different builds will grow for sure!

I hope the library evolves, this has potential to give new build ideas in a very simple to use package !

you should let people photograph their builds step by step and upload them. user generated content would create a nice network effect.

Oh I was looking for something like this but with a bit of a different intent: building from instructions, but of course you have to look for pieces in a random pile of legos. Use CV to identify the piece(s) you are looking for.

Would be awesome! Actually thought that was what this was doing at first as well.

Just downloaded it and tried it out. For the instructions it has, you can tap a part and it lights up the bounding box for that piece in the original image.

Curious about who you are and why you built this. What’s the story?

One of the founders got sick of looking for Lego pieces while playing with his son. And he thought that there should be an app for that :)

This is so cool! UX is so simple and smooth. I was impressed by just it counting how many bricks I put out. This will be fun with my daughters as they get older and we collect more.

I'm a big fan of all Lego communities like https://www.mecabricks.com/ and https://www.reddit.com/r/lego/ and your Brickit just made my day. Thank you!

> but we did not intend to go viral either :)

This is hilarious and makes me wonder about the relationship of the value of going viral with the repercussions in going viral before you feel you are ready for it.

Great, lots more people know about the app.

Not so great, some people's first impressions are negative because of flaws that you are probably already working to fix.

Overall it is an impressive app with lots of potential to be even better.

So true!

We were hoping to start marketing alongside with the release of Android app )

I always wanted to ask this to someone who made this choice, why build the iphone app before Android app? Also are there no good (in your view) frameworks where it will just work for both the platforms?

Cant answer for their reasons, but anecdotally I've seen this happen a few times and in each case the 'founder' of the app already had a load of experience developing for iOS / macOS / Swift. They then hired someone / learned how to do Android once the prototype was complete.

A lot of these things start out as ideas or hobby projects too, so generally the app starts being built for iOS because why wouldn't you write it with the stack you're already comfortable with / can use your own phone to test it.

I don't necessarily think its always a conscious 'we must do iOS first', but if you own an iPhone already and do iOS dev at work...

Kids don’t need an app like this, they still have imagination we’ve long left behind. :’)

Not at all. This idea that kids just spontaneously form ideas from a vacuum is obviously false. Inspiration comes from all sorts of guidance. I would have loved to know more techniques for building things when I was a kid and was stuck in the rut of similar ideas for a long time. I craved books that showed how to build things. I also created my own stuff but instruction material was gold. It took you to more exciting things than you could think of yourself.

My kids are way more creative than I at same age... because they watch speed builds on Youtube compared to me in a Vacuum in the 90's. I absolutely agree that inspiration comes for all sort of things, not only those videos. I applaud this app, I hope it evolves to offer more value to older kids.

> Kids don’t need an app like this, they still have imagination we’ve long left behind. :’)

My 5-year-old likes playing with built models a lot, and also likes building with instructions, but not free building. More instructions for the huge pile of Lego we have, he would like.

My 3-year-old is pretty much exclusively into (often quite elaborate) freebuilding. Even when kids are super creative, it doesn't manifest in the same way.

I agree, I observed my 3 year old often and he is more creative by applying his imagination. He tends to lose interest in building it himself he see something better than what he created. When I am helping him color and I color it precisely he just lose interest in coloring and say color it please you are better. If I don't help him at all he will finish the whole piece and even be proud of it. Now I color just like him so he will not lose interest during the coloring.

Whenever I got a new Lego set, I built it once, using the manual and then would take it apart, a day or two later. For the bricks to be absorbed into my pile and let me build something new, better. Something I had imagined myself.

For me there are definitely two kinds of Lego users. Those who struggle with the building instructions. For them following these is almost meditative. Then there are those who don't and who have their own ideas. They will skim through the instructions and build the set once (if at all). And in general just welcome the addition of (new) bricks.

For me the resp. set served mostly as fuel for my imagination. If it was a spaceship, the next thing I would build was one. Just not the one in the set.

My dad built me a chest of drawers with lots of boxes for sorting Legos. And sorted them for me when he gave it to me as a present, for my 5th b-day. It was almost as tall as me at the time. I.e. I could just open and peek into the top drawer.

He also told me why it was important to sort the bricks again, after I deconstructed my opi:

I could let my imagination run free and find the bricks I needed quickly when building my next piece.

Before, when they were all a pile in a big box, very often the creative flow was killed by the annoyance of searching a pile of bricks for the one I needed. You start building something and eventually you just stop.

This thing looks like a tool for people who lack both. Sorted Legos and imagination.

The chaos of mixed bricks can be creative in its self. I used to play this game where I had a small baseplate as an isolated island; Every morning, I'd grab a fistfull of bricks from the grand pile to "wash up" on the island shore, then challenge myself to build from them a home for the stranded minifig on the island. It was exciting not knowing what pieces you'd get!

I haven't had an online comment resonate with me as much as this did in a very very long time.

I’m surprised there isn’t any one-the-shelf solution for sorting Lego. As you said, it’s the first step.

The indictment of people who would enjoy this as not having imagination is ridiculously smug and self-congratulatory. Maybe use some of your imagination to think of scenarios where this could augment creativity?

Imagination and 3d visualization are different things. Some people enjoy refinement and improvement from an existing state versus starting from zero.

Having quick generation of ideas for small builds means you could focus on being creative at the scale of a whole scene or diorama.

It doesnt have to be considered self congratulatory or smug, or indeed an indictment. The gradual erosion of any need for individual thought or creativity is a real problem nowadays. The existence of a tool like this is just a shame.

I've tried it this week and it's really cool to see it process a big batch of bricks.

I was a bit disappointed by the suggested builds, because they're small, but I also understand it's early on and a difficult thing to do.

Not vital, but I was surprised that it doesn't look for colors at all.

One thing I'd love is the same technology, but used to look for a particular part. It does that once you pick a build, but imagine building your own thing rather than suggestions and having the app tell you where you can find the piece you're looking for. If you're adventurous, you would even have a custom tray with a spotlight directed at your pile :)

I've been trying to take thousands of mixed parts back into their original sets and I was hoping this might be helpful. The technology itself probably would, but not packaged as it is for the moment.

All that being said: really well done!

Finding builds is the easiest part of the whole app, it's just a simple subset check on each design.

"Finding" the builds is easy, but first someone has to design all the builds to be found.

I wonder if the tech could be used to take some pictures of a completed model, recognise pieces on the outside, guess pieces on the inside that would work and produce instructions for the model.

Tried it just now and it’s amazing. A very palpable experience of I assume ML classifiers that I hadn’t experienced before. Showing me where pieces are located is the best part IMO as I’d usually spend most time looking for specific parts when building something. That said I’d love to just have an index function where I scan the bricks and can then search for eg a 2x4 red brick or similar.

Lastly I won’t show this to my 5yo just yet because I love the things build by them from imagination only and I also think having to search for parts, maybe finding something else that’s a better fit or gives a new idea is an essential part of play

> Showing me where pieces are located is the best part IMO as I’d usually spend most time looking for specific parts when building something.

Exactly that pain brought Brickit to life!

Any chance of getting a search function? Or simply a sorted list of all bricks found with “tap to show in scanned pile”

Technically it can be done and we had prototypes that did something similar.

Hopefully mobile team will manage to implement all the improvements that come from users :)

Making an inventor of all my lego and adding a search function would be so amazing. This is in my opinion much more appealing than getting suggestions for what to make. I'm also not in what I assume to be the target audience, but I would likely pay for that feature.

This is very impressive given that reliably recognizing a particular brick with custom built optics and near ideal lighting is hard enough. Props to whoever built this, I very much appreciate how much hard work this must have been, especially labeling the data.

Do you have any figures on the error rates?

I don't have numbers but they're pretty high. At least, similar bricks are often mistaken for each other (1x2 and 2x2, similar but slightly differently sloped angles).

They should integrate this with bricklink - I feel for all those people who have to catalog all their little bricks to sell them or just 1 or 2 cents a piece

(but a great way to spend the holidays, reconstructing your old sets and having to order the missing pieces from all over the world)

This is a really cool concept. The scene from the video where it is identifying bricks from a pile seems like a crazy awesome computer vision application. Does anyone have any references to how they managed that brick identifying algorithm?

Several neural networks, lots of hard work in data labelling, more hard work in data synthesis, even more hard work in dataset curation.. et voila!

Source: it was me with the team who did all of that.

This is really cool! Do you have any in depth posts describing the ML behind this?

Not yet, but it seems that we will talk about it

To dig deeper, there are couple of interesting nuances there:

- extreme number of objects on single photo, typical number of visible pieces in large pile is 1500-2000

- extreme number of classes in multi-class classification, there are ~1000 most common Lego bricks and up to 30000 classes if you include rare bricks and different patterns

- really hard data labelling: one photo can take up to a 5 work days to label

Very cool! How do you deal with occlusion? What kind of NN are you using? Would an off-the-shelf model like Yolo/SSD/FasterRCNN work for this or did you have to devise some new tricks?

I am really curious about the data labelling. Do you mean that you took photos of piles of Legos and labeled them by hand. Or individual Legos from different angles?

Also you mentioned data synthesis. How would this be possible? Unless your suggesting that you rendered photo realistic piles of Legos and used trained on them because if that is the case, please do a write up of the project. I can't imagine more interesting way to generate training data.

Super neat! With all this hard work, how is it free? Am I / my data being sold or is it just a “student tech demo” with an Apple dev account ?

It seems like "go viral, get acquired by Lego" would be a reasonable game plan.

Yea, this. An app like this is a total and legal potentially huge competitor to Lego. I’d wager Lego will happily pay 100 M€+ if this would never come to fruition in order to keep the new box with book flow not slowing due to this idea. The cat is out of the box however. This is disruption in optima forma.

Since downvotes are not my thing, I’ll explain my logic. The value of Legos is return customers. Build, break, drop it in the pile and never build again. (This with great fun obviously, but my experience is no box stays separate in the long run.) Any outside disruption that massively increases the replay value of boxes (perhaps even find that old box from among 20k pieces?) is a threat to Lego. They make about 1 € billion a year on 6 € billion gross. You have to take the lifetime value of the disruption, hence my 100 M€ estimate. It is an estimate that says the repeat value of Lego will rise for the customer with a fall in repeat sales for Lego in the 1% range.

I tend to disagree. Lego's main competitor isn't their own old sets; it's alternate entertainment like XBox or YouTube.

I would estimate that any innovation like this that encourages people to pull out and play with their Lego strongly increases the chances that they'll sell new sets to those people.

Congrats! I have been working (or rather I tried to find someone competent to work on an app like this) but it is still in proof of concept phase.

example video also found on their instagram (which is where they posted their 'how it works'? weird)

https://twitter.com/AlexanderNL/status/1410253599502962692

Since that page has something publicly viewable while https://brickit.app/ seems not to, we've switched the URL to it. Thanks!

WTF! Instagram wont let you view the page without logging in, which, I DONT WANT TO! F*K off Zuck! so, to anyone who has a product like this or wants to promot it, put a video on Youtube (or anywhere for that matter that doesnt require a login to view) so I dont have to give Facebook (or anyone else for that matter) my info... /end rant...

Try viewing it on Nitter [1]. It's one of those (no|min)-JS frontends, along with Teddit [2] (Reddit), and Bibliogram [3] (Instagram).

But still, does anyone know why video quality on Twitter is so bad?

[1]: https://nitter.eu/AlexanderNL/status/1410253599502962692

[2]: https://teddit.net

[3]: https://bibliogram.art

The amount of businesses who use Instagram and Facebook as their primary method of serving customers is outright depressing these days, as both sites now don’t let you view anything unless you have an account.

I can’t see the menu and opening times for half our local cafes and restaurants, can’t see what’s on for local events in my town, etc etc.

It’s infuriating.

Governments too. Live streaming press conferences on FB, so much about data protection and privacy.

We downloaded this last night and my 4-year old has been having a lot of fun working through his 50 or so build options.

We routinely get various lego build books from the library, but it's frustrating how almost every build requires a few critical pieces that we don't have. It'd be cool if this app could tell you if there's some set or kit or odd-lot pieces we could buy that would suddenly make a large number of builds available.

Thats NP-hard.

But how big is n?

I don't think it's NP-hard to identify the individual pieces you need, then show you how to buy them on bricklink.

That's what Lego does.

It seems almost intentional that each set has several rare pieces that can not be found in other sets.

I think that's what make a model such a nice experience. It's different than building yourself: you end-up with something that looks definitely better than an amateur build. You didn't work on imagination, but end-up with an amazing build. Then you can use those unique pieces to put some highlight into your own build !

You can forget about the 'amost', it is intentional and has been for many years. The time that you could build a model perfectly from the bricks you already had is long gone.

>but it's frustrating how almost every build requires a few critical pieces that we don't have

This is to add exclusive value to each set. Look at bricklink.com and find an expensive set, and you'll always find one or two pieces that are a unique color or completely unique to the set that are astronomically high in price. The original Millennium Falcon UCS set for example has two grey (I forget which specific grey) ladder pieces to highlight the engine, and those are the pieces that were most expensive last I looked.

Well I'm happy someone posted this video to Twitter to be shared because the "See how it works" link on brickit.app seems to take me to an Instagram account creation page.

Very cool, I expect I'm not alone among HN's audience in having grown up building legos and continuing that hobby as an adult. Sending this to my parents to see what happens if they point it at one of the boxes of random bits left over at home. Fortunately I can send this tweet because they do not use instagram.

Edit: I see that in the time it took me to post this comment I became the third or fourth person to have registered the same complaint. Apologies for the clutter.

Now kids can learn what it means if your job is taken over by AI.

Ideally kids can now be creative at new levels, but the cynic in me also sees children simply not coming up with their own design anymore. Anyway, while I can certainly romanticize the tedious search for pieces and perhaps getting ideas in the process, quick detection is probably a net positive.

Are you gonna do android? I have an old iPhone 6 otherwise, will it work on that？

Yes, Android app is being actively developed right now.

iPhone 6 would be very slow for a pleasant experience. If I'm not mistaken.

We use iPhone 8 as a "low end device" for testing.

Latest iPhones are performing extremely well on this task

Thanks, I explicitly came to this thread and `ctrl + f` then searched for Android to find this comment!

I'd pay money for this on the Android store for my kids to use it!

Cool idea. We should have something to scan our code base & build something without writing single line of code.. but just pulling code from existing repository.

Very cool, but I don’t want to sign in with Google, Apple or Facebook. How about a stand alone login? For some reason I trust you more than big tech.

Should incorporate this technology in some kind of wearable device that scans the floor as you walk and warns you before stepping on a lego.

Super impressive. With a young LEGO builder who has cranked through every set and every instruction booklet he gets his hands on, I recently delved into the world of MOCs thinking that’d keep him busy, but even with sites like Rebrickable (which is awesome!), the instruction options are surprisingly limited.

How are you sourcing instructions for Brickit?

What’s MOC?

It stands for My Own Creation, what the AFOL (Adult Fan of LEGO) community calls a creation of any kind that they build without following official instructions.

Looking forward to trying this with my 6 year old. Not sure if already baked in but suggestion would be for it to enter ID of a set you know you have in the pile and help you put it back together again...

My son (12) and I pulled out the old box of unassembled legos. It works fairly well but misidentifies some pieces. Also it's not totally clear how to use the "find brick" feature- does it update after you remove a brick? Can you make the "find brick" feature work in realtime, as you move the camera?

Also, can you disable "show builds with missing pieces"?

I gave it a shot with a small sample of my kid’s legos and it seems like it did pretty well, even with a bunch of technix mixed in.

Tho, I’m not sure how well it would really work for approx 10k pieces, as there’s not really enough flat space in the house for them to be spread out.

Anyway, he’s off to play with the app for a bit.

Inspired by this thread, over the week-end, I just made an open-source implementation of something similar.

Show HN: Lego Brick Scanner – open-source Boilerplate

https://news.ycombinator.com/item?id=27738232

This seems really cool. At least I think it is - I have Android. But how does it solve the problem of more complex bricks lying on the side and actually hiding their complexity? They may look like a standard simple bricks from certain angle.

This is awesome, can people submit creations too? would be cool if folks can submit randomly created combinations (as well as the pieces required) and it gets added to the directory of possible creations.

Please do this for my fridge/pantry and show the dishes I can make!

Have you tried taking a complete set and laying it all out for view, then seeing if it matches the set correctly?

I don’t have a single complete set anymore … but I do have happy nieces, which I vastly prefer :)

Would it be possible to export some kind of list over the scanned legos to put into rebrickable and other sites? It would be really usefull to catalog new lego purchases.

Please charge for this app. Don't do in-app purchases.

Isn’t the point of legos to use your creativity and come up with new ideas?

I really dislike this future of choosing between different options, rather than creating new ones.

Such an arrogant and pretentious attitude, honestly. Come on. It's like a carbon copy of "kids these days, reading books instead of just using their imagination, what a shame".

You don't have to use the app, you don't have to use Lego, you don't have to do anything. You can even bunker yourself up and never communicate with the outside world if you want. Just stop being so negative with people's cool apps. Please.

Your reply here is about 100x more negative than my comment.

I'd like to see a pick-and-place robot to build LEGO models, but probably too much to ask for.

Maybe I used legos different than everyone else as a child, but this would have ruined the experience for me. I don't a device to tell me what to build I want to imagine the things myself.

From a technology perspective its very cool, but it doesn't appeal to my childhood love of legos

The cool part isn't telling you what to build, it's finding the pieces you're looking for.

? It tells you what to build. It doesnt help you find pieces.

It does, it shows you where the pieces are.

I can’t wait to get home and try this out. Does it work with duplos?

Not yet, but Duplos and Technic support is widely requested, we will add support for them

Having trouble finding the Android version.... :-(

Now someone make one for K'nex!

Now just add a feature to show me where the relevant pieces are in the pile :)

If you watch the video, it shows exactly that at the end (albeit quickly).

Awful. What's wrong with just using your imagination. Truly awful.

Examining and replicating the work of others is a significant part of creativity and harnessing imagination. There is a record of this stretching back into history with painters, sculptors, authors, musicians and more.

Do you think that playing Mozart or Beethoven on piano ruins the imagination of a potential composer? That just isn’t how it works.

Wow!

To see how it works I need to log in with an Instagram account? No thanks.

(thanks to the person who posted an alternate link, but it's weird that I can't view the video linked on their homepage)

You do not need any login to scan.

Just download the app and core functionality is ready to be used.

The problem is that your "See how it works" link prompts people to log in. I don't want to log in, so I guess I'll never find out how it works, or download the app.

I see. It would make sense to self-host this video.

Will forward it to product guys

Yeah, I can't see the instagram thing despite being logged in.

Just host it on YouTube?

Self-host might be a good option, or youtube or vimeo. Or even Twitter. But Instagram is a wierd choice.

Probably not twitter, any remotely popular link shows 'sorry we can't show this, maybe try reloading' for users without an account, unless they're very persistent.

Meta: this had gotten much better in late January, but is back to where it was again.

The issue is that the "How it Works" link points to a saved Instagram STORY, which requires viewers to be logged in.

If it was a regular Instagram post of a video, people could view it without logging in (just tested in an Incognito window).

But yeah, probably better to self-host, or at least put it on YouTube.

Peripheral question. The app appears to be iPhone only.

I'm honestly curious why people do this, and I've never gotten a satisfactory answer. iPhone market share is 15% at best while Android is 75%. It seems like releasing an iPhone-only first iteration kneecaps the enterprise out of the gate. What am I not understanding?

Like many things it’s a combination of many answers, but from my own experience it’s a simpler pipeline with Apple phones. On Android it’s supporting tons of screen sizes, out of date versions, different vendor modifications to the OS.

Looking at it as iOS vs Android isn’t correct because Android is a fragmented mess.

iOS has more market share in the US than Android. Regarding paid apps, iOS has higher app store revenue than Android overall.

Technical aspects aside (no OS variance, etc), iPhone apps make more money than Android (2x or more, last I checked, despite being a smaller share of total phones). So targeting iPhone first is a common strategy

Android app is under development right now.

You are overthinking it.

This looks like something that evolved from a side/hobby project, so it must've been made for whatever devices that devs themselves use. That's it.

Not signing up for Instagram to 'see how it works'. I'm kind of turned off by the whole thing now.

Edit: Downvote all you want, throwing details behind a paywall isn't helpful. I can only assume that the app requires me to login with Facebook or some other nonsense. If you can't share the basics without a login to another service then maybe you are not legit, or maybe you just don't look like you know what you are doing.

Reply on news.ycombinator.com