Hacker News Reader

Comment by heyitsguay | Hacker News Reader

Amazing! I do work in 3D computer vision and I've really wanted to get into 3D generative asset design as a side project at least. If the author or anyone else is interested in collaborating, I'd be down to talk and shop something.

View on HN

Sure! I've been wanting to incorporate this exact thing for a "And let there be light simulator!"

Essentially, people in VR would be able to point and speak to create objects in a shared space.

Cool! I've got some Unity VR experience, though i don't know much about incorporating ML pipelines into it. I'm guessing there's resources out there? So basically you'd want speech -> text -> object?

Exactly!

I've got some of this working in Python, outside of Unity. Last time I looked at integrating with Unity there seemed to be some Google API requirements that were hard/annoying to meet in my few hours of digging. Should take another look. All the VR HMDs have mics.

This would be really incredible! Could maybe be a way to quickly make video game assets for testing and such. Although to make the meshes performant in a game engine you'd need to decimate them quite a bit as the marching cubes I used here produces really dense meshes.

Do you think the resulting meshes are amenable to auto-decimation tools? Also (directed to the previous poster), while VR applications sound cool, i must admit my interest really lies in improving the asset generation tools themselves. The work done in the original post is super inspiring, I'd love to see a more robust asset generation pipeline built up from tools like that.

Oh wow, count me in. You could use Google Cloud Speech to go from voice -> text, and then Google Cloud Natural Language to go from text -> simple command interface.

I've got both working in Python, but don't do anything with the parsed text.

I want to do things like "create sphere" "move that up" "no, bigger"

etc.

Reply on news.ycombinator.com