Blob Opera | Hacker News Reader

Blob Opera

artsandculture.google.com

456 pointsBerislavLopac59 comments

There's so much great stuff here:

- The state machine that seems to handle all edge cases, both in things like the tutorial and also in the procedural singing

- The animation, both the rigging (maybe these are bones?) and the spring physics. They blink randomly. Their eyes follow the mouse. Matching the mouths to the vowels and consonants is really well done

- The overall look and feel, including shaders, shadows, DOF, style, UI overlays

- The synthesizing of a human voice with vowels and consonants in a fairly convincing way. Choir synths for music production can run for thousands of dollars

- The real time harmonizing in a seamless way.

- And on top of all that, it works on all device sizes. It actually runs better on my phone than my laptop, probably due to large monitor size (the bane of any WebGL project).

Fantastic

Yep. I dabble in orchestral music, and was floored that this thing had simple articulations. Entry-level chorale libraries sometimes only have bare vowels, and this thing just... works.

And it works perfectly on the phone too.

Even the small things, like the eyes going into a hypnotic trance when the blob is keenly following the blob to their right (our left).

> Choir synths for music production can run for thousands of dollars

Do you have an example? Why are they so expensive, are there no community projects in existence?

I don't know of any "choir synths" that run to thousands of dollars, but there are plenty of sample libraries that cost hundreds of dollars.

I'd guess this is procedurally generated. Different tech.

The graphics are delightful, but the vocals are actually quite crude compared to what's possible, even without any ML.

Two technologies called FOF and PSOLA have been around for decades, and can sound not unlike this.

With a little mouse wrangling, you can get them to do a fairly good rendition of counterpoint: https://g.co/arts/feoYyFonGVXicfk47

There's a small delay while the other singers catch up to the harmony changes of the lead that makes it sounds more natural, it's a nice a touch.

Someone wrote a tool to control the blobs with MIDI. https://mobile.twitter.com/Overlapping/status/13389792569032...

Nice. I've already tried blob Megalovania :) .

Incidentally, does FF have the "pretty print source file" function somewhere that chrome does? The App.js in this site is minified, making it difficult to set breakpoints.

It would be amazing if someone would create a hosted blob opera midi player.

When you have a JS file open in the debugger tab, there's a `{}` button at the bottom of the file tab that prettifies the file.

Apparently the Blobpera already listens for MIDI input devices!

https://twitter.com/thealexweaver/status/1339535386444521472

This is now published as a standalone module https://news.ycombinator.com/item?id=25523521

This is one of many amazing projects by David Li http://david.li/

When you can pull this off on the web and yet people will still insist on making me download an 100mb app to order pizza.

I submitted this yesterday, and it somehow didn’t stick, but here’s a list of the stuff I managed to do with it so. Far:

https://taoofmac.com/space/links/2020/12/16/0904

How do you get them all to sing at the same time?

You just lead with the leftmost one. Follow the tutorial, it's fun.

The lower voices harmonise with the highest singer by default, unless you explicitly mute them.

It does sometimes get itself into a state where that harmonizing doesn't happen, even with nobody muted. I haven't worked out what the trigger is.

Fun. I wonder how Google decides what projects to choose, the artist behind Blob Opera made something similar back in 2018: https://www.adultswim.com/etcetera/choir/

It's the same artist!

Yeah?..

> the artist behind Blob Opera made

I'd love to see the source code on this. At least part of it is a language compiled to WASM (e.g. https://gacembed.withgoogle.com/blob-opera/harmonizermodule....)

My wife was convinced I had suffered some kind of major head injury after I made her watch/listen to this.

I still think it's incredibly cool and cute though.

That's sad :(

this is why I don't show anything like this to mine. But I will show it to my daughter :)

To quote my father, who is a huge opera fan: "I've heard worse!"

Imagine having a company so big you can have an arts and culture division...

The budgets for arts & culture projects are very small, most companies could afford said division

It's very much a hearts and minds project, with stakeholders in engineering, marketing, press and policy. Each dept uses it to aid their local and regional initiatives. For the first few years it had to fight hard for its existence (originally as a 20% project), scraping budgets together from random sources until stakeholders eventually saw the value.

> until stakeholders eventually saw the value

What's the value if it's gone in 2 years? I wish they'd put similar effort into preserving the app/servers (simple life support) instead of shutting them down.

It's been running in one form or another for 11 years now.

Let’s be honest, the arts and culture division is really just a part of marketing. Maybe not in name, or even on the org chart, but in practice.

The automation of opera singers came earlier than expected

Is there a paper or blog post detailing the training process? I could not find any information about that.

This is fantastic. The overall experience is magnificantly done. As a Frontend Developer i am really impressed by the sharpness of the UI and the funcionality the interface makes possible. Does anybody know what the Techstack is or where it is posted how the built this?

Why does need a techstack? It just looks like custom WebGL and custom Web Audio.

Launch one of the Christmas songs, let it play for a few bars, then grab one of the blobs and sing a solo. Amazing!

I enjoyed this, but couldn't get the address bar to disappear on Firefox mobile. Perhaps that's ironic, but I feel like the devs of this have done a great job, and shouldn't be worked about multi browser support.

You can "install" the PWA to launcher homepage. If it's accessed via the homepage shortcut, the address bar would be hidden.

Has anyone reverse engineered the binary format which represents a song?

If anyone else has troubles loading this, It didn't work when I was logged in with my G Suite account. Using an incognito tab works fine though.

Are there machine learning parts here the actual voice generation? Writing the vocal parts, and that note interface don't require ML AFAICT.

Making Blob Opera with David Li: https://www.youtube.com/watch?v=ZfLYuXi6sDI

Reminds of pink trombone

Link: https://dood.al/pinktrombone/

Reminds me of the silicon vocal cords robot "motormouth robot"

https://www.youtube.com/watch?v=Bht96voReEo

Is there any way to get the blobs to sing staccato?

Sort of, if you do it for one at once. The UX guides you the other way.

Really amazing thing and at the same time shame for not embracing this in open source way.

I don't get it! Can someone explain why/how this is special? Seems like it is only a matrix of like 40 states per blob and that the lower blobs just follows with the same state? Of course well executed but..?

It's cutting edge A.I implemented in a fun, geeky way. It just works effortlessly. What's not to like?!

The lower blobs are creating harmonies based on your blob's state. The fact that it produces results that are both musical and interesting is very impressive to me.

I would like to know what the machine learning part of this is. Not knowing any different, and seeing that you can only drag one character at a time, it seems like it could be entirely done with relatively simple harmony rules.

ML or not I still think it's super cool but it would be great to know what's the ML part.

After experimenting a bit, the harmony appears not to be a simple function of the note you play. Perhaps it's taking into account previous notes, and putting harmony in the context of a 'piece'? (that would be cool) Or maybe it's more simply random.

The voices are ML generated based on hours of recordings of real opera singers. It seems that the real time synthesis includes details like realistic switching between vowels, realistic trills on long notes, etc. There's a lot going on to make those voices sound natural.

Addicted in a heartbeat, greatly executed. I'll use before Google decides to remove it, or shutdown the entire Art and Culture department.

Reply on news.ycombinator.com