Neothesia: GPU-accelerated open-source MIDI visualization game | Hacker News Reader

Neothesia: GPU-accelerated open-source MIDI visualization game

github.com

77 pointspizza16 comments

Hi! I'm the author of Neothesia, I was trying to figure out the sudden source of traffic on the repo and I ended up here, so thank you for posting this!

Since I'm already here, I may as well answer your questions if you have any.

I know it's far from the scope of the purpose of being a plain visualizer/game, but I would love to suggest - and I guess I could try to learn how to do it myself I suppose, but I don't know rust just yet :^) - what about adding a plugin system/interface to an online repository of MIDI files to this?

Would be awesome to have a centralized frictionless way to share/grab midi files. And thank you for all your hard work!

I'm thinking it would be awesome to have a general-purpose plugin system - eg if someone else wrote a 'tutor' plugin, or 'multi-user' plugin, etc. Plus, I'm just spitballing here, but it might be possible to write it so that any language that compiles down to .wasm would be allowed, right? Huge flexibility there. OOooh and maybe a package manager too, while I'm just listing pie-in-the-sky desired features, lol

As long as there is a nice rest API for midi archive I would be open to add an in-app midi browser, but I don't want to maintain and pay for hosting of such an archive myself.

This is planned for Parture if you like to help out (Rust+Elm+SkyNet/IPFS): https://parture.org

amazing!!!!!!! thank you for the link

Looks amazing, thank you for doing this. I use Synthesia for a few years now and it was also quite helpful teaching piano to my daughter. She takes piano lessons but Synthesia is how she learns the songs she likes by her own. If there will be an open-source variant we can be sure that we can revisit and relearn our favourite songs even in many years, when commercial Synthesia might have long been gone[1]

Also, I think Synthesia made a business out of producing Youtube videos of the Synthesia visualization. If there was a free way to produce this kind of videos many Youtubers would be grateful.

EDIT: Seems I'm wrong with the above paragraph. I thought to remember that you had to pay for the rights to use the visualization on Youtube, but that is not the case. The conditions[3] look quite fair to me.

One small remark regarding your visualization: From the screenshot it looks as if the keyboard shows the black keys centered around the gap between the white keys. This is not how a real keyboard looks like. Better explained than I ever could in a thread[2] from a few weeks ago.

[1] Another software we use is PianoJacq. For learning and practicing it is even better suited than Synthesia.

[2] https://news.ycombinator.com/item?id=29943081

[3] https://www.synthesiagame.com/Video

The main driving force for writing Neothesia was video creation, in contrast to Synthesia, Neotheisa has a CLI version that outputs video to a file directly. Also, the open source nature of it allows anyone with a little know-how to create some fancy effects particles etc. depending on whatever you like.

I'm aware about wrong key placement, I was doing mathematically correct placement at some point, but at the end decided to just use the simplest possible thing. Don't worry, math correct keys will come back at some point!

I like this method of visualization it is so much simpler than conventional music notation. This is what "player pianos" use, right?

But it makes me wonder could this representation be any good for actual musicians playing from a notation like this? Has anybody tried to play a player-piano-scroll by hand?

> Has anybody tried to play a player-piano-scroll by hand?

https://en.wikipedia.org/wiki/Klavarskribo

Amazing work here!

Currently rocking Aladdin - "A whole new world" with the Super Mario brothers soundfont from Kirb7890.

https://bitmidi.com/a-whole-new-world-theme-from-aladdin-mid

https://musical-artifacts.com/artifacts?formats=sf2&tags=sou...

Oh cool! I'm a big fan and supporter of synthesia but love the idea of a new open source alternative so things can evolve further.

Very cool! How’s your experience been building guis in iced with wgpu? Thinking about trying it out myself

In general iced is awesome, but in my case it's a little bit cumbersome, as I just replaced my own UI with iced, so my whole project is its own thing and iced is just weirdly tack onto it, but if you start from the ground up and write the project with iced in mind you will most likely be happy with it.

As for iced + wgpu combo you also should be happy with it, and I'm making sure that iced is always up to date with the latest wgpu updates, so you don't have to worry that iced will hold you back when updating wgpu.

just tried it on Mac with Bohemian Rhapsody midi, feedback:

- the resolution seems to be broken

- the CPU meter in Activity Monitor shows 100%

- would it be possible to play notes automatically when i scroll into view?

- the UI/UX is not very user-friendly, i suggest adding a "?" menu that shows the shortcuts

other than that it's absolutely incredible!

how portable is this?

could Neothesia be ported to browsers/mobile?

It's using wgpu so it will run on nightly Chrome and Firefox through Webassembly and hopefully anywhere once the WebGPU standard has settled.

Reply on news.ycombinator.com