Moonlight Sonata Visualized (2012) [video] | Hacker News Reader

Moonlight Sonata Visualized (2012) [video]

youtube.com

89 pointsnetgusto30 comments

Offtopic, but this sonata is connected to one of the most witful naming (in my opinion) in IT industry:

The Silverlight clone was named to Moonlight (a Mono project - RIP), which on the surface seems like just a simple association, but Moonlight is a sonata in C#! :)

This is the gold standard as far as I'm concerned:

http://www.musanim.com/ArtOfFugue/

This guy does visualizations of lots of pieces, using a variety of techniques. He did the entire Open Source Art Of The Fugue (Kimiko Ishizaka), and explains his choices of viz tools for each one.

This playlist contains his favorite version of each piece:

https://m.youtube.com/playlist?list=PLtj_HurkS7Zy5ocS2jbmtgk...

What do the highlighted blocks and moving "pistons" within mean (see [1])?

[1] https://www.youtube.com/watch?v=Tybs1T9OLBg&list=PLtj_HurkS7...

That's the fugue ( https://en.wikipedia.org/wiki/Fugue ).

> In music, a fugue (/fjuːɡ/ fewg) is a contrapuntal compositional technique in two or more voices, built on a subject (a musical theme) that is introduced at the beginning in imitation (repetition at different pitches) and which recurs frequently in the course of the composition.

The box (fugue) reoccurs throughout the piece and reoccurs several times in various variations.

Looking at the bottom (zoomed out map area), you can see the fugue as the "bolder" sections of the piece and their shape is clear that it repeats throughout the piece. There are also times (single pistons) where the sequence occurs outside of the fugue proper.

It's explained on his website, though perhaps not super clearly. The composition is a fugue, so it has a subject (main recurring melody). Each occurrence of the subject is represented by a large box on the timeline, and when it plays, a slow column moves along the box.

Also notice the many countersubjects (secondary recurring melodies). They are made clearly visible in the video: wherever they appear, their notes are highlighted and connected so that each occurrence looks like a small, recognizable constellation.

It might have been interesting to treat all subject and countersubjects equally with the "constellation" style (which I find clearer than the weird box-and-column), then try representing harmonic aspects of the composition, which are so far completely left out of the visualization. Not that it would be an easy task, of course...

This is a very cool take on the piano roll visualization. I’d love to see something similar that focuses on harmonic relationships rather than linear pitch relationships, perhaps using something like the lattice: https://en.wikipedia.org/wiki/Lattice_(music)

The only lattice visualizations I know of are Gary Garrett’s: http://www.garygarrett.me/

Funny thing, I recently attempted exactly same thing, just as interactive toy.

I had notes arranged in lattice (Tonnetz, harmonic table layout, however you want to call it) with full chromatic range of 3 and half octaves. The lattice responds to touches by playing correct note. The patterns really become obvious on lattice layout. Major and minor chords are visually recognizable and harmonic shifts are just graphical translations due to lattice isomorphic properties.

I extracted notes of Moonlight Sonata, and did simple analysis to determine how likely note M appears after note N (Markov chains). After few notes are played on lattice, the Moonlight Sonata analysis is used to suggest (highlight) some choices that could follow played notes.

If you follow these suggestions, the Moonlight Sonata 'feel' is easily recognizable to listener. Player still has full control of tempo, composition structure and embellishments. It was meant to be an experiment into interactive music, sort of auto-complete embedded in musical instrument.

Unfortunately the highlighting worked only in original key of composition (C#). When I tried to analyze intervals instead of absolute pitches, the suggestions became erratic and no longer musical. Also I found it hard to analyze and visualize polyphony. I might revisit this project and make it work in the future.

I've done a few experiments in visualizing pitch-class relationships.

The one I use regularly is to map the color wheel to the circle of fifths (for a fuller description, see http://www.musanim.com/HarmonicColoring/ ).

y MIDI Player has a hexagonal pitch grid mode (for a list of features, see http://www.musanim.com/Player/ ).

I've done some experiments in showing interval classes (to highlight differences between consonance and dissonance), but the techniques don't work well for most pieces, and this is the only sample I've published: https://www.youtube.com/watch?v=7Xu2cSEko6M

When I learned more about auditory neurology, I did some experiments with that (see this page in my project timeline http://www.musanim.com/mam/hist33.html ).

But mostly, I don't think any of these approaches (or anybody else's, for that matter), really do the job.

This is really cool to see:

I’ve always wondered if such a visualization can help folks learn music better and more intuitively by stimulating their visual cortex.

The base notes are the darker notes. The red notes are the treble. The ring shows the notes from low to high (left side to right.) The lines look to indicate the sustain.

Chords are easily shown when multiple notes are hit at the same time. It also looks like the measure shows up as an outer ring.

Also what’s cool is how future notes/chords are illustrated by the up and coming items from the middle.

After watching a handful of these videos I came back to say that I actually feel like I understand music a little better than before.

They are very cool - I have just spent the best part of an hour watching / listening to several of them.

Now I am wondering if there is some way to map the consonance / dissonance / diatonic / chromatic structure of a piece of music visually, in addition to (or instead of) pitch. I know next to nothing about music, so I cannot begin to imagine how one might do that.

Maybe, but I don't think this visualization in particular is well suited to that - it's too hard to keep everything in the field of view, and it doesn't draw enough attention to the more subtle shifts in the right hand.

Tieing the shifts in the patterns in the right hand more clearly visually to when a new chord is struck in the left I think would make it a lot easier to see exactly what's going on.

I think also making it clearer how the fingers move up/down would make it visually more obvious...

It's a very interesting piece to visualize given how much it utilizes repetition with slight variations.

I'd also like to see visualizations which aren't based on the standard linear mapping of increasing semitones. For example, following the circle of fifths instead, and/or contextualizing to the subset of notes used in the song's musical key. Making the root note visually more dominant and the others in proportion according to mode, etc.

I don't mean to take anything away from this beautiful visualization, just something which I think would take it to the next level.

As baddox mentioned - gary garrett does this! eg https://www.youtube.com/watch?v=jA1C9VFqJKo

I have been wondering for a while if it's possible to show both harmonic and melodic relationships at the same time - garett's visualization shows the b7 to 1 movement as a large harmonic leap, but hides the fact that it's a small melodic movement

I understood more about music watching this than in 2 years of (casual) musical training

Check out the smalin versions of the Open Source Art of the Fugue:

http://www.musanim.com/ArtOfFugue/

He explains his choice of visualizations for each piece... All based on a world class recording.

Here's the playlist of his favorites.

https://m.youtube.com/watch?list=PLtj_HurkS7Zy5ocS2jbmtgk-mu...

My favorite is the Canon in reverse motion, where it's laid out to show you how Bach uses mirror versions of each melody.

https://m.youtube.com/watch?list=PLtj_HurkS7Zy5ocS2jbmtgk-mu...

Or maybe it's the final Fugue on three subjects, done to show you where Bach stopped composing and how ishizaka completed the piece from elements of the entire work.

https://m.youtube.com/watch?list=PLtj_HurkS7Zy5ocS2jbmtgk-mu...

Anyway lots to explore here.

I came here to say the same thing. Moonlight has a complex harmonic structure which is completely hidden by the repetitive arpeggio and tiny interval changes. Seeing it on the circle of fifths would make it much clearer.

While listening, it occurred to me that Beethoven probably did not use a piano tuned to equal temperament. I wonder what temperament he would have used.

Here's a video that uses a circle-of-fifths display: https://www.youtube.com/watch?v=-lNhkEgdrmo

I've often enjoyed smalin's visualizations, which I find a little easier to follow visually than andy filebrown's. E.g. https://www.youtube.com/watch?v=LlvUepMa31o

Seconded! I remember being wowed by his Debussy Arabesque visualization many years ago. Still worth a watch/listen! https://youtu.be/A6s49OKp6aE

Moonlight Sonata has a special place in my heart. Early 2000s I was listening to an Internet radio called Gotham Radio - The Dark Side of Metal, it kind of matched my music interests. And then Sirenia - Seven sirens and a silver tear comes up and it's totally not metal and it's the best music I have ever heard. And I have been looking for similar music ever since, more than fifteen years now. And then Reddit tells me all of them are derivatives, sort of, of Midnight Sonata and they are right. Mind blown!

If you are curious, the collection is at https://www.youtube.com/playlist?list=PLvL3SharHGKut2spUcOB0...

I like the infinite space view. Continuing with the theme of the post: this is a clever music video visualization for J.S. Bach Cello Suite No. 1 - Prelude:

> https://vimeo.com/31179423

As visualizations these are pretty, and relaxing. They're less informative than piano-roll displays. The space taken up by the '3rd dimension' pretty much limits them to solo instrumentals.

This (highly stylized) rendering of the opening of Bach's Matthew Passion has helped me appreciate the structure of this extraordinary piece of music: https://www.youtube.com/watch?v=AAafyK44fCc

andy fillebrown's visualizations are so captivating, his video of Bach's Toccata and Fugue in D minor is particularly excellent: https://youtu.be/f5vRrt0Q9Ew

Very interesting. Did they publish the code that takes MIDI and turns it into such visualization, or it's hand crafted?

Sweet! They should do the third movement!

The whole piece's playlist:

https://www.youtube.com/playlist?list=PL9CSjAx04SPvTyIBZpE2S...

Awesome! Always get bored by the first movement, that one is so much better!

idea: wearable music: bracelet (refer to visualization in parent link) for deaf people to listen through skin interaction

Reply on news.ycombinator.com