The woman who could 'draw' music | Hacker News Reader

The woman who could 'draw' music

bbc.com

134 pointshappy-go-lucky21 comments

Oh wow. I am days away from finishing a PhD thesis on sketching sound and music [1], and Daphne Oram figures highly into my background research. A friend of mine built an iPhone app that emulates Oram's system [2].

Another notable figure in the world of drawing music is Norman McLaren, who literally painted or drew directly on the soundtrack portion of his animated films. The soundtrack for his film Neighbors [3], which won an academy award in 1952, was produced this way. Theres a short documentary on his process here [4].

On the more avant-garde side, Iannis Xenakis had designed a computer-based system called the UPIC in the late 1970s in which you could directly draw waveforms and then direct their frequencies over time by drawing into a graphics tablet. He used this to compose his piece Mycenae Alpha [5].

This page [6] has a really great run-down of optical synthesis in general, which includes a number of individuals and systems involved in directly drawing sound and music. Some of the visual sound designs used in these systems are quite striking, for instance the variophone [7] or Yankovsky's painted soundtracks [8].

[1] Demo of my research software here: https://www.youtube.com/watch?v=Tdj5e82nPHQ

[2] https://itunes.apple.com/us/app/oramics/id454505541?mt=8

[3] https://www.youtube.com/watch?v=P-o9dYwro_Q

[4] https://www.youtube.com/watch?v=Q0vgZv_JWfM

[5] https://www.youtube.com/watch?v=yztoaNakKok

[6] http://www.umatic.nl/tonewheels_historical.html

[7] http://www.umatic.nl/tonewheels/historical/vario3.jpg

[8] http://www.umatic.nl/tonewheels/historical/painted_soundtrac...

Neat links, thanks for the references. Would be very interested in knowing more about your thesis when its something you're able to share ..

One wonders what Daphne would have thought of tools such as yours, and as well of course things like U&I's MetaSynth:

http://www.uisoftware.com/MetaSynth/

For sure. Its a long-ish video, but I defended a few months ago. The recording is online, and hits all of the major points: https://www.youtube.com/watch?v=B4xZQxVVnJA

Maybe Ill post a Show HN here when I publicly release the app over the summer.

This is a very interesting way to draw music, lately i have done some experiments with something akin to the Russian ANS synthesizer which was a photoelectronic synthesizer where the sound spectrum was sketched onto glass disks, in my program all is driven by visuals produced by a GPU fragment shader, one can use a webcam as a texture to replicate something akin to the ANS workflow where you draw the sound spectrum although it is much less limited, one could also capture a painting software and feed their drawing in realtime.

The synthesizer is available here : https://www.fsynth.com

Whoa, very cool. I like the integrated live coding aspect.

I have a hard time visualizing FM algorithms and Auraglyph looks like it would be really useful. When will it be available?

Awesome. Releasing it publicly later this summer!

Thanks for the links, your PhD sounds promising.

Not sure which machine came first, but around the time of the UPIC system you mentioned the Fairlight CMI was released, which (amongst its other functions) allowed users to draw out waveforms using a lightpen:

https://en.wikipedia.org/wiki/Fairlight_CMI#CMI_Series_I

This video shows Peter Gabriel playing around with a Fairlight CMI, from what I remember I don't think it shows the waveform drawing but it gives some idea of what it was capable of:

https://www.youtube.com/watch?v=ON8lVgJxMQA

In the McLaren video you linked, the uploader edited out his music and put in some other track. Here's the link to his film with the original score:

https://www.youtube.com/watch?v=e_aSowDUUaY

Thanks, good catch- its really worth watching the whole thing.

Love your work, Spencer. One of these days I'll get around to forking/pushing changes I've made to Chuck to support DirectX 8 and later while still allowing Chuck to build with the older DX5....

Thank you, awesome, please do! Though hopefully some day ChucK will have proper WASAPI and ASIO.

Your app looks amazing. I love how you can leave handwritten notes next to the audio control nodes. I know these kind of hybrid sketching/programming environments have been a HCI dream for a long time now but I haven't seen it look as good as in your demo.

Very interesting list of links. This is such a good topic for popularization because of its visual and concrete aspects. (e.g. Papert and Turkle's idea of the "revaluation of the concrete" http://www.papert.org/articles/EpistemologicalPluralism.html ). (Drawing is always concrete, I guess, not just because it's a hand-eye coordination thing, but because of its particularity.)

FWIW The most general wikipedia article and category that comes close to covering what you describe as "optical synthesis" is

https://en.wikipedia.org/wiki/Graphical_sound https://en.wikipedia.org/wiki/Category:Graphical_sound

The page emphasises the optical film soundtrack (on celluloid) as a medium for experimentation, which is something I'm fascinated by.

The gender aspect is interesting too, maybe related to ideas like Papert and Turkle's. There are at least two female film artists who have done interesting critical work with the optical film track:

Ute Aurand http://www.uteaurand.de/filme/paul_celan_liest.php

Lis Rhodes https://lux.org.uk/work/dresden-dynamo

also, in a different way,

Aura Satz https://www.thewire.co.uk/video/p=15036

whose work has, AFAIK, included providing replica tapes for the Oramics machine when it was displayed at the Science Museum.

There's also of course the very famous and female Delia Derbyshire, working in a similar milieu to Oram, but without pursuing the graphical aspect.

I'm sure you're aware of this, but for the record, there's a physical archive of Oram material at Goldsmiths in London: http://www.gold.ac.uk/ems/oram/ The man in charge is Mick Grierson, an old Dorkbot London hand.

Your software looks excellent. Very intuitive version of what others try to do textually e.g. http://functional-art.org/2013/hudak.pdf

On slide 16 of that set Hudak uses the dreaded Haskell monads (well, arrows) to compose a synthesis setup (a model of a flute), but it's an excellent example of something that could be done so much less opaquely in a graphical environment like yours)

Awesome, really interesting thoughts here and a lot of material for future study. Ive not been to the archive at Goldsmiths, though I once gave Mick a haircut as "performance art" :)

As adults we get locked into modal thinking - breaking those modes is not something we (usually) do naturally.

As children those modes aren't so deeply grooved so coming up with arbitrary models of the world makes for a lot of eye-opening exploratory activity.

I recently went to an exhibit on the Reggio Emilia approach (famous for pioneering this method 70 years ago) and one of the videos on display was an activity involving the children drawing sound [1] - I hadn't thought to look at the world this way but for these kids (and a bit of guided prompting) it came very naturally. Things like painting the sound of water rushing over stones[2] - at first glance don't parse - but if you tilt your head a bit you can 'see' it.

[1] http://www.acousticecologyaustralia.org/symposium2003/procee... [2] http://imgur.com/a/RAOmy

Previous Daphne Oram submissions:

https://hn.algolia.com/?query=daphne%20oram&sort=byDate&date...

There was a year long exhibition of her Oram synth machine in 2013. Here's a good mini-doc on it and Daphne:

https://www.youtube.com/watch?v=mJ08diPUv6A

Not entirely unknown fortunately.

She seems like a kindred spirit with Kandinsky. Both tried to connect the audio and the visual arts, and both were mystical about the fundamental humanness of their respective art forms.

Don't forget the man who could 'see' music, by looking at grooves in a vinyl record: http://www.tonedeaf.com.au/362661/vinyl-vision-meet-the-man-...

The woman who 'painted' music:

"Unraveling Bolero" by Anne Adams - http://imgur.com/a/wkwKZ (photograph of work)

---

> "In this podcast, a story about obsession, creativity, and a strange symmetry between a biologist and a composer that revolves around one famously repetitive piece of music.

> "Anne Adams was a brilliant biologist. But when her son Alex was in a bad car accident, she decided to stay home to help him recover. And then, rather suddenly, she decided to quit science altogether and become a full-time artist. After that, her husband Robert Adams tells us, she just painted and painted and painted. First houses and buildings, then a series of paintings involving strawberries, and then ... "Bolero."

> At some point, Anne became obsessed with Maurice Ravel's famous composition and decided to put an elaborate visual rendition of the song to canvas. She called it "Unraveling Bolero." But at the time, she had no idea that both she and Ravel would themselves unravel shortly after their experiences with this odd piece of music."

Radiolab - Unraveling Bolero (S14E01) http://www.radiolab.org/story/217340-unraveling-bolero/

---

> "Most neurological lesion studies emphasize performance deficits that result from focal brain injury. Here, we describe striking gains of function in a patient with primary progressive aphasia, a degenerative disease of the human language network. During the decade before her language deficits arose, Anne Adams (AA), a lifelong scientist, developed an intense drive to produce visual art. Paintings from AA's artistic peak revealed her capacity to create expressive transmodal art, such as renderings of music in paint, which may have reflected an increased subjective relatedness among internal perceptual and conceptual images. AA became fascinated with Maurice Ravel, the French composer who also suffered from a progressive aphasia, and painted his best-known work, ‘Boléro’, by translating its musical elements into visual form. Later paintings, achieved when AA was nearly mute, moved towards increasing photographic realism, perhaps because visual representations came to dominate AA's mental landscape during this phase of her illness. Neuroimaging analyses revealed that, despite severe degeneration of left inferior frontal-insular, temporal and striatal regions, AA showed increased grey matter volume and hyperperfusion in right posterior neocortical areas implicated in heteromodal and polysensory integration. The findings suggest that structural and functional enhancements in non-dominant posterior neocortex may give rise to specific forms of visual creativity that can be liberated by dominant inferior frontal cortex injury."

Seeley, W. W. et al. Unravelling Boléro: progressive aphasia, transmodal creativity and the right posterior neocortex. Brain 131, 39–49 (2007). https://doi.org/10.1093/brain/awm270

(note- edited for formatting)

Reply on news.ycombinator.com