Everything we can't describe in music
hazlitt.net
hazlitt.net
In the mid-to-late 1800s, Herman Helmholtz was building spectrographs of musical sounds by listening to them through a bunch of resonators of varying sizes that acted as band-pass filters, and recording how loud each harmonic sounded.
His writings are still relevant today.
https://www.amazon.com/Sensations-Tone-Dover-Books-Music/dp/...
I've heard that the Sethares book (which is much more recent) is really good too, but I don't have a copy of that one.
https://www.amazon.com/Tuning-Timbre-Spectrum-William-Sethar...
> Timbre is even more crucial in contemporary genres such as electronic dance music and rap. EDM, said Swoger-Ruston, is really “experimentation with timbre.” The widespread use of sampling means the sample, not the note, is increasingly the basic unit of music.
I didn't assert this.
That's a pretty harsh objective take for a highly subjective topic.
A lot of people like it - myself included. Just because you don't like it doesn't mean it's not valid.
EDM at least has to be considered, and likely, no matter how crap it is, someone put some sort of effort into it. Speaking into a pitch bender takes nothing but breath, and I'm stuck listening to this garbage in traffic every day.
Also moving back to one of the most well known autotune users T-Pain is a great singer without the technology. It's obvious there's more to it then just auto-aim function.
Some people use it that way, but it doesn't mean the technology is fundamentally limited to that scope.
But that aside, this take reeks of sampling bias. Every amateur with a laptop uploads their music to SoundCloud/Youtube/whatever even while they are still learning. You probably don’t have much exposure to the crappy piano fugues and poorly orchestrated droning sonatas that are written by early university students. Trust me there’s a lot of that as well. Everyone starts somewhere after all.
"Helmholtz concluded that the spectral description of a sound had the most straitforward correlation with its timbre.... For instance the qualitative description of 'brilliant' or 'bright' characterizes spectra that have a great deal of energy at high frequencies [e.g. brass instruments] ... A spectrum with little energy in the even-numbered harmonics characterizes the particular timbre that is produced by the clarinet in its low register...." (2nd Ed. Section 2.6)
Understanding that each instrument produces a spectrum of harmonics peculiar to it ... and that those harmonics have relatively different pitches and amplitudes and evolutions ... that vary with how the instrument is played ... is key to quantifying its timbre.
(At first, not an easy thing to grasp let alone quantify. But Partch made a lot of progress!)
https://commons.wikimedia.org/wiki/File:Timbre_analyser-CNAM...
In most genres you have a palette of timbres to work with for each instrument based on the genre conventions. You can push at the edges or add things in, but have to balance audience expectations carefully as you do it. So for example the way an electric bass sounds in contemporary metal is just not normally going to "fit" into a chicago blues band, even though both are heavily dependent on the sound of electric bass.
Different genres have different relationships to this constraint, for example western classical has a huge palette available in an orchestra but is relatively averse to using instrumentation outside of that collection. Except in percussion, where there is a lot more flexibility! Jazz has a fairly small & rigid set of acceptable timbres for its core instruments, but is fairly tolerant of experimenting with new instrumentation.
So then the two genre-timbre relationships I find most interesting are electronic and pop. Pop is, more than any other genre I think, curious about how timbre effects emotional response in music. An album by the same artist could have a huge range of different sounds for recognizable instruments, using the tension between them for different effects.
And then large swaths electronic music are built largely around active change of timbre over time through a piece. Something you see used conservatively and carefully in most genres becomes almost the central practice.
IDK it's hard to articulate and I'm not trying to set a reductionist framework about how music sounds or anything. Just a line of musical thought I've been exposed to and found valuable.
And that's why I love folk-ish metal so much.
Why shouldn't we have a lead/harmony violin on top of a metal rhythm section? https://www.youtube.com/watch?v=F17sxgZdXZw
Or a hurdy gurdy? https://www.youtube.com/watch?v=LT7HqIP55tI
Or a pipa, which was btw very hilarious in poland: https://www.youtube.com/watch?v=MwpKZtPBIEQ
I'm honestly thinking of dropping some money to get myself an A-tuned drone flute[0]. Drone flutes sound really cool, and the drone would be tuned to just about the highest two notes on the guitar... and if you work with it a bit, drums/bass/rhythm guitars helping with rhythms, chords and harmony could really push that somewhere really interesting.
Was brilliant, a lot of groups focusing on the use of ML to characterise the "unexplainable" in sound synthesis.
We ended up submitting a tool for interacting directly with Ableton using LLM agents after becoming disenchanted with text2audio models, wrote about it here - https://montyanderson.net/writing/synthesis
I have been out of it so long muff wiggler is now mod wiggler. Come on, that is absurd. Muff Wiggler was the best name ever.
There's a neat documentary floating around which includes some scenes where he explains how he increased his understanding of waveforms in the frequency domain by physcially modeling them in the time domain.
Different harmonics have different ADSR curves (Attack, Decay, Sustain, Release).
Above all, one cannot overstress the importance of the attack transient. There are famous experiments in psychoacoustics that show that, when deprived of their attack transient, the sounds of two different instruments may become hard to tell apart.
Personal anecdote: as a classic guitarist, it took me three years of experimentation to find the right way to cut my fingernails in order to have a better sound. The Electronic Engineer in me says that those were three years spent to look for how to improve 0.1 seconds of noise at the start of each of my notes.
A neat trick is playing guitar through a volume pedal. You mute the sound, pluck a string and then swell in the volume pedal. The rest of the ASDR envelope can be created with other effects like delays.
"Eddie did two takes, and the volume knob froze completely at the end of the second take due to the heat generated from rolling it on and off."
https://www.vhnd.com/diver-down/cathedral/
His signature Peavy Wolfgang guitars later featured low-friction pots that are still available for sale as individual components
Roland used this fact to produce surprisingly realistic sounds at low cost. When memory was too expensive to sample the whole note, you could get excellent results from sampling just the transient and using traditional subtractive synthesis for the rest:
Classical composers had a set of fixed timbres to play with, each instrument having their own. With synthesizers everything is possible.
More to the point, the synthesizer has more options for expressiveness through change in timbre, glissando, microtones, etc.
Bass is easy to obtain out of a synthesizer: just make sure you have a subwoofer in the speaker system. It doesn't have to be a True Scotsman's synth or subwoofer.
https://www.youtube.com/watch?v=4R8RaCLzGgE
https://www.youtube.com/watch?v=iWgXVbI5ZRw
https://www.youtube.com/watch?v=pjIzyeQrG7U
https://www.youtube.com/watch?v=PEDhoRruflk
There is, however, some real progress being made. For controllers, check out the Linnstrument, the Osmose, the Continuum. MPE was added to the MIDI spec a few years back, which makes it a lot more feasible to sell expressive instruments and have them "just work" with existing synths. As for synthesizers, just about anything is possible. Physical modelling is becoming more popular.
A keyboard with polyphonic aftertouch, such as the Yamaha CS-80 (famously played by Vangelis), is objectively more expressive than a piano.
This is just not true at all. The action of a piano is a very sophisticated mechanism that conveys both acceleration and velocity, along with the initial position of the key (e.g. fully released or half-pressed). This allows for a huge amount of tonal variation, though it's more subtle than a ribbon controller, and obviously a synth can do many things than a piano cannot.
https://en.wikipedia.org/wiki/Action_(piano)#/media/File:For...
All the complexity you mention only affects how the repetition lever (part 9) briefly bumps against the hammer shank (part 8). The only information this imparts to the hammer is velocity, and once it happens there's no more contact until the hammer falls back down. The hammer can't somehow remember what caused it to travel at a specific velocity and strike the string differently. There's only a single degree of freedom in its movement. What you perceive as "a huge amount of tonal variation" is mostly note timing.
It's entirely true for the attack, because of the escapement mechanism. At the point at which the hammer makes contact with the string, the hammer is mechanically disconnected from the key and moving purely through inertia. Once you've tripped the escapement, the hammer is beyond your control and pure physics takes over; the only variable is the velocity of the hammer when the escapement is tripped.
The release phase is controlled by a single parameter, albeit one that can vary over time. The key is holding up the damper, which returns to the string under gravity. A continuous value representing the range of motion of the key would be sufficient to model this with absolute accuracy. Digital pianos invariably sense release velocity rather than key position, because it's much cheaper to implement and is perfectly sufficient in virtually all cases; the range of motion between partial and full damping is so limited as to provide only a very limited degree of expressive control.
Obviously some of their sound is tied to specific guitar models, but their tone is still easily identified on other guitars. And although these players were creative improvisers, that's not what I mean. Simply put: if it's a great guitarist, you can hear their hands like you can hear a singer's mouth. (Though hands are clumsy compared to mouths, and guitars are clumsy compared to horns.)
I am not anti-synth or anti-electronic music. But I do not believe any existing synth controller provides the expressive possibilities of an electric guitar, even Allan Holdsworth's goofy breath controller. Maybe I'm wrong and there's a great example out there on YouTube. But it seems to me that for an individual to express their individuality with a synth, they need to play a longer musical fragment and color their expression with timing, volume, vibrato, etc. Three notes "with pizazz" will not really be enough.
Vangelis and the CS-80 is maybe an exception, though even if his songs are very recognizable, probably a lot of people could convincingly imitate him.
Maybe Wendy Carlos could count too.
"Everything is possible" is a bug and not a feature in many ways.
It certainly is possible to synthesize an expressive instrument to the level you describe but what usually happens is you move on to something else.
"What does this knob do??" Wahahahaha
That kind of thing is just so much fun compared the guitar that you have to actually learn to play.
Generating that sound in a live performance that is partially or fully improvised is a harder problem.
The evolution of solutions to either problem depends on which instrument you're interested in, and is based partially on Means, Motive, and Opportunity. There's a reason to synthesize a piano or Hammond organ, which is that they're hard to transport and maintain. Less motive for synthesizing a violin.
Vocal timbre is a big identifier of different rap subgenres and artists, seems like. Certainly seems like much of the vocal delivery on a track explores fluctuation of the timbre, even when the pitch is constant or has little dynamics.
It's interesting to think how the timbral effects of a regional accent would interact with the beat construction/development. Is the trap hi-hat a result of the Southern drawl?
I would go so far as to say understanding and selection of timbre is 50% of the work of a synth player.
Violin and the bow are both made of wood and a rainy damp day can easily sound different than dry summer's day.
You can get good carbon fiber bows and they are much less weather sensitive.
You can also get carbon fiber violins but I haven't heard one that sounds as good as one made from wood.
> The utility of the piano for composing was its neutral and uniform tone color: in theory (although not in reality) the tone quality of the bass is the same as the treble. In any case, the change in tone color over the whole range of the piano is, or should be, gradual and continuous (there are breaks, of course, when the notes go from one string in the bass to two and then to three in the treble). The monochrome piano might be used therefore just for its arrangements of pitches, and the quality of the sound could-absurdly in many cases-be considered secondary.
> What made it possible for composers to refuse to acknowledge the difference on the piano between treble and bass and leave whatever problems arose to be solved by the performer was the fact that the change in tone color over the span of the keyboard is not like the leap from a bassoon to a flute but continuous and very gradual when the instrument is properly voiced. These imperceptible gradations are the result of a deliberate policy of a unified sonority on the part of musicians and instrument makers. All attempts over the history of piano construction to incorporate anything analogous to the picturesque changes of registration in the organ and the harpsichord had little success, were not exploited by composers, and were finally abandoned. Radical contrasts of tone color were traded for the possibility of making a gradual crescendo or diminuendo. This was a decision that took place at the same time as the preeminence accorded to the string quartet over all other forms of chamber music; that, too, emphasized the importance of a unified tone color. Chamber music with wind instruments, while the occasion for several masterpieces, became the exception, an exotic form. That is why the use of colorful sonorities in the orchestra has so often been considered somewhat vulgar, as if calling attention to the sound were paradoxically to detract from the music. The prestige given to pure string sonority is part of the asceticism of nineteenth-century high culture. Contrasts of tone color were given a significantly lower place in the hierarchy of musical elements. This is one reason that only the piano repertory rivals the string quartet as the most respectable medium for private and semiprivate music-making from Haydn to Brahms.
"I think the tone becomes perceptible by virtue of tone color [timbre], of which one dimension is pitch. ... Pitch is nothing else but tone color measured in one direction. Now, if it is possible to create patterns out of tone colors that are differentiated according to pitch, patterns we call 'melodies', progressions, whose coherence evokes an effect analogous to thought processes, then it must also be possible to make such progressions out of the tone colors of the other dimension, out of that which we call simply 'tone color', progressions whose relations with one another work with a kind of logic entirely equivalent to that logic which satisfies us in the melody of pitches. That has the appearance of a futuristic fantasy and is probably just that. But it is one which, I firmly believe, will be realized.
...
Tone-color melodies! How acute the senses that would be able to perceive them! How high the development of spirit that could find pleasure in such subtle things!"
-- Arnold Schoenberg, Theory of Harmony, 1911, toward the end of chapter 22.
Personally, I anticipate that if humanlike AGI is achieved, then it will understand the illusion of music. Not merely because music understanding is an attribute of humanlike, but because there must be something intrinsic in the process of human cognition that causes the illusion of music occur. Music understanding could be viewed as an essential waypoint on the path to AGI.
Despite the different sensory systems (sound, or visual) and the difference in actual data (I don't know how many photons or audio waves entering your brain), we can abstract them down and understand/relate them with a simple pattern.
Our model of perception is very finely tuned with that of space-time.
Edit0: https://en.m.wikipedia.org/wiki/Talking_drum
It begins by talking about this, maybe that's a good start! It documents that they are able to transmit complex messages like jokes and proverbs entirely through rhythm!
When I get mentally crystallized, I hear music as proof-of-work in the vein of Bitcoin hashes.
Maybe a reason for calling music an illusion is to try to point out that it's all in our heads. Sure, the physical instruments are over there, but they are just some atoms and molecules rattling about creating vibrations that, like everything ultimately, share the meaninglessness of the physical universe of just more material stuff (ignoring the quantum and the unknown). In which case though life seems also to look more like an illusion. Ah but the meaningless vibrations enter our ears and are processed and become music, so like color there is none but in our heads, thus illusion? But these notions also fall apart. The redness of apples isn't all in our heads; it requires a certain relationship to light frequencies that we see as redness. It's not an illusion it's just what red looks like through a living human body. If not through our senses and brain, and then mind, where should we experience it?
Music is only ever meaningless if it is unheard, as apples can only not be red when they are unseen (or are Granny Smiths).
So music is only meaningless if there is nobody -- no body -- to listen. And maybe that's where all of this leads. Your AGI needs to be embodied and alive.
Somehow, the physical situations of our bodies become perceptual experiences. Usually, those perceptions are mere descriptions of the physical world, for physically navigating it. Some exceptions exist, such as music and optical illusions. The translation from air pressure fluctuations to a tree branch falling is one type of experience, the link to love or hope is another. Rather than being insignificant, my sense is that music understanding will prove to be an emergent capability available in whatever architecture might produce humanlike AGI, because it connects human perception and cognition too thoroughly to not be baked-in. Going further, building models to understand music would be an excellent step toward embodied AI with humanlike cognition.
I have what I think some interesting ideas and implementations, but no one quite grasps my thought process and I am stuck between believing Im onto something and that I am furiously chasing nothing.
The closest thing Ive found to my own is Wolframs writings.
Seriously I know I seem like a delusional nut but this train of thought has really grabbed me. Ive written hundreds of pages of (mostly intelligible) notes/figures and spend countless nights trying to implement what I can so clearly see in my head.
Unfortunately I dont have anything concrete yet that would warrant building a team, and its hard to even find people I can discuss my ideas with.
I think there ought to be interest in this area in mathematics, philosophy or even linguists though.
Skeuomorphic design, to me, largely feels lazy and unattractive; rather than designing for the new, maybe unfamiliar, medium you're working in, you attempt some facsimile which inevitably cannot live up to the original, either out of lack of ambition, or lack of faith in your audience to understand or appreciate without the anchor of a common metaphor. It lacks idiomaticity - there are particular details, quirks, associated with different mediums that lend themselves to different sorts of designs, and a skeuomorphic design language ignores these. (The same is true of musical instruments. A piece written for the lute, say, played on piano, would feel very different than a piece written for piano, played on piano; what is easy, or possible, on one instrument, is not necessarily as sonorous on another.)
To me, the use of synthesizers to emulate acoustic instruments, where the express intent of the composer/producer/whatever is to evoke the sound of that acoustic original (read: where the creator would prefer the sound of the real instrument), is, at its most generous, telling of a lack of creativity and at worst laziness. (If what you want is expressly the sound of a synthetic imitation for the purposes of your art, none of this applies.) It is a lack of creativity in the case where the creator simply does not have access to "real" instruments they'd rather use, and it's laziness where you have access and ability but opt for something you yourself deem inferior.
Of course, this all presupposes that there is something lost when say, a synthesizer plays a violin patch in an earnest effort by the creator to emulate a violin - where a real violin is actually what's desired. I think this isn't controversial to say, but of course it's a spectrum; I have much less of a problem in cases where the delta is smaller; e.g., simple legato harmonies from a string section can be emulated more convincingly (by orders of magnitude) than some virtuosic cadenza by a solo instrumentalist.
The shape of an instrument, the material used, the type of strings, their tension, the temperature and humidity in the room. The way each player holds it, the size of their hands, the specific strength of their fingers. Some of these factors stay constant, others change from day to day or player to player. The sum of all of these together creates a specific timbre.
I've met winemakers who think of planting in terms of multigenerational experiments with highly intentional controls for different aspects of soil and micro-climate that may vary on a scale of a single meter, who work to find and train their successors because they will not see the results of some of these experiments in their lifetimes; winemakers who absolutely understand that blending is not just a linear combination of the characteristics of blended wines, but a truly complicated system where certain flavors may heighten others; winemakers who rely on centuries of experience in how funguses and other microorganisms can be used in safe and flavor-enhancing ways.
Winemaking is engineering, just as musical instrument design is (both physical and digital), and just as building a startup is. The presence of human subjectivity as an evaluating oracle in a complex system does not imply that its creators work without intention or without rigorous systems-level thinking.
It's no accident that musicians spend a lot of money on instruments -- even electric guitar players -- an instrument that is electronically amplified and often distorted and otherwise augmented -- will have strong feelings about the particular type of guitar they use. Leaving shape-based decisions out of it (and pickups), hollow bodies are going to produce a different sound than solid. Different wood types will produce different tonal characteristics.
So the things that are directly controlled are less like terroir, more like choice of grape used maybe? While the make of the instrument, the shape of the room a recording was made in, these are much more like terroir I think.
But it's probably just not a great metaphor.
True for acoustic guitars, but for electric guitars this is thoroughly debunked:
https://www.youtube.com/watch?v=n02tImce3AE
Previous HN discussion:
The idea holds across many acoustic instruments though.
A terroir is totally describable, as by definition you have uniform geological and climatic properties (on the other hand if it can't be delimited it's not a terroir)
Anyone who's played Outer Wilds should disagree with this.[0]
To say we can't describe timbre is a bit misleading, because there are concrete mathematical ways to analyze sounds; they exist in a three-dimensional space of frequency, amplitude and time. But that's helpful in the same way that describing programming languages as collections of 1s and 0s is.
What's lacking for describing timbre, I suppose, are the steps between "this sound is a sum of a particular arrangement of sine waves" and "this sound is a piano". There are common terms such as ADSR or "brightness" and "warmth" but those don't tell the full story.
The question is, how valuable is that intermediate step when you could just say "this is a piano, this is a banjo"?
You certainly can do minor with a banjo.
> Watch our audio director Ben McCullough describes how he created a unique audio environment and soundtrack for a Hardspace: Shipbreaker.
It's not so much the timbre as the envelope. We have ways to talk about the not envelope. That is complex though, because different components of the sound can have different envelopes: e.g. high harmonics dying off before fundamentals. Envelope is not separable from timbre.
Not exactly about death, but certainly grieving things like the emotional blunting of psychiatric medications, inability to accept love or forgiveness, and lines like:
> I'll pretend bein' with you doesn't feel like drowning
Sounds decidedly less hopeful to me than that track from Outer Wilds
live: https://www.youtube.com/watch?v=psIRw0d509w
studio version: https://www.youtube.com/watch?v=G-XICfi4j3Q
There's a whole sub-genre about this that's commonly played on banjo:
Edit: I was thinking of The Outer Worlds, not Outer Wilds btw
That’s what stops music from being math.
I'm scandalized. /s