Dynamic music and sound techniques for video games | Hacker News Reader

Dynamic music and sound techniques for video games

blog.gingerbeardman.com

101 pointsecliptik36 comments

While not as procedurally advanced as some of the techniques explored here, in my opinion the absolute pinnacle of dynamic music in video games is the soundtrack to Mount Wario in Mario Kart 8.

While the level itself (an unusual point-to-point, highly vertical course) is its own masterclass in terms of game design, the music, comprising three separate movements, each with its own clever way of masking the transition to the next, is unparalleled in ramping up the excitement and drama in concert with the gameplay.

A short video on the music of Mount Wario: https://www.youtube.com/watch?v=-8TBS75U1p8

A longer deep-dive into the music theory of the score: https://www.youtube.com/watch?v=-6K4NiDP04I

Fantastic, Mount Wario is the greatest!

Thanks for sharing. I learned something new! That’s my favorite level, and I love the crazy shortcut.

Reminds me a bit of a game dev talk from a Valve employee. Having trouble finding the video, but one of the interesting things that stood out to me was all the challenges with sound effects in VR. For a door, you can’t just have “door opens” being a simple creak. Because in VR the user physically drags the door open. They can drag it part way, they can open the door quickly, slowly, etc. What in other games was a simple audio file became very complex in VR.

That might be from their hour-long talk on Doors: https://m.youtube.com/watch?v=9kzu2Y33yKM

Nothing will beat IMUSE from Monkey Island 2. Maybe because of the way MIDI worked versus the "pre-rendered" audio used today?

https://www.youtube.com/watch?v=AjtxK_WT784

I recently listened to a 2015 interview with Ron Gilbert where he discusses this, it's here: https://youtu.be/cktmhqXMsGI?t=3638 and summarised in this reddit comment: https://www.reddit.com/r/gamedev/comments/4bwnzm/comment/d1d...

Maybe real-time AI generated music, at least for transitions, can eventually do the same thing iMUSE did:

https://www.youtube.com/watch?v=Lxxu1dVtg5g

As a fellow Monkey Island 2 and iMuse nostalgiac, I also really enjoyed the user-controllable musical segue effect used in the recent game Sea of Stars.

https://www.youtube.com/watch?v=4dOG4DECEGk

(technically speaking I assume that is nothing like the MIDI-based iMuse, but rather, something like a cross-fade between 2 synchronised versions of the same song, but still! - it sounds neat, and makes me feel a little like iMuse did back in the 90s)

Musical invention and interaction is a wonderful aspect of video gaming.

Yeah there was a big push towards this in the 90s, I seem to remember the Wing Commander games and Ultima Underworld trying it. It's a similar tragedy to voiceovers in games - RPGs had increasingly interesting dialogue systems, Wizardry and Ultima would let you talk to NPCs using keywords or whole sentences. Voice acting killed that entirely. One can only hope that the era of LLMs brings back some of this interactivity.

> One can only hope that the era of LLMs brings back some of this interactivity.

The Wheel turns, and things Old become New again.

I just recently played Nier Automata for the first time, and was amazed by not only the soundtrack, but the way it handle dynamically changing as you interact with other characters, or when certain story elements are triggered.

There’s even an in-game jukebox that lets you explore the music, and it’s crazy to see how nested it gets based on different environments.

Definitely recommend checking out if you, like me, are elevated by games with great music.

I was just about to mention Nier Automata too. The dynamically changing music was a huge wow moment to me when I played it.

Actually this game is such an art. Its story is touching, full of unexpected plot twist, and inspiring deep thinking afterwards; its graphics is cozy and unsettling, heart warming and hear breaking at the same time - yet the game is not really gpu demanding; its combat is slick, snap, and satisfying; its music literally echo with your emotional roller coaster along your gameplay. And what's crazy is that, all of these nice things about this game actually blend with each other really perfect, so you don't think about any single aspect of this game, you feel the whole game as a package delivers you one of, if not THE, finest japanese anime/game experience in so many levels. I was really depressed for like 6 months after finishing this game because I left wanting more of Nier Automata games. It will literally make you become a huuuuge weeb.

I especially like the seamless transitions between different orchestrations of the score when moving between certain areas, and the chiptune transition when entering the hacking minigame, e.g., from

https://archive.org/details/nier-automata-original-soundtrac...

https://archive.org/details/NieR_Automata_Original_Soundtrac...

They wrote up a pretty good blog post about how they made the music transitions work: https://www.platinumgames.com/official-blog/article/9581

Definitely on my list of games I wish I could play again for the first time.

My first thought was Untitled Goose Game! So cheeky and fun how the music works in it

Fun fact: they built that in after their first trailer was made.

The "dynamic" music had apparently set some unexpected expectations.

Mine was Soldier of Fortune. It blends smoothly between high alert and sneak-around segments, eg. tracks 2 and 3 on the first level: https://www.youtube.com/watch?v=ZI_14lp1U2w&list=PLALRZSnKU7...

I was thinking about SSX tricky and 3. How the music loops and/or fades in for example big air time or landing a trick.

There were certain layers of the playing track that would fade away when in air such as vocals. I always thought that this was done really great in these games.

Tangential to this topic, my favorite example of dynamic music is in Need for Speed Most Wanted 2005's police pursuit soundtrack: https://youtu.be/YtSQCYStwew

While certainly much more primitive than modern games (it's merely looping segments of a premade track), Most Wanted 05 does a great job of live scoring the pursuits, playing, looping, and smoothly segueing between the track segments. From experience I guess it selects the segments based on the heat level, player speed, police proximity, and other factors that I'm not even sure about.

There are 4 pursuit soundtracks in the game, I highly recommend people listen to them if not try the pursuit mode itself in game. If nothing else it's thrilling as hell, a feeling that the modern entries still somehow fail to capture fully.

I've noticed while playing PC and Android games, volume will sometimes apparently increase or decrease on it's own - without me touching the volume knob or any visible change in the volume setting. Is there some known bug in common (eg. Realtek?) drivers or hardware that may explain it? Has anyone else encountered?

Is some sort of volume "equalisation" or compression being applied maybe? Could also be reacting to the change in ambient volume conditions otherwise.

Sonic Frontiers had a fun take on the music for boss battles, with each song being broken up into not-so-obvious loops, with the action masking transitions so that each phase of the boss fight always goes with a specific portion of the song. If you can lean into the cheesiness it's very fun.

Reminds me that there's some interesting audio techniques in the new Avatar game, like ray traced audio:

https://youtu.be/LRI_qgVSwMY (skip to 13:16 for the audio section)

I remember a talk by Borislav Slavov (composer for Divinity: Original Sin 2, BG3) where he was showing how the music changes depending on the player's actions in D:OS2.

One of the examples, that I found really cool, was switching from major to minor scale when the player enters in stealth mode. There were other interesting bits and tricks, but this one really stood out for me.

Also, it worths mentioning that the player was able to choose the lead instrument for the in-game music during the character creation - flute, tambour, oud, and cello.

Not sure if that talk is available online. I've got the chance to meet him in person a few years ago.

DirectMusic had aspects of this.

https://en.wikipedia.org/wiki/DirectMusic

I thought about for long how SFX in f.e. airsims are done nowadays: different engine characteristics are just one thing. Are there still samples / recordings involved?

Same with racing or train sims.

I expect so, but it would depend on the game. I know they still use engine samples (and probably hugely advanced systems for playing them back) in car racing games like Gran Turismo.

You might find this software interesting:

https://www.youtube.com/watch?v=RKT-sKtR970

I don't know if any of the state of the art flight/driving simulators are doing this kind of engine noise synthesis already, but they should be.

Thanks!

The problem here is mostly computational cost. Simulating finer grained details needs a proportionally higher tick rate. In this case although the engine sim seems to be 2D and the gas sim 1D but both would need really high resolution to output sound synthesis. Driving games for now are largely putting that work into tyre dynamics as it’s such an integral part of the driving experience. Maybe there’s head room to do this x number of cars on the track but I don’t think it’s a foregone conclusion that they can just slap in a similar simulation.

With modern systems having so many cores there's plenty of compute available for at least simulating the player's engine sounds more elaborately... They don't have to do it for every vehicle in the sim.

That’s certainly the hope how well it matches reality is the problem.

Super Crazy Rhythm Castle just came out and it’s entirely about procedural sound and music.

> I mostly use royalty free music of Japanese origin

Like what? I’d like to hear what royalty free Japanese music sounds like.

Here's the music that I used in my most recent game (which I call: Fore! Track): https://twitter.com/gingerbeardman/status/173255553386375169...

This track is by an amazing, long-standing musician that goes by the name watson and I listened to their entire catalogue and made a playlist of my favourites to be able to easily select a piece that fits any game I'm working on. Their umbrella website MusMus is https://musmus.main.jp and whilst you'll need to run browser translation on the site you will see all their music sorted across many categories. That said, it is definitely easier to listen to their music through the albums they've released on streaming platforms: Spotify and Apple Music for certain, and probably others. They're also on YouTube.

Of course, there are many more Japanese services and people writing great royalty free music. Another of my games used royalty free music by a young Japanese musician named YuyakeMonster, again picked specifically to fit a particular game: https://soundcloud.com/mac-vogelsang/sets/sparrow-solitaire

Reply on news.ycombinator.com