You can also (at least in Big Sur) tap and swipe right/left quickly and let go right after and it'll bump up/down the vol/brightness 1 bar, depending what icon you tap and swipe on. If the brightness/vol expands then you've held it down too long.

I do wish Apple would improve media controls on the Mac in general (and touchbar). So often the media icon just doesn't show (e.g. Spotify), or I've had several media apps running and macOS gets confused and the media icons no longer do anything for what's currently playing. It's soo buggy.