The same thing will probably happen here because this is not one of the purposes of DMCA takedown letters, period. Even if there are inappropriate test cases or something in the repo, or if they’re correct that youtube-dl bypasses DRM in violation of a different part of the DMCA, it’s still not a valid takedown because GitHub isn’t hosting anything the RIAA/those it represents own the copyright to. The correct way to do this is to go after the lead youtube-dl developer(s) and/or GitHub for facilitating infringement or whatever, but I don’t think RIAA wants to do that because they probably don’t have much/any legitimate grounds for legal claims against them, so they abuse the DMCA to look like they’re doing something.
Edit: for a more concrete picture of what happens next, read GitHub’s DMCA policy.[2] Basically youtube-dl can file a counter notice assuming they disagree with the claims, after that the RIAA has 10-14 days to get a court order or the repo goes back up.
[1] https://torrentfreak.com/github-reinstates-popcorn-time-code...
[2] https://docs.github.com/en/free-pro-team@latest/github/site-...