So if this is the actual reason, shutting down completely seems a bit of an overreaction. Why not just get a new logo and rebrand the app ?
But if they create a new thing as a new team and claim that it shares nothing with vanced; and, critically, they never release the source code from the original vanced (which presumably hasn’t been subpoenaed); then they can reuse the source code, because nobody will know the difference.
This, presumably, would be what they mean by “complications” if they release Vance’s code: it’ll retroactively prove that their new """unrelated""" project is actually related after all.
Redistribution of copyright protected material requires authorization of the right holder. Code (or programs compiled from code), usually fall under this rule.
(In the US) hoster privilege usually allows them to not be liable unless they know about a specific infraction. In some jurisdictions some platforms (such as youtube) are required to monitor uploads proactively for known copyright violations.
Bytecode inspection is a thing. It isn't too hard to know if two apks share a large amount of source.
Vanced itself is based on decompiled bytecode of original Youtube app. Another team can build on top of Vanced app just like Vanced devs built on top of Youtube app. So we wouldn't necessarily say whether it is using a hypothetical "source code" leaked by Vanced team or not by bytecode inspection (nor would it matter much).
It wouldn't cost much, compared to litigation, for Google to send them a letter outlining these issues. It's possible they made Vanced an offer they couldn't refuse (i.e., shut down now and avoid litigation)