I'd imagine a complex game engine has some bugs or weird behaviours in specific conditions. If it can be proven that the closed source game has a lot of the same bugs/behaviours, that is likely enough to win a lawsuit.
>Reverse engineers have entered the chat room.
Finding this would be pretty trivial depending on how much was stolen. And proving this would be more of a matter for Rockstar lawyers
Decompile it and see things work exactly the way the original game does, bugs and all