This is extremely fascinating to me. In what Andrej Karpathy calls "software 2.0"[1], leaking your model is equivalent to leaking the main IP of your company. Unlike source code where it loses value quickly out of context (e.g. twitch leaked their code yet that didn't spawn a bunch of twitch clones), models can be fine-tuned and transfer-learning-ed for many other purposes
Since these models take millions of dollars to train, I see these sort of hacks becoming a thing! I wonder when companies will start adding "trap streets"[2] to prove that others are using their stolen models?