Well, that's a really broad question in itself. It is possible in the future some vulnerability will exist in the framework, a security researcher just hasn't discovered it yet. If you're referring to a framework written in C/C++, then yes a buffer overflow is possible. Hopefully though, modern day protections such as ASLR, CFG, CFI, etc. protect you. I'll refer you to an old classic http://insecure.org/stf/smashstack.html, however it is very hard to do this attack today. Modern compilers are pretty good at preventing developers from shooting themselves. <3 clang's memory sanitizer.

Arbitrary execution is more possible IMO. The way a lot of ML models are stored is by being a serialized file using a framework like pickle or Java's serialization. Theoretically, you could add code into a precompiled model that when someone loads would execute arbitrary code. This could be done using a simple technique like a code cave seen here https://en.wikipedia.org/wiki/Code_cave. I haven't had time to dig into this myself, but I honestly don't think it would be hard.

I think in the next couple years you will see more vulnerabilities pop up in these frameworks, but finding security vulnerabilities take time.