Agents mostly don't run ML/AI code; they're a structured loop around LLM calls and exist mostly to give an LLM access to local tools in some application domain; think "reading my email for me" rather than "driving ML systems".
I think I'd condense this out to "this is not a really important deciding factor in what language you choose for your agent". If you know you need something you can only get in Python, you'll write the agent in Python.