And not only does the ORM itself have big performance impact, your own code will be designed around the assumptions of this particular ORM, e.g if it is easier to fetch a few row objects from different tables using the ORM and then do some comparison computations in your relatively slow interpreted language rather than writing one fast SQL JOIN the former will almost always be the solution that ends up in production.
ORMs invites you to write bad code, instead of thinking ahead of what queries I need for this specific operation, you just start writing ORM based code aimlessly and try to make it work with the tools available in your language of choice.
And the ORM entities will eventually infect every part of your project and it will almost be impossible to remove them after the fact.