You don't have to use all Hibernate features. You can write your own SQLs too if you want to.
Hibernate can be super fast if you use the correct strategies [1] at your bottlenecks. Any other performance issues are typically ORM related.
[1]: http://docs.jboss.org/hibernate/core/3.3/reference/en/html/p...