Not just you. I've been using it on and off since Nightly and it drains my battery fairly quickly. I'm back to using Chrome.
Ironically, I read this blog post (https://hacks.mozilla.org/2017/11/entering-the-quantum-era-h...) in Chrome and Firefox, and on Firefox the scrolling was noticeably slow to the point where I found it distracting.
Edit: Turns out scrolling a simple page up and down causes Firefox to use 40% CPU. Is it not using any hardware acceleration?