We'll, there's the obvious argument that any open source software is better for your privacy and security than any closed source software, just because you know what it's doing. Then there's chrome's history of repeatedly discovering "whoops, we weren't supposed to be collecting that"... For me, it was when a researcher noticed that chrome kept his microphone activated all the time, because it was passively recording and sending all your audio to google. That one even made it into chromium! Then there's the abuse of their products to force people into their browser and OS - like Hangouts only working on chrome, or chrome not enabling hardware acceleration for any device but Chromebooks, despite a years old (trivial) patch to do so, and years of testing reports. Arch Linux maintains its own version of chromium just to have that patch included.
But for me, it's the conflict of interest that makes the real difference. Fundamentally, Google as a company makes it's money by selling information about me and my behavior on the internet. That's it's objective. Firefox exists to promote open standards, OSS, and privacy on the Internet. If chrome at the moment doesn't seem to be contributing towards Google's objective, that's by definition only a temporary state of affairs. And I know which objective I would rather support.