I was also sceptical about the size of the Nexus 6, but when Google dropped the price a few weeks ago I decided to give it a try (if you buy from Motorola or Google, you can try it out for two weeks to see if it works for you and return it if not).
I absolutely love it. I even sold my 7" tablet, as the extra screen size of the Nexus 6 is great for reading and web browsing, on which I spend several hours per day. However, I have always used phones with two hands, so if you're someone who likes to use one hand then you probably won't like it.
Coming from the Moto X, I do miss the excellent Motorola software, gestures and IR sensors, but the extra screen size, better speakers, better camera, wireless charging and most importantly for me, the rapid android updates make the Nexus 6 worthwhile.