As someone who purchased a Note (1) on near release day (technically before release in the UK as I got an unlocked one before most of the contract networks had theirs on tap), I absolutely remember that.
Everyone online and off was claiming that the Note was simply "too big" and how it was a "tablet" and would never "fit in your pocket." But once I physically got the Note in my hands and they had a play, the tune quickly changed (at least for men with larger hands). Plus outside of skinny jeans it absolutely fits in my pockets.
When it came time to upgrade my Note (1), my choice of phones was much wider as almost every flagship had significantly closed the gap (e.g. 4.2-4.5" was the standard when the Note(1) got released (5.3"), but it was 5-5.1" when I had to upgrade).
So when I upgraded I went from the Note (1) to the LG G2, which is a 5.3" to 5.2" change. I didn't really notice the extra 1" but I did notice the significantly smaller brazil on the LG G2 and lighter weight.
If the Note (1) got released in 2014 people wouldn't even notice it as being particularly large. The Note 3 is 5.7"(!) by contrast. That's how far things have come.