Youth and timing! I dropped out of college after my 3rd semester. I read widely and kept seeing W&P cited as the greatest piece of literature ever. I read it and liked it -- in two weeks, which was about 100 pages a day. I read it during breakfasts, dinners, coffee breaks and lunch at work, and then at bedtime until 2 in the morning. So, I was able to concentrate on it and enjoy it; being young, I could slough off the lack of sleep.
I read AK next and it was great. A customer at work saw me carrying AK and suggested I read BK, so I did. I too found BK to be my favorite of FD's novels. I also read most of his other novels, except I only got part-way through The Idiot. When I returned to school, I took a Russian Literature course, for which I reread W&P, AK, BK, and Crime and Punishment (instead of using the Cliff's Notes).
(My professor pointed out the humor at the end of AK, something I had failed to see in two readings: the narrator is talking to what's-his-name, who has a horrific toothache and therefore doesn't really care that the most beautiful woman in Russia ... oops, no spoiler here! It was kind of funny when he explained it and he was a zillion times more knowledgeable than me about FD and FD's writings.)
Again, I had youth on my side and, as you noted is important, I had the time to concentrate on the books. Decades later, a few years ago, I finally read The Idiot all the way through -- it was a long slog; it being a weird story anyway didn't help.