I don't really notice much difference in first and third person perspectives once into a book, although it's more noticeable when starting out. Second person is more jarring - I've read The Raven Tower and Harrow The Ninth recently, both being all or largely in second person.
Interesting point you make with regards to RPGs - open worlds are addictive to me (I played Skyrim for many, many years - and have played almost everything in the Elder Scrolls series).