Could you point to some examples where this has worked? It seems that the entry fee would either 1. limit the community's growth severely (not necessarily a bad thing), or 2. filter out good candidates.
Something Awful has used this to great effect to keep trolls in line. I wonder what % of their membership revenue comes from banned users purchasing new accounts.
Neither of these is anything close to a subreddit or forum which is the "community" to which the thread is generally referring.