Yes, fraud is an issue, but it doesn't scare me from the idea. You risk 'fraud' when you purchase a rug from a souk in Marrakech. There is a chance that rug was made in China, but you chat with the store owner, inspect the rug, and you make an educated decision. Buying on Gumroad will be like that, and with a reputation system I feel most of the fraud can be mitigated.
Concerning if enough people would want to use it, Sahil could speak to that better than I. However, I feel like enough people 'use' flea markets, right? What's stopping them from doing the same with their digital goods - especially if there is a known digital 'flea market' to publish your gumlinks to.
Or maybe a consignment store is a better example with the website owner purging and sorting the digital 'product'. Gumroad marketplaces like a poetry corner or upcoming garage bands might start cropping up.