You asked about selling books outside of the software realm. The In 30 Minutes guides are aimed at mainstream users, and I found through some early pricing experiments that attempting to charge more than $10 for these types of ebooks is very difficult. I have also found through more recent experiments that pricing ebook versions less than $5 leads to an immediate and sustained rise in sales.
I sell ebooks through Amazon, iTunes, and B&N as well as PDF versions on Gumroad. I have also made mobi/epub versions available on Gumroad, but very few people buy them -- Amazon, Apple and Android tablet manufacturers do not make it easy to transfer these types of files to their devices, which is a roadblock to most people in my target audience.
In my limited experience promoting fiction, factors such as categorization, whether or not the book is in a series, and the number of reviews on Amazon or Goodreads influence awareness and purchasing decisions. Because there is so much competition in the fiction realm, and most new titles by relatively unknown authors are priced at $4 or less, I think the reaction to an expensive fiction ebook by another unknown author would be negative.
For anyone who is interested in self-publishing, I wrote another HN post last year that a lot of people found useful. It includes tips on how to get started using a lean approach to writing, production, and marketing (2). Good luck!
It began as a beta, then a full PDF + supplement package, then a video course a year later. I'm currently working on a calculus series.
I'm not sure how it'd work with fiction (author interviews maybe?) but for non-fiction I think it can work well.
I know what you mean, though I'd like to point out that there is a serious and successful historical precedent for the publication of unfinished fiction: serial publishing (the way big novels used to be published, cf. Dostoyevsky, Tolstoy, Dickens, et al). Novelists would publish chapter-by-chapter, before finishing the whole thing, and would even change their plans for their novels in response to feedback from the public.
Now, of course for novels published serially, people do expect that at least each published chapter is finished when it is published, and is not a draft subject to change. However, as a former academic type, I find reading early drafts to be fascinating and meaningful; watching the jumble untangle is an exciting process. I've spent a lot of time reading early drafts of poems, and comparing how different versions of the same work are published by the author over time.
(btw I work at Leanpub)
Sell only the first few chapters. Or sell it in serial format.
[1] http://www.tor.com/ (For those unfamiliar, Tor was very early with DRM-free e-books as well as offering the first books of series for free download (expecting readers to purchase following books)).