Sources:
1. Peter Suber's Open Access monograph, chapter 5, p. 107-110 (itself open access, of course). https://cyber.harvard.edu/hoap/Open_Access_(the_book) argues that OA books facilitate "searching and sampling," and then people want to read the whole thing in the nicer print format.
2. University of Michigan Press says it happens sometimes thanks to increased "awareness and visibility": https://www.press.umich.edu/openaccess
3. A phd thesis reviewing studies, and finding a negative relationship between OA and sales: https://openaccess.leidenuniv.nl/handle/1887/68465
4. A swiss study of matched pairs of books, one OA, one traditional; no statistically significant difference in sales http://www.snf.ch/SiteCollectionDocuments/OAPEN-CH_schlussbe...