Just to add another tale from the past:

Sometime in the late 90s I ordered a Red Hat CD from cheapbytes.com (CD burners weren't common yet). I didn't have a credit card yet, but I did have a checking account, so I paid for the order by mailing in a check.