That's pretty much the key with this sort of thing. I've done very similar experiments and as long as you choose good source material, even fixed-length samples cut at random and then sequenced rhythmically will sound good.

The author's description of the music is slightly corny but I had a very similar awestruck and eerie feeling after the first time a python script I wrote generated some pretty passable electronic music.