The candidate went to all the trouble of engaging with you; you owe them (at the very least) a timely and professional rejection email if you don't want to proceed.
Yes, a rejection letter, because that's what you're doing -- rejecting them. Ghosting is still a rejection; it's just a rude rejection. There is no way to candy coat that, and you shouldn't try. Grow a spine, and write that rejection letter in the kindest way you can.