I actually specifically wrote about this after receiving an email calling imposter syndrome out as the reason the person wanted to stop the interview process. Some thoughts on how to overcome it are in the article.
https://medium.com/@DavidMcKayV/obliterate-imposter-syndrome...