I have a repo with 3000+ files and I can't notice any delay searching with CtrlP. I have the following custom file listing command in my vimrc:

    let g:ctrlp_user_command = {
      \ 'types': {
        \ 1: ['.git/', 'cd %s && git ls-files'],
        \ 2: ['.hg/', 'hg --cwd %s locate -I .'],
        \ },
      \ 'fallback': 'find %s -type f'
      \ }