I've just tried this and in my case it leads to "cd \~[...]" which obviously does not work.
export FZF_ALT_C_COMMAND="bfs -type d -nohidden"
It won't let you go back up the tree, but if you are clever you can use `cd -` to jump back to your previous location.