Similarly being unhappy with pulling dependencies without using a system package manager, in part because it seemed tricky to make them work with a compiler version from system repositories, this is something I wanted to try as well, and finally tried now. After having rustc installed, had to install 157 more packages in order to get argument parsing (librust-clap-dev), but it is still faster than cargo updating its package index, and then it actually builds!

Would be even better if the dependency tree for a basic package was more modest, about 157 times smaller.