Then you'll need to create own version of migrations, routing, configure asset building pipeline etc etc etc. Phoenix have everything in place, without need to re-implement lots and lots of basics
A bit pedantic, but migrations are provided by Ecto [1], not by Phoenix. You don't need Phoenix to use Ecto.