You're arguing against a straw man: most existing voucher programs are means-tested already (ie. only available to your "high cost" students) which negates your argument that the low cost students will use vouchers leave the system.
Did you know that many schools out source highly disabled kids to specialized school, in essence running their own voucher system?