The problem is that inevitably "but they're morons" becomes the excuse to deny them agency and freedom. And then it's just a matter of who gets to decide who is "smart" enough to deserve freedom. Every single time we've gone down that path it's led to oppression and abuse.