Which you can do via, for example

sudo /sbin/ip addr del <ipv6address>/<prefixlength> dev ethX