You can just drop RST packets.
iptables -I INPUT -p tcp --tcp-flags ALL RST,ACK -j DROP
iptables -I INPUT -p tcp --tcp-flags ALL RST -j DROP