Now, if the incoming swarm is large enough, then either each drone carries a tiny amount of explosives, or each drone is large and expensive. The US Navy has been building ships to protect against mass swarms of incoming cruise missiles since about the 60s or so, so if the incoming drones are large enough then you could also use larger missiles against them and rely on the RAM only for whatever leaks through.
In practice, there's a world of difference. Drones are cheap and slow, but can be a big threat in a saturation attack style scenario. Do you want to spend a $1M+ missile to shoot down a $5000 drone?. Do you even have enough missiles to shoot down a swarm of 100 drones?
An anti-ship missile is much bigger and more expensive, but flies at around 300 m/s for a subsonic one, and up to 900 m/s for supersonic ones, giving much less time to shoot it down. And while the missile per se isn't armored, the warhead has a thick casing designed to penetrate deep into the ship before exploding, so hitting the missile with some shrapnel at short range isn't gonna help as the wreckage of the missile will still likely continue on a ballistic path and hit the ship. And the warhead is often big enough to cripple a warship with a single hit (well, maybe a carrier is big enough to keep on trucking).
The real winner is the swarm of highly effective missiles.