Aluminum is about 3x the weight of ABS, but is significantly stronger. It’s not at all obvious that any weight could be ‘shaved’ while still making the device as durable.
We do already have an example of Apple using aluminium for a wearable device that's traditionally made of plastic - the AirPods Max. It's well known for being significantly heavier than the competition, they weren't able to reduce the volume of material enough to offset how much heavier it is.
What makes you think they were trying to match the competition’s weight?