Owning the domain name isn't a singularly valid way to assert trademark control; it has to be used. Elon had an "X" company way back when, but it became Paypal; they didn't really use the X name in a public fashion for very long at all.

The reason is probably more-so that Google's "X" isn't apparently a legal entity; its a division within Google. So, it doesn't have consumer sentiment around it, its not a product name they're selling, its not a registered entity, etc. In other words: "Skunkworks" as a term originated from a division at Lockheed, now its a term many companies use, oftentimes in legal & public ways, but Lockheed really doesn't have grounds to stop them from doing so.