Cartridges[1] are prohibited on planes for good reason, I think. It is possible to set them off accidentally, and once you have a cartridge, building a one-time use zipgun is relatively simple, so if you want to keep firearms out of an area, you really need to keep cartridges out of that area.
Of course, if the bullet was really just the bullet without the propellant (that was not clear to me) then yeah, I agree with you.
[1] I think It's the explosive powder that matters here. A bullet with no propellant is just a lump of lead, which I think would be okay.