Probably because the only things you can do on that phone are things for which there is a feature. That's what distinguishes it from a general purpose (smart) phone. Similar to the distinction between general purpose computer vs an embedded system.

That's a real good question. I always supposed their name was out before smartphones (which are definitely more "featured") existed.

Like a safety razor, which is much less safe than modern alternatives.