This would be impractical. For one thing, scuff-resistant packaging material is also necessarily less attractive than glossy and colorful packaging that holds high detail photographs & illustrations better.
This would also be very inefficient because you would have to store the postal-ready boxes at the fulfillment warehouses in an inefficient way instead of just putting them into bins. When shipping products to customers, also, it's often more efficient to pack multiple products into a single box to ensure both durability and cost efficiency.
Most of the goods available on Amazon also do not come direct from the factory. They come from various intermediaries. Not all products are shipped from the Amazon FBA warehouses, either. So, there are many reasons why it is impossible.
It is possible to ship to customers direct from the factory, as some other comments have mentioned. However, as you might imagine, this is impossible for many businesses. Processing customer orders is time and labor intensive particularly because it involves making many tiny shipments versus shipping out entire truckloads at a time only. Furthermore, contract manufacturing is very common. A given factory may be manufacturing products for many different brands at the same time and a certain brand can have its products manufactured in entirely different countries at the same time.