Because the fruits have evolved to be eaten, most of them are tasty, but they have a very low content of nutritive substances, so the plants do not lose much by giving them away.
On the other hand, the parts of the plants that are the richest in nutritive substances are the seeds (this includes the edible parts of nuts).
A human could easily live by eating only some combinations of seeds, without eating any other plant-based or animal-based food. On the other hand, it would be quite difficult for a human to live by eating only other parts of the plants, because those have low levels of nutritive substances, like proteins and fat, so they have to be eaten in very large quantities to provide enough.
Because the seeds are the most valuable, most of them contain various harmful substances, attempting to discourage the animals that would eat them. Due to that, many kinds of seeds and nuts require special preparation methods in order to be used for human consumption, in order to remove or decompose the undesirable substances.
It’s a fun theory for people who don’t demand evidence.