Theres a ton of fig and dates in socal people don’t pick. That being said its hard to pick these trees. They might be located in clumsy areas like by some razor wire in a parking lot. You aren’t sure when its time to harvest unless you know this tree well. Harvesting might not even be practical considering the best fruit is picked not dropped and picked up, and these trees could be huge. Then of course these being unsprayed trees the fruit is going to be full of wasp larvae at least for the figs. Then the rats will eat most of whatever is there thats worth eating anyhow before you realize its there.

Commercially its a different story. Trees are planted and pruned to maintain easy harvesting. They are sprayed and rodents managed, as well as being overall outproduced by the yield on the farm. Fruit trees are a lot of work. Even literal money growing on trees would be work, against what the saying might imply.