Fruits in terms of calories are essentially sugar, and despite the fiber, more than 1-2 a day will start to impact insulin, fat storage etc. negatively for most people. Especially fructose-heavy fruit.
Non-starchy vegetables are not nearly as sugar or calorie dense, can be eaten in much higher volume with little effect, and as such should be thought of much differently than fruit.
(non-avocado) Fruit falls more in the dessert/candy category IMO.