Our HOA does a lot more than that: organise the occasional party, drink, or movie, manage a street-side gallery that's part of the building, manage a commonly owned apartment that can be rented for guests or other purposes, etc.
But regulating people's gardens is just weird. Well, trimming the commonly owned greenery is also something the HOA members do together and we're always asked to help (and never do), and of course we discuss stuff that affects our neighbours with those neighbours, and we barely have gardens because we live in a city, but anything interesting garden-wise is generally welcomed. The idea of a HOA banning vegetable or eco-friendly gardens is nuts.
And this is in a single building. If you're talking US-style free-standing suburban houses, it quickly gets a lot more ridiculous.