> does anyone really care about building a fictional garden?

I think the market already answered your question, given that some of the most popular video games from the past few years have been based around gardening and farming.