True! Even the Benin Kingdom itself was, after all, an empire that sought to (and did) expand its borders within the region. What I am saying is that you cannot justify imperialist expansionism with "but they did good for the people they subjugated", because that good could have been accomplished without the looting and destruction of other nations.
In other words, some things simply don't need to be defended. We can acknowledge that they happened and discuss them levelly without trying to put bad actors in a better light. To invoke Godwin's Law a bit, we do manage to discuss Nazi Germany without constantly saying "Well actually we are all better off because they did lead to some advances in science and technology". (As an aside, colonialism is arguably more recent than Hitler - Nigeria for one did not gain independence from Britain until 1960. Yet one is considered "too soon" but the other is treated as though it happened in some distant past that we should all have gotten over by now.)