I don't know if that's actually the case, given that SE is noticeably larger than Mini.
The only people I know who pick SE are those who either still want the physical home button or just want the cheapest new iPhone. In all other metrics, including smaller physical size, Mini wins hands down.
So I find your claim about people who loudly demanded a smaller iPhone cannibalizing Mini sales by picking a larger SE instead a bit difficult to believe.