At 2.5 Gbps having 5 ms of buffer requires >1 MB for that single hardware queue which was either at or over the available buffer for the entire switch IIRC.
Having too large of a buffer, particularly on gigabit class software routers/nat points which use CPU+RAM instead of expensive hardware, is definitely the more common problem. There it's too easy for the trap "I have 512 MB of RAM, why shouldn't I let as much as is available be used as buffer, it's helping" which then makes you wish you had 4 MB of buffer instead.