One alternative to leaving out the questions that take a long time is to calculate the average of log(minutes).

Harder to interpret, but it accounts for the questions with long delays without letting those questions dominate the calculation.