How hard to push back depends on the nature of discussion. Being too pessimistic in a research phase is not useful. But if it’s a contract worth half a year or more, spending few days evaluating assumptions is wise.
Remember that fast thinking is not actually fast, but shallow and/or optimistic.