If you're going to quote Silverstein and give him the benefit of the doubt, then at least quote the full quote. The end of your quote is "... and then we watched the building collapse..." leading you to understand that he's referring to a cause and effect.
As in, they decided to pull and then the building collapsed. This means one of two things:
1) They pulled as in the context of some kind of an act of a pre determined demolition of the building
2) Something closer to what you're referring to, but with the foresight that the building WILL collapse and actually did as they have expected it to by their decision to pull the men (and women) and equipment. To which of course the first question is, why then? why at 5pm and not any of the hours before Larry made that decision. It only took about 40 or so minutes for the big buildings to fail.
Option (2) is equally problematic as option (1) in the context of 9/11. Given that the claim is that all 3 buildings (all the WTC complex really) has suffered an equal progressive fate made by 2 projectiles hitting WTC 1&2.