Sure, but the way we build buildings, there's no way that's actually the case. Transport alone to the sawmill, then the lumber yard, then the job site might actually release more carbon than the tree is storing.
A 25 year old maple will sequester 400 pounds of co2 in it's lifetime. An average commuter car will emit that in 2 weeks. Hauling all that carbon out of the forest will likely emit far more than that.