[...] the value of the average Japanese house depreciates to zero in 22 years. (It is calculated separately from the land, which is more likely to hold its value.) Most are knocked down and rebuilt. Sales of new homes far outstrip those of used ones, which usually change hands in the expectation that they will be demolished and replaced. In America and Europe second-hand houses accounted for 90% of sales and new-builds for 10% in 2017. In Japan the proportions are the other way around. — The Economist
There are old Wattle and Daub[1] houses near me that haven't been washed away by the UK weather.
Sound yes, heat not so much. While it can store heat due to it's thermal mass, it will release it quickly due to a low R-value, depending on moisture content the R-value could be below 1 per inch. Cellulose has around 3.5 per inch.
Without adding additional insulation to restrict thermal transfer, you need to build very thick walls. In sunny desert climates where days are hot and nights are cold, you can build your wall in a way that allows it to go through the heat exchange synced with the 24hr sun cycle, and then it's very efficient. Thinner cob walls need to be wrapped with insulation to slow down heat transfer.
You could say the same thing for houses now, which are built out of wood...