The early colonists were substitance farmers and there are no journals or diaries or documents which talk about building walls.
They do have diaries of farming and painting and all other topics but not building walls up mountains and sprawling over legde and landscapes.
Search up liDAR and New England. The region is still being scanned but each time they scan more and more walls are illuminated.
https://www.atlasobscura.com/articles/crowdsourcing-to-map-t...
https://nhdes.maps.arcgis.com/apps/webappviewer/index.html?i...
What if the colonists occupied the land those people had farmed!
Edit: I'd add that these stone walls are still used by surveyors, especially when the historical records may be spotty, but also just because they in many cases happen to still fall along existing property lines. There _are_ a few "ancient" stone sites in New England of disputed origin, including "Mystery Hill" (also known as "America's Stonehenge"). But there is a strong argument to be made that these were constructed by modern Americans as hoaxes.
It's simply impossible for the farmers to have built all the walls, even if thats all they did was build walls all day every day for generations.
The early colonists were sustenance farmers and there are no journals or diaries or documents which talk about them building walls.
There is not one piece of written evidence that the colonists created all the walls. There is not one piece of written evidence that the native Americans built the walls.
We don't know who built the walls.