Well you are productive as long as your kids have access to good teachers, family has access to good doctors etc. Once you need those things frequently and can afford it , small town life can turn into a big headache.
On the contrary, most small towns (I have lived in) have much finer grained community support networks, many ad-hoc, that can provide better quality of life. This is however highly dependent on your local community. However it is never too late to start something. I find if there is a need, there is a huge untapped well of hands-on support you just don't find in urban environments. The fact that people still know their neighbors is a huge enabler and stimulus of compassion and support.