But no, "beautiful" in this context just means prettier physical outlook of buildings. You can't over-interpret it (at least in Chinese).
Good aesthetics, yes, part of '美丽'. But I won't say '美丽' stops short at simply say 'we will build our cities to look pleasing', at least I won't expect the Constitution to be so pragmatic, because it serves as an guide on the ideology (sort of).
However, I beg to differ. Here is the full latest constitution:
Chinese: http://www.12371.cn/special/zggcdzc/zggcdzcqw/
English: http://news.xinhuanet.com/english/download/Constitution_of_t...
As you can see, the paragraph on "beautiful" is just a tiny part of the constitution and the word "beautiful" only appears once throughout the entire constitution. So it is probably not something very "guiding" and "central".
Also, the constitution touches on a lot of aspects of the society, so I wouldn't be surprised if it is amended to include a small mention of the "look" of the physical landscape.
I would say that the word is added just to make sure that the party keep environmental concerns and "good outlook" in mind while working on all the other tasks.
I don't believe abruptly inserting a term that's never mentioned is good writing style.
A plausible way to understand it is that, the concluding remark is a summarizing call-to-action, laying out the spiritual and ideological framework for the rest to act upon. It certainly needs to be abstract, just as the four words preceding 美丽.
You can see the meaning of '美丽' from this sentence '生态环境根本好转，美丽中国目标基本实现' (the ecological environment has fundamentally improved, and the goal of a beautiful China has basically been achieved).
This is the speech Xi delivered at the 19th National Congress of the Communist Party of China. It contains this sentence. If you skim the speech, you will find multiple mentions of '美丽', all of them environment related.
The other four words are related to other aspects of policies of China.
富强(prosperous) is related to economy.
民主(democratic) is related to politics.
文明(culturally advanced) is related to culture.
和谐(harmonious) which was added to the constitution by Xi's predecessor Hu Jintao, is related to society.
And 美丽(beautiful) is related to ecology or environment.
------
富强, 民主, 文明 and 和谐 are the first words of the 'core socialist values' which contain 12 such words and which were laid out at the 18th National Congress of the Communist Party of China, when Hu stepped down and Xi become the paramount leader of Communist Party of China.
'Core socialist values' are taught to students from kindergarten to college. And these words of 'core socialist values' are plastered over walls, buildings and flyovers at every cities in China.