It's should also be noted that average heights were about 4 inches shorter than the height of a modern American. Not having good enough nutrition to reach a full height seems like a pretty sure sign of poverty to me.
They did not so much have the crowded slums that are typical of modern poverty. People would die of disease before cities could reach such densities. But relative to us, people were impoverished in terms of health, nutrition, or material comfort.
EDIT:
One more point. Overall violent death rates ( including war and homicide) in 16th-19th century Britain were about the same as the violent death rate in modern Houston, Texas. See Greg Clark, A Farewell to Alms, pages 128-129.
You realize most of the world would think this is abnormally high?