Manhattan has twice the residents based on U.S. Census population data, but actual daytime population is more than just residents. A more important statistic is commuter-adjusted population [1], i.e. number of people in an area during normal business hours, including workers. That's where Manhattan and San Francisco really diverge.
The commuter-adjusted population of Manhattan is 3.1 M [2], compared to 1.6 M residents [3]. San Francisco has a commuter-adjusted population of 1.0 M [4, 5], compared to 0.8 M residents [5]. In other words, Manhattan's population booms almost 200% during normal business hours, while San Francisco's increases a modest 25%. Manhattan may have 2x more residents, but it has 3x more daytime population than San Francisco.
Given it has about half the land area, Manhattan's daytime population density is thus 6x that of San Francisco.
Attitudes on development -- namely transportation infrastructure and building construction -- certainly contribute to that difference.
1. http://www.census.gov/hhes/commuting/data/daytimepop.html
2. http://www.citylab.com/commute/2013/05/most-important-popula...
3. http://quickfacts.census.gov/qfd/states/36/36061.html
4. http://ww2.kqed.org/lowdown/2014/01/10/how-city-populations-...
5. http://factfinder.census.gov/faces/tableservices/jsf/pages/p...