Maps that show the anatomy of America’s vast infrastructure
washingtonpost.com
washingtonpost.com
There's definitely room for improvement though. A big part of the issue is the mismatch between the private and municipal sides of the project. As someone on the private side as a consultant, it can be really frustrating to constantly jump through bureaucratic hoops that come across as massive wastes of time and money.
I could write a 5 page list with just individual single line item bullet points of all the factors/sub-contractors/sub-systems to take into consideration for a successful inter city fiber project.
And that is just at OSI layer 1 before you even start lighting the fiber.
But it's been fascinating to compare this to how inner-city rail projects went in the early 20th century— basically some company would get permission, lay down a bunch of track in the middle of the street, and be running trains within a few months. You see similar stuff with early electrification projects, where basically whoever could just come along and string their wires between lamp posts.
I totally see that that's insane, and we can't have a wild west with our infrastructure, but I do also wonder if there's not a middle ground that would allow somewhat more agility with these things.
For people working their way up through the ranks at an ISP, it is really helpful to spend as much time cross training and working with different groups of people, responsible for different things. Spend enough time in massive-scale hosting/colocation/datacenter operations that you're familiar with how to properly set up a facility for thousands of 1U size servers, but also spend time with OSP fiber people, learn the fundamentals of high capacity PTP microwave links. Spend time with the sysadmins who run back end infrastructure like large scale DNS, SMTP and IMAP servers, billing systems. And of course, all the time, continue to increase your IP/layer 3 networking knowledge until you're comfortable on something like an ASR9010 or MX960.
I think it's NYT who a few years ago had a series of articles about the work of maintaining the underground aqueducts that bring water to NYC, and in building a new one. [1] It was to me a more engrossing read than any OS deep-dive article, despite the latter being more in-line with my passion and work.
It's also why I loved the fantasy book "Two Serpents Rise" [2] by Max Gladstone, because it's basically about the water infrastructure of Los Angeles. Which has a fascinating real-world history of its own! [3]
[1] can't find NYT, but here's a related article: http://www.theverge.com/2013/10/19/4853636/underground-with-...
[2] http://www.goodreads.com/book/show/16059411-two-serpents-ris...
I specialize in water, so getting access to the treatment plans is my favorite. Drinking water plants are mostly off limits for the public but I've gotten tours of a few.
If you're around the NYC area try visiting the Brooklyn Wastewater Treatment Plant. Those egg shaped digesters are really well designed. I even heard of a couple getting married there.
Two of my favorite books:
The Works: Anatomy of a City https://www.amazon.com/Works-Anatomy-City-Kate-Ascher/dp/159...
What Do People Do All Day https://www.amazon.com/Richard-Scarrys-What-People-World/dp/...
I wish Amazon would show a few random pages from the middle--the actual content of the book--so we can see what we're going to be reading. It seems like common sense.
If a library is nearby, you could look at it there, or even check it out: http://www.worldcat.org/title/works-anatomy-of-a-city/
e.g. her papers from the 90s
Steps Toward an Ecology of Infrastructure: Design and Access for Large Information Spaces https://pdfs.semanticscholar.org/f0c2/2ec49e87aecec1bd54506b...
The Ethnography of Infrastructure http://www.imtfi.uci.edu/files/articles/Star.pdf
On the same topic, another inspiration for fiction is the book Cadillac Desert by Marc Reisner:
The issue with technology in municipal governments is with budgeting and the "It works, why do we need to change it?" mentality. Governments will still insist on things sent in formats that were modern 20 years ago, why should I have to send 5 paper copies by mail to the DEP or mail a CD?
The civil engineering industry as a whole is still very paper based as well. Paper copies are signed and sealed by hand and sent in copies out to clients/bidders. Having to spend time printing everything gets extremely time and resource consuming. The argument for this style though is that some projects have a big public liability and records must be kept.
There are pushes in the right direction, but change is slow and many don't see the incentive of upending entire systems even though they can provide a real benefit.
There is a very underserved segment of the municipal government open/closed software space: for village-scale governments that declare independence from annexation from a nearby larger municipality seeking tax bases with dense enough populations. Generally these villages that get annexed are taxed just like their big-city counterparts, but still have to drive all the way into the city to avail themselves of most the city amenities they pay dearly for.
In these smaller village governments, they are extremely open to staying modern to save money. Some of their annual budgets are measured in six figures in the US, with one part-time clerical employee. They'll spend money to retain attorneys who specialize in this sector to do all the paperwork to interface to regional civil authorities, and state and federal authorities, because it outsources complex work to someone who knows all the right boilerplates to use. There perhaps is a market opportunity for someone on the software side to package up the complexity of processes in the operations side.
It is incredibly hard to get public institutions to update their processes. The way to do so is, sadly for us geeks, not technologically driven. I'm sure you know this, but to articulate it for other commenters: building and offering a better technological solution is not even 1% of the work. You are a technologist talking to non-technologist people. Getting people to agree on it and then use it is 99% of the work.
I don't get it... It's much, much easier to preserve records in digital format than in paper format.
Digital records are exponentially more complex to maintain in the long term. You need to deal with formats being unreadable, turnover of media, etc.
Future generations will find this era to be a black hole. All of the primary sources are electronic in formats that will be long forgotten centuries from now.
No they aren't. There are many open, non-proprietary formats that have existed for many years and will continue to exist for decades. TIFF is a great example.
In contrast, paper records cost a ton of money to store (they take up physical storage space, which means paying rent and/or opportunity cost) and they are also very easily destroyed in accidents such as fires or earthquakes. Oh, you want to create backups? Good fucking luck making copies of all the crumbling pieces of city legislation from 1960s.
Think of a catastrophe (natural or otherwise) which yielded a long-term interruption to the power grid— all those cloud-archived images and documents carefully striped with redundancy across multiple disks in different availability zones? All gone.
And yet a stack of papers in a fire-proof safe will be there and perfectly readable for centuries, with no intervention or maintenance required, and no special technology (at all) on the receiving end.
There are any of dozens ways this can happen; suffice to say that Romans in 400 AD also thought their way of life was permanent, and a few hundred years later, there was basically nothing left.
So picture a future archeologist exploring an abandoned data center populated by AWS hosts or Backblaze storage pods or whatever, attempting to recover information from the hardware he finds there. Even assuming all the disks are fully intact, how much chance would he have of recovering anything at all of meaning?
We don't know tons about the Egyptians or Romans or Aztecs from what they left us as their worlds collapsed, but it's possible we know considerably more about them than we'll be leaving for those looking back on us a millennia or two from now.
(And it doesn't even take a catastrophe— look at the effort YC is putting into getting that Alto working, and that's a machine that's just a few decades old!)
The abstracts written in WordPerfect v.whatever? Not so much.
Does anybody actually use TIFF? I've almost never seen one in the wild.
Any organization that uses a document management or records management software will have most of their documents in either (or a combination of) those formats.
So I started another software company to solve this problem: https://www.podaris.com/. The website is crap (we're bootstrapped and our priorities are currently elsewhere), but what we're developing is very, very cool, and will make infrastructure planning a lot more like using modern collaboration tools for document editing or software engineering.
Instead it's all e-mail and face-to-faces (at their high billing rates). Slows me down so much compared to my internal projects where we can have a Slack chatroom or google-docs/evernote organization system to keep track of everything.
Maybe this is it?
And speaking of, is there any way software could automate this regulation? Seems like it could save a ton of money if so.
These projects are happening incredibly fast, so I can't imagine it really being less efficient than in the 1930s, when buildings of this magnitude where few and far between.
I met with so much resistance from the local Regional Flood Control District that I finally walked away from the project (it was based on very local specifications, so the market was rather small). They ultimately "upgraded" the existing hydrologic component of what I was working on by making a web based form - but it was a step back from their desktop app (for one, it had no database, so the user is required to download and store hydrologic computations back to their client).
I have my PE but ultimately walked away completely from Civil. I understand why Civil is conservative - bureaucrats, especially those in public safety, are incentivized to always say no; such conservatism is not for me.
Another recent article on some issues of apps for Gov: http://www.nytimes.com/interactive/2016/11/13/magazine/desig...
I'm not arguing against those safeties, but I believe there can be improvement in the regulation process. The NJDEP has only recently moved to a digital permitting system.
(for example, most oil is for transportation, most of which is highway, about 3/4 of which is light-duty vehicles, which is roughly 50% cars and 50% light trucks, about 1/3 of which are to earn a living...)
via https://www.fastcoexist.com/3062630/visualizing/this-very-ve...
via https://plus.google.com/+TimOReilly/posts/DLcTGBpwZSF?sfc=tr...
Seriously, though: context is very important. You can't have a meaningful discussion if people are comparing tailpipe to tailpipe. You need a cradle-to-grave view of the system and its inputs, chained back, at least one step and maybe two. It's cumbersome, but otherwise, people keep debating with their comparisons between apples and wheelbarrows. I used to think that people who were picking a micro-location for their PoV/stance were acting disingenuously. But, I've learned that - they really don't know! (Some remain remarkably incurious once shown, though...)
Anyway: thanks for that link. It's one of my all time favorite Sankey diagrams, which I found by accident after digging after that earlier one from LLNL...
Yeah, I understand your frustration. My feeling is that people want to feel like they're doing good, and if you come along and show them that they're just pissing in the ocean... your rhetoric better be really smooth.
Which is why, truthfully, I usually don't bother. It's not my skillset.
You're absolutely correct. We used point data from BTS (http://www.rita.dot.gov/bts/sites/rita.dot.gov.bts/files/pub...), and did not filter out non-train station types.
We're fixing this now.
Mapping how the United States generates its electricity - https://www.washingtonpost.com/graphics/national/power-plant...
http://www.eia.gov/state/maps.cfm
There's a layer there for Nuclear plants too, so it is easy to adjust the map to only show them.
Not that I agree with that. A simple Google search would be sufficient for someone just looking for a location.
I'm sure there are limitations that make this much more difficult than I'm expressing due to the volume of traffic that hits washingtonpost.com - I'm more remarking on this so if other people are looking to do data visualizations like this they know there is a good dynamic option out there they can try.
Thanks for the reply!
ESRI/ArcServer is just too expensive for smaller organisations especially once you add in the cost of licensing SQL Server or Oracle.
I spent most of a year 2yrs ago trying to sell support for QGIS and PostGIS and GIS support in general to local municipalities but that died a very slow death as I had no idea how to sell to any government agency :(
But yeah, everyone who had a set up was ESRI :( Arc works with PostGIS, supposedly.
I work for an organization with several GIS groups and there is a strong push to get everyone under the IT umbrella. When I see the sums of money IT are looking to pay for ArcGIS Server/Portal/Online + database licensing I see a LOT of potential developer hours being burned up in licensing. Sadly many IT shops strongly favour COTS solutions and ESRI are the Microsoft of the GIS world....
For everyone else I had a really nice, slick web interface. You could make basic additions, edits, or lookups from it.
It would be nice to distinguish the two.
Kind of obvious, but a lot of infrastructure (esp. for resource extraction) is far away from centers of population.
Energy generation is another rural activity. Although solar is fine on roofs[1], wind is rather annoying to live next to and with the possibility of fire when those turbines go best kept away from town. Coal, gas, and nuclear power plants aren't very welcome either.
1) even then, I get the feeling a lot of architects would rather have green spaces instead.
2) ok, well turkeys as we raise them are not an outdoor animal
It was interesting to see just how notably absent windpower was in the South (excluding Texas). What's the reason for such apparent opposition to it? Curious.
http://www.nrel.gov/gis/wind.html
It's not very windy in the south, relatively speaking.
First of all, the South is quite flat (sort of surprised at other response saying it's not.) That means that there aren't ridges where you can get some easy benefit from elevation. The South also isn't very windy.
Corrosion is also a huge issue. It's humid, and on the coasts, where there might actually be some wind, you get salt too. Think of how quickly cars fall apart in regions that salt heavily in winter. Now imagine doing that year-round to machines that have to be built with some measure of precision in order to extract the most energy reasonably possible.
The only place I've ever seen windmills in the South is Corpus Christi, and I can only imagine the unfortunate people who have to keep those working. That's pretty much it for the South; you don't see many more until you get into the mountains (some in VA and probably TN.) Texas has tons of wind power, but but only in the parts that wouldn't really be considered the South; they're all in the NW part of the state.
I won't purport to be an expert, but as I've driven around, the places that struck me as having pretty successful wind farms were Central valley in CA, Eastern CO, and Southern MN and WI.
http://www.ktxs.com/news/big-country/new-wind-farm-going-up-...
Plus, wind power (traditional) does kill birds and bats so there is a problems there.
1) it does apparently given the UK experience improve the fishing
The interstates and railways through ID, WY, or NE aren't there for the people of those states - there to connect CA and WA to the east coast.
Really? How else are the people of those states going to get the things they produce (whethee by mining, farming, or otherwise) to outside domestic or, by way of seaports, overseas markets?
I was using 3g, I bet 4g is currently the best option.
Furthermore, signal doesn't mean jack. I have a great 'LTE' signal all over where I live in Austin but only get about 0.25 mbps of download speed.
Hell, I'm in the industry and not sure what I'd really get from something better than the DSL I have today.
If your DSL is 24 mbps that is not so bad, but many people are stuck at 2 mbps or worse.
In this area everybody has at least ADSL but speeds range from 2Mbit to 20Mbit. There are also a couple hundred households on Cable (max. 150/20).
With Fiber everybody will get a 100/100 connection for about 50 euros / month. Other packages are available, up tot 500/500.
Even though the people with 20Mbit and up don't complain about their connection a large part of them signed up for Fiber anyway.
Why?
Because we told them that if we fail to build this network, the people with ~2Mbit connections are screwed. Out of solidarity 65% of all households signed up, 2 years before they would be able to get it, resulting in a valid business model for the cooperative.
The U.S. Army Corps of Engineers estimates that more than 95
percent of overseas trade produced or consumed by the United
States moves through our ports.
I also would not have dismissed air freight so easily. I would think that while it sure delivers a lot less mass I would assume the parts carried by air probably are much more valuable than what's on the ships, so in terms of monetary value I would not have been surprised to see a higher share. Just think of UPS and FedEx and their big fleets.
I personally would have guessed a higher percentage would come through air than 5%.
California has been sitting ducks on some really important road proposals or last so many years despite the deaths they have been causing. My suspicion is that politicians want to push trains and other larger projects where they might get kickbacks instead of upgrading existing roads.
Here are some observations:
1. Highway 1 seems to be under repair all the year for last 3 years. 2. Highway 152 is the only major road that connects SJ with I-5. despite this it is one lane for a large part. It has very high fatality and traffic jam rate. Decades have passed since government planned to broaden it. [https://en.wikipedia.org/wiki/California_State_Route_152] 3. Panoche road. Not sure why this road exists. But since it does then making it an expressway could have helped create an alternative to 152 reducing traffic and accidents. 4. Then there is Route 130 that connections bay area to I-5. Several times it was proposed that this route be widened as an alternative to 152 but it never went ahead. I once saw a car fall several feet below the road just because he got spooked to see me coming in opposite direction. It is that desolated.
I also lived for some years south of SF, and from that vantage point it seemed like a ton of money was being put into roads too. Highway 17 (San Jose to Santa Cruz) has had hundreds of millions poured into widening and safety upgrades. There was just a few years ago a major tunnel-boring project on Highway 1 to replace the "Devil's slide" portion between San Francisco and Half Moon Bay with new tubes put right through the mountain. And the continual freeway-ification of US-101 south of Gilroy has had money going into it just about every year.
And heck, although much of it dates back a bit earlier, the Valley is the only place I've ever lived that has not only the normal three tiers of: interstate freeways, U.S. expressways, and state expressways, but also county expressways, like Montague Expressway and San Tomas Expressway. And yet doesn't have even halfway decent public transit (Caltrain barely counts).
Neither LA nor SF Bay area has any good roads. I think your are judging them by lower standards compared to population density, commute distance etc. It is barely sufficient.
Just look at this [2] data. California's rate of building new roads is significantly lower, expenditure stagnant and quality is bad. This despite the higher gas taxes. The money clearly is going somewhere else.
1. https://www.washingtonpost.com/news/wonk/wp/2015/06/25/why-d...
2. http://www.accessmagazine.org/articles/fall-1993/why-califor...
> The principal cause of declining freeway development was the dramatic rise in construction and maintenance costs
I.e. California would have had to significantly increase its gasoline tax, in line with the increases in construction costs, to fund continued 1960s-scale freeway development, but didn't do so: The gas tax today is approximately the same in real terms as it was in 1960 (it was 4.7¢/gal then, which is roughly 40¢/gal in 2016 dollars). Well that, or find a way to bring construction costs back down. A small portion of the money diverted to rail doesn't really move the needle on the big-picture here.
Southwest Expressway is the most puzzling, running just from Stokes (by Bascom) to 280. Dunno if it was originally planned for more.
Tons of money goes into VTA rail to serve very few passengers. Its routes were chosen for political reasons rather than actual demand. (Note that the MV line runs right by where Lockheed Martin is/was, and its primary Congressional advocate as head of the Transportation Committee, Norm Mineta, was a Lockheed executive after he quit Congress mid-term after losing his chairmanship.)
Now, I often get to wait at the VTA crossing of Central Expressway for empty trains to cross. Yay.
Honorable mention: San Francisco is at 13th
Using their methodology the order is: LA, NYC, Chicago, SF, Houston, DC, Miami, Atlanta, Dallas, and Phoenix.[2]
[1] https://www.oecd.org/gov/regional-policy/Definition-of-Funct...
[2] https://www.oecd.org/gov/regional-policy/functional-urban-ar...
https://en.wikipedia.org/wiki/List_of_primary_statistical_ar...
0 - https://en.wikipedia.org/wiki/Metropolitan_statistical_area 1 - https://en.wikipedia.org/wiki/List_of_Metropolitan_Statistic...
SF is artificially small due to how the line was drawn. Even the Census Region is not really correct since it considers SF and SJ as different zones, which artificially pumps up SJ's significance.
In reality "SF"-the-region includes Silicon Valley and the East Bay, and includes the non-agricultural (basically northern part) of San Jose as a suburb.
I kind of expect people will say it is San Francisco and speculate that the other is closer to the truth.
NYC's Metropolitan Division includes Ocean County (Jersey shore) but not Essex County (which includes Newark and all of the PATH trains).
The Metropolitan Statistical Area includes Pike County (not accessible by commuter rail from NYC) but not New Haven or Fairfield (which are).
These rankings are based on political borders of the cities and not the commonly perceived notions of what is a city. Honestly for me entire SF to San Jose belt must be considered as one single city to compare it with something like L.A. That is just me saying though. I might be compeltely off on this because I have never been to NY, Chi, Phili, etc.
(I generally look at urban regions that are larger entities than official borders by transit. If there are large daily people flows from area A to B, A and B are part of that region, no matter what the tax borders say.)
I have mild red-green color blindness (deuteranopia), which I think is why I'm having trouble reading a couple of the maps. For example, I can't distinguish the bridges in need of repair from the rest of the bridges. The pipelines map is also pretty difficult to read.
Better color choices, and high-resolution images, would be much appreciated. Thanks!
Sorry for this, I'll look into something to make this easier for you in the article.
In the meanwhile, we published some high-res versions of the maps on reddit, which hopefully are better for you.
Electric grid: https://i.redd.it/2nkdq2bx101y.jpg Pipelines: https://i.redd.it/mu945myovz0y.jpg Airports: https://i.redd.it/m9p0mpy2wz0y.jpg Ports: https://i.redd.it/2khvrwe6201y.jpg Bridges: https://i.redd.it/enf1rtz9z21y.jpg
Excellent book that goes over all the systems making up a city. I read it a couple of years ago but off the top of my head, it covers electricity, telecommunications, roads, public transport, three water systems (freshwater, sewage, rainwater) and how they are all interconnected.
I am also amazed at how much freight still comes into SF/Oakland and New York. Perhaps the "SF" traffic is actually Pittsburg and Stockton and I just can't see it clearly at this resolution.
As far as apples and oranges, I think that's right, 2 lines dozens of miles long crossing in the middle of nowhere is no problem at all.
Not to mention implementing policies for the addition of new infrastructure. For example adding more windmills or installing more solar plants etc etc.
Actually one can't safely say that as one can see from the 2007/8 bail out. The supposedly "shovel ready" projects weren't (if they were indeed shovel ready they would already have been funded). There was a lot of approval that had to be done and many of the jobs were for skilled, not unskilled labor, so really couldn't suck up that many new people. The economy is structurally quite different from the 1930s when the WPA could, if not bail the economy out, at least reduce the burden and get something for it.
In addition, where's the money? The current plan, AFAICT, is to sell off infrastructure (bridges, highways, water systems) in perpetuity and let the new owner charge tolls and the like. A system that hasn't worked that well in the US since the 1800s.
Any use that a terrorist could find for them can also be found by a security auditor role-playing as an attacker. If a terrorist could find a weak point in the infrastructure that may be exploited, so too may a defender, who may then devise a countermeasure, rather than an attack plan.
Those countermeasures would not necessarily appear on the maps. For instance, there may be a motion-sensing surveillance camera watching a locked gate on an access road to a reservoir for a municipal water supply. Such measures are sufficient to foil impulsive attacks, and planned attacks very often generate suspicious activity reports and probable cause for investigations well in advance of the planned event.
In theory, the maps may be a security risk. In practice, they cannot substitute for in-person investigation of specific sites, and most important infrastructure elements are at least partially protected against impulse crimes and accidental damage by chain-link fences, concrete barriers, motion sensors, and cameras. The typical terrorist candidate doesn't have satellite-guided ICBMs--they have to actually go to something in order to attack it. And they are constrained by the need for their attacks to be significant, public, and obvious.
If the existence of the plants is too dangerous for civil discourse, maybe we shouldn't have them at all, to play devil's advocate.