For some libraries you have to pay a small fee to get an access card. The mentioned Grimm Zentrum/HU library is free of charge:
"The Humboldt University Library is a scientific library open to the general public. Every person over sixteen years of age and resident in Germany may register to use it in accordance with the Library's Benutzungsordnung (Regulations of Use)." => see http://www.ub.hu-berlin.de/using-the-library/registration
Some libraries limit the borrowing of books (depends on the library) to university members (usually because of high demand) but you can almost always read the book while at the library.
Of course it's not allowed to do commercial work like coding on your startup, but if you don't annoy the people, probably nobody will care...
I go there for anti-procrastination as well, it's great.
Haven't been to the Grimm yet, but I remember passing by on the S-Bahn maybe a year ago and being excited about the huge new library under construction...I'm not in Mitte very often.
I feel like in the states it really varies. Brown university and RISD (down the hill) both require a Brown or RISD ID to get in but in high school in Seattle I did all of my research for history in University of Washington's largest library which is open to the public. I sometimes wish any of these were open 24 hours though.
http://library.wisc.edu/visitors/borrow.html#wiscres
Informally, I checked the websites of a handful of state universities, and all had some way for residents to obtain borrowing privileges. It's far from a comprehensive survey, and I may be wrong in my assumption that borrowing privileges would allow normal building access. Still, at first glance it seems promising.
When I lived in Washington State, I had no particular connection with U Dub, but I could also walk onto campus and walk into the library shelving areas with no problem. When I wanted to circulate books, I looked up the state administrative regulations (it helped that I was working as a lawyer, so I knew how to look those up) and found out that a state regulation basically allows most state residents to obtain circulating privileges at University of Washington libraries for the payment of a small fee. That library has a GREAT collection of materials on east Asian languages, and I circulated some of those while I was living in Washington state.
I also don't know whether the new Hunt Engineering Library is going to be public access or not, but I hope it will be because it's going to be awesome.
It might be worth taking another look at your nearest Uni library, as I think quite a few are similar to Exeter/Durham, at least in the day and if you don't want to take out books.
That said, I might go and explicitly ask sometime, as there may be an unpublicised way around the problem.
(I can't get into UTS' library anymore; my old student card must apparently be blocked now.)