I can't speak for Germany, but in the United States practices vary. You can't just walk into the library at the University of Wisconsin, but at North Carolina State you can.
http://library.wisc.edu/visitors/borrow.html#wiscres
Informally, I checked the websites of a handful of state universities, and all had some way for residents to obtain borrowing privileges. It's far from a comprehensive survey, and I may be wrong in my assumption that borrowing privileges would allow normal building access. Still, at first glance it seems promising.
When I lived in Washington State, I had no particular connection with U Dub, but I could also walk onto campus and walk into the library shelving areas with no problem. When I wanted to circulate books, I looked up the state administrative regulations (it helped that I was working as a lawyer, so I knew how to look those up) and found out that a state regulation basically allows most state residents to obtain circulating privileges at University of Washington libraries for the payment of a small fee. That library has a GREAT collection of materials on east Asian languages, and I circulated some of those while I was living in Washington state.
I also don't know whether the new Hunt Engineering Library is going to be public access or not, but I hope it will be because it's going to be awesome.