Not sure if this is the place you're talking about, but this sounds a lot like Portland, Maine. Portland has one of the highest restaurant densities due to being a tourist destination. It has a semi-dense downtown with a lot of very nice walkable areas. Look up "Portland Maine Old Port" to get a sense of what it's like downtown.
Unfortunately, there are relatively few places to live inside the dense part of town. The city is overly-zealous about maintaining historic districts and you'd better believe people get all up in arms about "changing the character of the neighborhood" every time a new development is proposed, even in non-historic and totally dilapidated areas. I've seen literally hundreds of millions of dollars worth of new housing and commercial projects be cancelled or indefinitely postponed due to this attitude just in the last 10 years.
While the city has a lot of potential, it's population has stagnated since the 1970s and the surrounding area has become a massive suburban sprawl. Many of the buildings downtown have been replaced with parking lots. Traffic is generally terrible, and the bus system has low coverage and poor frequencies.
Around 100 years ago, Portland was a thriving metropolis with electric trolleys going to every imaginable destination. All of those trolleys are gone now and their tracks have either been removed or awkwardly paved over.
Portland was once a vibrant inter-city train hub with hourly connections to the rest of Maine. The two primary stations, Union Station, and Grand Trunk Terminal, have been torn down. The current train hub, "Portland Transportation Center" is much further from the city center to make room for a large parking lot. It is about a 40 minute walk from what is considered the downtown area. 20 minutes by bus if your schedule happens to line up.
It's pretty disappointing how much better it once was. I'm doubtful that it will ever be as good as it was back then, but still hopeful. There are plans to move the train station back to where Union Station was. New routes to Lewiston and Rockland have been studied (although would apparently cost hundreds of millions to repair the tracks). The state has recently subsidized certain interstate train and bus routes to make them very affordable. And generally I think a lot of problems could be immediately solved by straightening out bus routes and increasing frequency, as well as obviously removing a lot of building-size restrictions and parking minimums.