Portland, OR has a number of these "bike boulevards". They make a huge difference. It's not every other street. But given a car-heavy street that is a major path from A to B, a parallel one block away is designated. Cars are still allowed to use them, of course, but not exactly encouraged.
Frequent speed bumps are laid down, and (just like for its partner car-centric road) all cross intersections have stop signs leading _into_ the boulevard. They're generally in good surface shape. You can roll right along at full speed, mostly just watching for pedestrians (the lack of cars also makes them pleasant for walking).
Not really possible without a mostly-grid road system, of course, but where it is, it's awesome.