Whoa so there's a possible that the visible universe is a black hole?
(Of course, we don’t actually know whether stellar or galactic centre black holes have singularities inside them - we just don’t know of any process that will prevent one from forming in current physics.)
They do according to the standard GR model. Speculations about holes not having singularities inside them are quantum gravity speculations and we have no way of testing them experimentally any time soon.
In our universe, it looks like inflation is the fly in that particular prediction: The universe is going to tear itself apart instead.
I assume you're referring to the "Big Rip" scenario? That's not considered very likely, at least according to our best current data.
So where does it go?
My intuition - which is almost certainly wrong - says it would turn back around, and be visible as a faint background radiation.
No. The universe is expanding. If it were a black hole, it would not be expanding. The calculation of how much mass has to be inside what radius in order to form a black hole assumes that the matter is static (or collapsing); if it's expanding, the calculation is meaningless.