Does anyone commonly set their priors to be a distribution? Perhaps a range or actually a normal distribution to represent a prior with uncertainty?
Yes, with two caveats. First, you can't have assigned zero probability to the right answer. Which means that if you have a probability distribution over hypotheses, and the right answer wasn't one of the hypotheses in the distribution, you're doomed from the start. The Solomonoff prior is a mathematical construct that gets around this problem by including every hypothesis that can be expressed as a Turing machine, but it gets used more as a philosophical token than an actual computing tool because it's unfeasible to use directly. The second issue is that if you have a bad enough prior, "enough additional pieces of evidence" can be an arbitrarily large amount, and there may only be a limited amount of evidence available to collect. In particular, rerunning an experiment over and over can only provide a limited amount of evidence, because of the possibility that some systematic error affects every instance of the experiment.
It's been argued that frequentist analysis is essentially a Bayesian analysis with a prior distribution centered on zero with bounds from positive to negative infinity.
Imagine 1/2 + 1/4 + 1/8 ... to infinity. The sum approaches 1 as the denominator approaches infinity. With calculus, we can determine that with mathematical methods.
See also: The calculus answer.
Essentially, two genuine Bayesian rationalists (with some hand wavy preconditions) cannot agree to disagree; ie, they will eventually converge onto the same understanding of an event.
Here's how it starts: "Theorem: if two people have the same priors..."
For a finite set of evidence (particularly chosen by someone with bias), bias + evidence can be arbitrarily far from the truth.
Almost everyone does this, and the solution is a posterior probability distribution. Most uses of Bayesian techniques that I'm familiar with are based on Monte Carlo simulations, where priors are drawn from the specified prior distribution, processed in some fashion, and result in samples of the posterior distribution.
This is especially helpful when you don't view the problem as having a 'true' single answer with some 'uncertainty' but instead have actual variability in the system, which is described by the posterior distribution.
Thinking about it, I'm really not sure in what instance the prior wouldn't be a distribution. It is, by definition, a probability. I suppose you could have a singular value, where the pdf is a delta function, but what's the point of doing the Bayesian inversion or estimation then? If you have a single value with prior p(x) = 0 or 1, you should end up with the posterior p(x|d)=0 or 1.
So the simplest case is where the prior is a binary: Say p(x=yes) = 0.6, p(x=no)=0.4. Or something like that. It's not a continuous distribution but it's still a distribution. The sum/integral of all cases has to be 1.
The technical term for the condition you're interested in is consistency. There is a theorem (Doob) regarding the consistency of Bayes estimates. [1]
In simple cases it means that under any non-silly prior your posterior will converge to the truth. E.g. mean of a normal distribution with a normal prior.
The complex cases include things like models that expand when given more data.
That's the general trend: pooling data tends to make priors converge. However, converging priors isn't quite the same thing as everyone converging on the truth.
Statistical expressions themselves are an incomplete explanation, we routinely use assumptions about the direction of causality that aren't captured in them. See Judea Pearl's "Why I am only half Bayesian" paper for a discussion as well as an intro to how his framework approaches such independence assumptions.
ETA - caveat to the 2nd question: ... unless you've restricted your prior to exclude the truth.