What "landongn" describes, where certain data 'score all over the place' - i.e. where the model tells you it is uncertain about some phrases, and where those phrases are subsequently manually annotated by a human, sounds like 'Active Learning'. [0]
If the model is telling us that it is uncertain about specific examples, and would like more information on examples like those, that's active learning.
That sounds different from what you describe in your post, depending on what you mean by 'sometimes the model is wrong, so we hand the data off to a human being'.
It depends on how we know the model is wrong.
If we know its wrong on a test datum, which is part of a big set of test data humans labelled without any input from the model, then its standard 'supervised learning'.
If, instead, the model is 'wrong' because it expresses uncertainty for particular test data, then, if we go and have a human classify that data it was uncertain about, and retrain the model, then we are probably doing Active Learning. In this case, the model/system is (at least partly) guiding the learning process.
Reinforcement learning is neither of these things exactly - it describes a more general framework, where the system is getting rewarded based on how well its performing.
Lets say you want to choose 1 of 5 labels for each datum.
In supervised learning, the system gets given the right label for each training example.
In a RL setup, it might be shown an example, have to guess a label, and maybe be told if it got the right guess, but if it guessed wrong, just told it was wrong - but not necessarily told what the right answer was.
There's a little fuzziness to how all these terms are used in practice.
[0] http://en.wikipedia.org/wiki/Active_learning_(machine_learni...