> utility is an ordinal quantity, not a cardinal one. I can tell you I prefer my soon-to-be-born daughter's first kiss to my sister's cheerful hello, but not by how much (how much "what"? What are the units?).
This has an answer, though, assuming you're using an appropriately strict notion of utility function. I do gather that "utility function" is used sometimes just to mean a function that reflects (deterministic) preferences, and making that cardinal has no obvious meaning. But a utility function in the more strict sense reflects preferences over gambles -- and there are theorems[0] to the effect that, given ordinal preferences over gambles satisfying appropriate conditions, these can always be represented by a cardinal (real-valued) utility function, which is unique up to positive affine transformations. In fact, the utility functions you get as a result of Savage's theorem -- which I think makes for a better foundation, personally, as it doesn't assume the notion of probability beforehand -- are always bounded.
(I'm going to ignore the notion of "utility function" as used by utilitarians. That's largely unrelated -- a fact which has caused no end of confusion -- and is truly ill-defined.)
Now, of course, it is true that actual humans do not have utility functions -- that is to say, they do not respect the conditions required to have one. Any agent that somehow did, however -- not that such a thing can actually be built -- would be acting exactly as if it did have one, and it would be meaningful to speak of its utility function (up to a positive affine transformation), regardless of whether such a function were explicitly programmed into it. Regardless, as I see it, if economists are getting decent results by approximating humans as having utility functions, well, good for them. That's not my area though, so I couldn't say whether they are or not. (Meanwhile, there have been attempts to modify utility theory to describe how people actually behave; prospect theory[1] is an example of this.)
But as to your question -- what do the numbers mean? As I said, they reflect your (ordinal) preferences over gambles. That is to say, if you have utility function u, and u(A)=10, u(B)=1, and U(C)=0, this means that you would be indifferent between a 100% chance of B, and a gamble where you have a 10% chance of A and a 90% chance of C. Or, in other words, utility is that thing you're maximizing the expected value of (if it existed, anyway). Note that it doesn't mean that you would be indifferent between some amount of A and ten times as much of B or anything like that; the numbers are purely about probabilities. It's not clear what that would even mean in general anyway (and real things have diminishing marginal utility, you know).
Note of course that no utility value is meaningful by itself -- again, utility functions are unique only up to positive affine transformations. But if you have three utility values[2] a, b, and c, then the ratio (a-c)/|b-c| is meaningful. Although, if you're dealing with bounded utility functions, then you could always normalize so that the infimum is 0 and the maximum is 1, and then the resulting numbers would be meaningful by themselves (because you'd be implicitly setting c=inf and b=sup in the above).
This then (once we expand it a little) is the answer to your question "What are the units?". You ask this as if it were a strike against the notion, but in fact utility is more similar to conventional dimensioned quantities than you might realize. The biggest difference isn't, as you might suspect, that (as opposed to, say, length) it is nebulous and hard to measure -- it's that it's affine rather than linear!
Let's examine the notion of "length" more closely. Here's the thing -- you can't really directly measure lengths, either! What you can do is measure ratios of lengths. That's why length is a dimensioned quantity -- when we say "this has a length of 5 meters", we mean, "the ratio of its length to the standard length 'meter' is 5". You can't just measure length and get a number; you have to pick a unit.
Utilities, then, are similar, except that they're affine rather than linear -- like, say, temperature, before the discovery of absolute zero. These days of course we know that there's such a thing as absolute zero and it makes sense to measure temperature on a linear scale, hence Kelvins; but let's say we don't know that yet and we just have Celsius. This is an affine scale. Unlike meters (or Kelvins), it doesn't make sense to add two Celsius temperatures, or to multiply one of them by a (dimensionless) scalar, or to take the ratio of two of them. It does however make sense to take the difference of two Celsius temperatures, and get not a temperature but a temperature difference. (Which, again, today would be measured in Kelvins, but we're ignoring that.) It does make sense to add temperature differences, or to multiply them by scalars; and you can add a temperature difference to another temperature to get a new temperature, or you can take the ratio of two temperature differences to get a dimensionless scalar.
This is exactly how utility works. Rather than picking one reference point (like a meter), you need two reference points (like the freezing and boiling points of water) -- these are the b and c in (a-c)/|b-c|. If we are measuring temperature and we set c=freezing point of water, b=boiling point of water, then when we plug in the temperature a we want to measure, we're now measuring it in... OK, not the Celsius scale, but a rescaled version that goes 0-1 instead of 0-100. (Just as with lengths, we can only measure a/|b|; and if b is the distance traveled by light in 1/299,792,458 of a second in a vacuum, we've reinvented the meter scale.) Whatever -- your reference points don't have to be 0 and 1, that's just convenient.
So what units are utility measured in? Whatever you want, once you pick your two reference points. As mentioned above, if it's a bounded utility function, you have a natural two reference points in the infimum and supremum, but otherwise, use whatever two reference points you want. Of course frequently this is just implicit -- you have some function, and somewhere it's 0, and somewhere it's 1, and those are your two reference points. It's just not that important a lot of the time! A warning -- utility is often expressed in the undefined unit "utils", but personally I think this gives bad intuition, because again, utility is affine, not linear; I think "degrees utility" might be a better term. Of course, differences in utility are linear, and I don't see anything wrong with referring to those in "utils".
So -- while this is far from justifying the general use of the notion of utility, which, as you say, does not actually apply to humans -- I hope this answers some of your questions and shows that it's actually on a much firmer footing than you imply (at least, as a description of abstract agents, rather than humans).
By the way -- notice how when I was looking for a physical example of an affine scale, I used temperature before the discovery of absolute zero? (I mean, OK, I could have used position -- as opposed to displacement, which is a difference in positions, and thus linear -- but I wanted to keep things scalar.) The thing is that units, dimensions, are implicitly a statement of symmetry. Before the discovery of absolute zero, temperature appeared to have a symmetry (translation symmetry) that it in fact does not. When I say "You can't measure lengths, you can only measure ratios of lengths", I'm implicitly saying that if we scaled all the lengths in the universe by a constant factor, the result would be exactly the same. And that was a statement that appeared to be true when we knew less about physics. But these days, we know that's not true for speed (because of the physically significant speed c); we know that's not true for action (because of the physically significant action Planck's constant, or its reduced version); and we know that's not true for... uh, whatever the hell Newton's gravitational constant G is. And so by combining these we can see that it's not true for length, either; the Planck length forms a "natural unit", making actual lengths -- expressed in Planck units -- potentially meaningful, and not just their ratio. So when it comes to physical things, what things are affine, what are linear, and what are dimensionless, well, it depends on the physics -- in a Newtonian universe things might be different -- and our knowledge of physics has changed over time. Of course Planck lengths are not exactly convenient (and not as directly physically relevant as, for instance, c) so we continue to typically use meters despite the lack of actual symmetry; and in many ordinary cases the symmetry is still approximately there. But utility really is only unique up to positive affine transformations -- you are going to have to pick two reference values. (Though, once again, any utility you get from Savage's theorem will be bounded, making for a fairly natural choice.)
Edit: Just to clarify the above point about "in a different universe things might be different" -- if we lived in a universe where the only physically relevant constants were, say, the speed of light c, the gravitational constant G, the Coulomb constant k_C, and the Boltzmann constant k, then you could double all lengths, times, masses, electrical charges, and temperatures, and get an isomorphic universe; from inside, it would appear absolutely the same -- any actual measurement you could make would have the same result. In fact, however, the Planck constant is also physically relevant, and since this change would quadruple all actions, it would not be a symmetry of our actual universe. (And of course there are other physically relevant constants with yet other dimensions as well.)
[0]The VNM theorem: https://en.wikipedia.org/wiki/Von_Neumann%E2%80%93Morgenster... -- for people who already believe in probability
Savage's theorem: http://lesswrong.com/lw/5te/a_summary_of_savages_foundations... -- does not assume probability, constructs it together with utility
[1]https://en.wikipedia.org/wiki/Prospect_theory
[2]By which I mean values that "have type 'utility'", not necessarily outputs of the utility function; for instance, a supremum of outputs of the utility function would certainly count.