The Drake Equation ( http://en.wikipedia.org/wiki/Drake_equation ) estimates between 1 and 10 intelligent life forms in our galaxy. The unmanned probe Voyager 1 is the furthest Earth born object that we have radio contact with (probably without, as well) and was launched in late 1977. It has traveled less than 2 thousandths of a light year. The nearest star (other than the sun) is over 4 light years away, and our galaxy is something crazy like 100,000 light years across. The nearest similar galaxy is the Andromeda galaxy which is 25,000,000 light years away.
In other news, I just recently finished an astronomy course.
f_l = the fraction of the above which actually go on to develop life
f_i = the fraction of the above which actually go on to develop intelligent life
f_c = the fraction of the above which are willing and able to communicate
For f_l, we've got a sample size of one. For all we know, life as we know it might be a fluke which has an infinitesimal probability of occurring.
As for how big I feel? Ohh gigantic, bigger than the sun, I contain so many cells, and these cells contain so many cells and they probably contain so many cells, I am a giant universe so giant that I have within me so many universes.
For all the stars in all the solar systems in all the universe across all time, for as large as that number may be, you start stacking together independent requirements for life (galaxy with available stable orbits, water planet, habitable zone, mineral abundance, perhaps big moon, and this is the tip of the tip of the iceberg for feasible potential requirements) and the odds against can go up fast, too.
As an example, it is hypothesized that only a spiral galaxy can support life, since only a spiral galaxy will have orbits that are not too close to the galactic center at some point. (Recently, there have been some suggestions that it may also be important not to be too far from it, either.) Right off the top, this eliminates a gigantic number of galaxies from contention. Certainly it leaves roughly an equal number still in contention (couldn't find precise frequency counts but 50% is close enough here), but even the mighty size of the Universe can't sustain an indefinite exponential culling.
We still don't know. Just because there's a really big number on one side of the question that's visibly obvious, doesn't mean that a much more subtle but equally large number doesn't lie on the other side.
(To save time, I've assumed "life as we know it". Feel free to substitute some other definition, though personally I'm yet to see a very convincing argument in favor of non-carbon, non-water intelligent life. YMMV.)
Please be clear: I am not advocating the "we're alone" position. I'm advocating the "we truly don't know" position. There's a difference.
(It may seem trite and old now, but I still think the Fermi Paradox is one of the most important questions of our time. The silliness of the Drake equation sort of got in the way, but it can be boiled down to: If intelligent life is easy and there's no terribly compelling reason why intelligent life can't at least populate the galaxy, why hasn't it? Note that "populate the galaxy" may and probably does take the form of some form of machine life, but I've seen no compelling reason to believe that's impossible yet.)
For instance he wrote some wonderfully insightful essays on the probability of intelligent life (maybe even just "l.a.w.k.i") on the grounds that most people misinterpret the products of strict evolutionary theory. He claimed that for il/lawki to have evolved one must posit the happenings of not just the simian evolutionary line, but all life and it's corresponding physical environment. If we take this literally, then we can say that for il/lawki, an unbroken line of events 13 billion years long are invested in the simple phrase "intelligent life" or "life as we know it"... and we need not restrict ourselves to any time period as it's a continuum, so we start to get an infinite on this side of the equation that divides the infinity of the universe into a simple and undeniable unity - us. To posit anything further than this is beyond the realms of scientific integrity.
That gives you another perspective--one that shows the possibility of life outside earth is pretty small.
But I wonder if there is some probability that can take both of these factors into consideration.
So if we're talking about more intelligent life forms than our own, then perhaps that probability is still small.
Who knows though.
How do we know that there aren't already "aliens" that know about us?
The time aspect also suggests that we have nothing to fear from aliens. If they've survived themselves for thousands of generations from where we are today, it's impossible that they don't have a viable and sustainable way of life, that doesn't require them to go "harvest" us :)