In my own experience, my employer is struggling to find qualified people. We are a fast-growing startup in LA and simply cannot scale as fast as we would like (mail me if you want more info). There is no shortage of foreign applicants but, afaik, I am one of only two sponsored H1B employees.
If you have a GC petition under processing, and switch jobs by transferring the H-1B, at the new job you will have to start the GC processing all over again. Nonetheless, you can transfer the "priority date" (PD) from the first (abandoned) GC petition to the new one.
This piece of misinformation cost me many years, so please don't spread it. I waited too long for the "right job" to come by and wasted many years. I could have started the GC petition at the first opportunity and transfered the PD as I switched jobs.
Your last two sentences are spot on... I am guilty of it too.
As mentioned by another poster, you can switch jobs if the EB category stays the same. In that sense, you do have to wait for the "right-enough" job, where they employer is willing to agree to you categorize you in EB2 or EB1 if you have a PHd.
In addition, companies/lawyers take their sweet time preparing paperwork. Which means:
PERM application in process? Can't switch jobs from 3 months to a year or more. My PERM took > 1 year to be approved. Others I know took more time.
A H1 transfer or PERM can also be autited for no good reason, and waste time for people. An audit will discourage people from switching.
I140 in process? Should I switch and waste time invested in PERM?
You can technically switch, but there is plenty of FUD to give people second thoughts.
Unless you have a crazy budget, you will spend a loooong time interviewing and going through one shitty candidate after another. We've interviewed close to 60 people, hired 3 (NYC). One of the 3 turned out to be a rat who delegated all the work to other developers, and had to be let go.
In cities like NYC or SF, you need a very high salary to attract true talent.
Actually, there's no upper limit. We've seen some crazy examples:
http://techcrunch.com/2010/11/11/google-offers-staff-enginee...
If companies actually paid "$150-200K" there'd be an applicant line visible from space.
Worth pointing out that one of the big problems the industry has right now is the assumption that those are the only two places to put a tech company.