If all of those companies offered $400k/year, with guaranteed employment for 7 years, they would suddenly find no shortage of data scientists.
Although other industries might suddenly wonder what happened to all their statisticians / physicists / etc.
And two years later there would be a glut of data scientists who had just completed their master's degree.
The compensation has to match the value created.
First of all, you have to ask yourself why there is a labor shortage. If the underlying reason is insufficient pay, then sure, increasing pay will fix it. This is often the case for fungible talent -- incentivize something enough, and people will shift resources to it. However in many situations, certain types of work are not fungible.
There are natural barriers to entry and qualification issues. Surgeons, for instance. You can incentivize and compensate all you want, but the fact is, not everyone is cut out to be a surgeon, so you have a funneling effect. It's not even a matter of pay.
Then there's desirability issues. Deep sea welding is a highly specialized (and dangerous) trade that pays handsomely, but not everyone wants to do it.
Some physically demanding jobs also have natural attrition issues in spite of compensation. When the inflow of talent is smaller than outflow over a long period of time, a shortage results.
There's also a training and timing issues. Let's say it's 2011 you want someone who can build the kind of infrastructure that powers Netflix, in 6-12 months. The talent and experience pipeline would still have been brewing at the time, so there's going to be a temporary shortage of talent and experience until folks gain experience and mature in the field. In that instance, you can still hire and grow personnel over time, but there would still be a shortage of existing talent.
Talent pipelines take time to build, are highly dependent on talent pools available and the ecosystems around it. Just throwing money at the problem doesn't always work. Compensation is just one end of it (the opportunity end); there are also significant long-term investments needed on the other end (the cradle end, which includes education, development, etc.). Texas Instruments did this -- they funded what eventually became UT Dallas... but it took many many years and the outcome was uncertain.
There's geographic issues. To use an extremely unlikely example, let's say you wanted someone at the level of Jeff Dean or Sanjay Ghemawat, but you would need them to relocate to Podunk, Iowa. Very few people at that level would want to relocate for any amount of pay, hence a shortage. I realize this is a pretty extreme example, but I worked at a company in an undesirable part of the country and it was difficult to get truly talented folks to move out there even with significant premiums on compensation. Now you can keep increasing compensation until you get someone who's willing to move, but they're usually not the kind of talent you were looking for in the first place. Also there's a break-even point at which the compensation doesn't make sense for the value the position is likely to generate, so companies will just not hire.
My point is there are all kinds of real and complex reasons why increasing compensation alone will not solve all shortage problems.
> Very few people at that level would want to relocate for any amount of pay, hence a shortage.
I'm not sure if you're using hyperbole or not. But I'm guessing that if you paid $1 million USD / year, there would be no practical shortage of data scientists willing to live in Podunk, Iowa.
Unfortunately, the issue there is that the majority of data scientists don't generate $1 mil of value (exceptions exist of course), and so it's difficult to justify that level of compensation to management, which means the position may never get created in the first place.
Absent an existential threat to their bottom lines, companies will just muddle on without hiring.
This may be different in high growth companies, but in most traditional companies that aren't sloshing around in VC cash, to increase headcount you need to provide a value justification vis-a-vis salary (unless it's for cost center positions, but even then..)
There's also an underlying assumption that talent flocks to the highest bidder.... but most HR folks will tell you that it's more complex than that. There are many quality of life issues that come into play, like weather, peer-group, spousal happiness, etc. Money doesn't buy everything, and humans aren't optimizers but satificers.
I'm a sample of one, but for me, I would absolutely not move to Podunk, IA for a $1mil salary. I'm happy taking a lower salary living in a city and intellectual milieu that feeds me. The kind of person who would take the $1mil may likely not be the kind of talent you want.
This is already happening.
Quite literally all shortages are solved by rising price. After all, the formal definition of a shortage is "a situation where an external mechanism, such as government intervention, prevents price from rising". If the price can rise, then a shortage cannot be. Which is the case here. If you have the budget for $1m/year compensation, it's yours to offer.
Think about it this way: If there was only one person in the world capable of being a data scientist, each company in need of a data scientist would increase their offers to try and attract this person to their company. But no company in existence has an unlimited budget. Eventually all companies will be priced out of the market except for the company who ultimately employs this person. Since the other companies stop looking for data scientists beyond a certain price, the one position that remains in the market is perfectly filled with the one person available to do such work.
To put it another way: No matter how much you want one, there isn't a shortage of Ferraris because you cannot afford one. You simply aren't in the market for one in the first place. Everyone who is in the market for a Ferrari can have one. Likewise, everyone who is in the market for a data scientist can have one.
The mid-level exec with a data science poster on his wall, but a small budget for such work, is no more in the market for a data scientist than a teenager with a Ferrari poster on his wall is in the market for a Ferrari. That's just hopes and dreams. They don't count as anything more.
Your definition is a circular piece of logic defined to justify an ideology. The word 'shortage' has been used for a while. Creatin a definition that disqualifies its use is simply disingenuous.
Makes sense to me. Famine suggests that someone goes hungry, so obviously not everyone is in the market. So it is either down those who can afford food get what food is available, and thus everyone else is not in the market to buy food. Or you resort to capping the price of food and rely on something like a lottery system, or first come, first serve to determine who ultimately gets the food. Only the latter would be described as a shortage of food.
I'm not sure that living has a market, but okay, sure. If living is something you can buy, it is quite possible that not everyone will have the desire and willingness to be a buyer.
> Would submarines never be short of oxygen if we raised oxygen price ?
If you are trying to use short in an economic context to stay consistent with shortage (which, admittedly, makes the phrasing awkward), and not using definitions of short that have no relation to shortage (which would make the question pretty strange in the context of this discussion), then I am not sure why you would want to sell your existing oxygen only to buy it back after the price has gone up? But I guess you can short your oxygen if you wish. Why not?
"there's a shortage of oxygen" fits perfectly well. My point here is that your definition is purposefully narrowing the problem in order to force one solution.
Let's come back to shortage of food. Several ways to tackle it have been applied in the past, that didn't rely on free floating prices: * rationing the limited supply * increasing production etc.
There is really no reason to stop at "these people can't afford to participate in the food market, let them starve". However, your definition of "shortage" tells us that the only course of action is precisely that.
I'll just go on with my tirade a little bit further, and tackle the "increase production" point. Some people will say that by raising the prices, rational economic agents will enter the market to produce and sell more. This is yet another hint that the definition is not there to define a situation, but to hint at a larger scheme of economic agency. This is precisely why the definition is ideological; it is not meant to define a problem, but to introduce someone's opinion on how to solve that problem.
By the way, shortage can be defined as the lack of something needed. This definition does not need further ideological body to make sense, it defines every situation we have touched here, and doesn't fragment the mean of shortage in specific situations.
It certainly can fit. Context is important though. If you are twenty thousand leagues under the sea and running out of oxygen, an external mechanism (being isolated from people with oxygen to sell) is preventing price from rising. There is nobody to offer a higher price to. You could have all the money in the world and it still isn't going to buy you oxygen. This is indeed a shortage, by very definition.
But this is not the same as being unable to afford oxygen. If your submarine is sitting in harbour, with oxygen vendors by your side, and you would be wise to have more oxygen before you use the sub again, but don't have the budget to buy it, then that does not mean there is a shortage of oxygen. It simply means you are not in the market to buy oxygen.
> My point here is that your definition
Let's be clear here, it is not my definition. This is the formal definition of the word.
> By the way, shortage can be defined as the lack of something needed.
This is what a shortage is, yes. Price is how we determine need. The one willing to pay the highest price is the one who is in the most need. Those without the desire or willingness to pay a given price are indicating that they do not have the need at that price point, and thus are no longer counted as part of the market. Only when everyone is prevented from offering more money to get what they need do we say it is shortage.
Everything that carries a price is lacking to some degree. That is why it has a price: To determine who gets it and who doesn't. The higher the price, the more something is lacking. A hypothetical post-scarcity world would mean that things like food would be given away for free. But that's not the world we live in. There is only so much food to go around, and price determines who gets it, and how much of it they get. Only when an external mechanism stops the price of food from rising would you call it a shortage, however.
If shortage simply meant "lacking something needed", everything that has a price would be in a shortage situation, and that would end up being quite meaningless. After all, if there was no limits to availability for a given good or service, it would be free. There is a good reason why the definition of shortage is more nuanced.