Signals of an unsustainable future coming from Davos
failedevolution.blogspot.com
failedevolution.blogspot.com
>Overall, there is a modestly positive outlook for
>employment across most industries, with jobs growth
>expected in several sectors. However, it is also clear
>that this need for more talent in certain job categories
>is accompanied by high skills instability across all job
>categories. Combined together, net job growth and skills
>instability result in most businesses currently facing
> major recruitment challenges and talent shortages, a
> pattern already evident in the results and set to get
> worse over the next five years
What I've observed is that in response to chronic talent shortages, larger organizations prefer to downsize, cut corners, scale-back offerings, and produce output of lower quality. Meanwhile, smaller organizations do not have the budgets necessary, so they try to portray working for them as a lifestyle choice. They offer foosball, on-site beer or coffee, and meaningless amounts of largely worthless equity.
What both types absolutely will not do is to compete via wages, benefits, or healthful employment accommodations (for example, private working spaces instead of undifferentiated desks in an open-plan office).
The "challenge" of recruitment is not how to attract qualified (let alone 'the best') candidates ... rather it is how to reframe the business such that high quality is not required for executives to maintain their lifestyles.
It's a massive flight from quality. It's very frightening.
> As someone who has been on the job market for a long time recently,
If you're willing to move to Atlanta, they're an option. Send me an email if interested, look in my profile. Heck, even if you're not, they're still worth talking to.
But I would also like to add that I have often been complimented by my bosses on my ability to communicate technical concepts in very entertaining and understandable ways. At one recent interview, an HR representative from the firm told me that the only feedback she received that was better than the feedback about my technical skills was the feedback that I would "mesh instantly" with the team and they all enjoyed my sense of humor. Sadly, that role was only budgeted to pay around 50% of what I had earned previously and there was no way to compromise.
I have occasionally worked in situations where a manager disapproved of my proactive nature and I need to improve my communication skills in those situations. It is very reminiscent of the type of situation from a recent research paper [0] about how extraverted managers tend to suppress the talents of proactive, introverted subordinates.
Thus far, I feel my communication skills have been one of my strongest assets and strongest attractive features for potential employers. But I do agree that I (like anyone) need to continue to work on them.
[0] "Reversing The Extraverted Leadership Advantage: The Role of Employee Proactivity" (2011); Grant, Gino, and Hofmann < http://homepages.se.edu/cvonbergen/files/2013/01/Reversing-t... >
> Thus far, I feel my communication skills have been one
> of my strongest assets and strongest attractive features
> for potential employers.
But I thank you for saying that you find my communication style defensive, angry, defiant, and self-important.
I apologize to you for coming across that way. It means I am failing to write my true feelings and I am not living up to the high standard of communication that I set for myself even in a medium like Hacker News comments. It is helpful to hear this criticism from you, even if it's uncomfortable, because it gives me the chance to reflect on what I've written and try to discover how I could communicate it in a way that still expresses my personal values (such as the failure of open-plan offices) but without giving the incorrect impressions of e.g. defiance or self-importance.
Thank you for sharing your thoughts about it with me. I sincerely appreciate it.
You were probably right, but it doesn't matter, it's not your decision.
It is the job of an engineer to present the various trade-offs inherent in different options for solving a business problem, including the way those trade-offs affect the business bottom line.
If an employer asks an engineer merely to execute a given decision, and never use critical thinking skill to evaluate the appropriateness of the decision (which is precisely what they are hired as an engineer to do) then it does represent a red flag about that manager, and is usually a strong signal that it would be unhealthy to remain working in that capacity.
In the few times when I have managed others on projects, I've always encouraged them to question my decision making and to reach agreements about our mutual efforts based on evidence, never on authority. I admit it's only a limited set of projects that I oversaw this way, but it always produced good results and the people working with me said they felt it was a positive way to be managed.
> Thank you for sharing your thoughts about it with me. I sincerely appreciate it.
you come across as arrogant and judgmental as well. If a company would rather get things done quickly instead of high-quality, that is their choice. Would you look down on me for driving a 20-year-old car because I want to save money?
We all want to work on interesting jobs all the time, but that's not what work is about. Work is about producing things other people want (otherwise no one will pay you).
In the cases where a company chose to prioritize speed of delivery, it was typically after an overwhelming number of engineers (not just me) had conducted research on the system, the business needs, and so forth, and presented quite a lot of evidence that the fast approach would actually cost the business more money than the more careful approach, even in the short term.
When I speak about prioritizing quality, I am speaking mostly from the point of view of the book Peopleware, especially the chapter called "Quality ... if time permits" in which they offer a lot of evidence to debunk the idea that there must be a trade-off between quality and speed of delivery.
In fact, in the few cases where I've been tasked with pursuing the higher-quality option, I have found that the claims of Peopleware are extremely accurate: people are often more motivated and more productive when they are asked to make something of high quality. As a result, they often actually make something that is both higher quality and delivered sooner than the alternative when they are asked to prioritize pure speed of delivery.
At any rate, I'm attempting to speak from a position where I am talking about adding value in the business's bottom line -- not at all just my personal desire for pet projects or Ivory Tower planning.
I can see how my earlier comments did not include these details and it would be easy to read my earlier comments and to think that I merely wanted to focus on quality for the sake of my own interests.
However, that is certainly not the case.
I'm very glad you pointed this out, because it helped me to see that I need to include more details about the basis of my feelings -- which is a long-running set of research results that illustrate the bottom-line productivity benefits of things like private office space and explicit focus on quality of engineering.
It doesn't matter. You were probably right in your analysis, but it's not your decision. Think of it like a lawyer does: give the best advice, then do whatever the client desires.
I replied to that other comment.
But since you brought up the comparison with a lawyer ... have you ever worked with them? I have worked with lawyers on several occasions and what you describe is not really the sort of thing a lawyer does.
If a lawyer senses that a client is willfully ignoring their advice, and particularly if that client is forcing the lawyer to pursue lines of action that are not beneficial, then the lawyer has many abilities to file to withdraw from representation (depending on the country and the type of field they are practicing in).
I've seen lawyers walk away from clients on a number of occasions, and usually all that is required is for the lawyer to say there is a difficulty agreeing on the necessary steps to best represent the client.
Lawyers have to be aware of this option because if they merely execute the instructions of their client, without lobbying for the client to use the best legal analysis to pick the best course of action, it can be damaging to the lawyer's career. So even if they can keep billing for the hours in which the client just wants to micromanage them, it's often not worth it, because it damages future earnings.
It is very much the same for an engineer. If a boss wants to micromanage (which is essentially the issue you describe -- when a boss ignores or overrides the engineer's critical thinking ability and personal analysis of the best trade-offs and instead demands that some non-optimal action be taken without justification or collaboration), that can be extremely damaging to an engineer's career and future earnings.
Putting up with it in the short term because "that's just how jobs are" and you should "just do it because you get paid for what your boss wants" is a really bad idea, even just in financial terms, because you will be damaging your chances to enhance your skills and your job experience won't cause others to value you highly. And there are other very serious risks too, like the risk of career burnout and prolonged cognitive health issues at work, that are arguably even more serious than financial concerns.
"Give the best advice, then do whatever the client desires" is terrible advice for a lawyer, and equally bad for an engineer (where 'client' is replaced with 'manager').
My primary experience is in Python, ranging from low-level stuff with llvmlite and writing extension modules in Cython, to database management stuff with SQLAlchemy, through on to tons of stuff with numpy/pandas/etc. I have about 8 years of scientific Python experience if you count grad school, and a little over 4 years of professional work experience.
My background is in applied statistics (mostly machine learning for image processing as well as a lot of MCMC methods).
I care a lot about software quality and best practices. I also have some experience in one job doing functional programming, and my best functional language is Haskell.
I think I'm most motivated by strong attention to quality, so as long as that is a heavy part of the team's ethos, I would be pretty happy working on a wide range of scientific computing / data-focused topical areas of work.
But what I find time and again is that people want me to sit in a large open-plan office with undifferentiated desks, where you can't think straight, while cutting every possible quality corner to crank out code for deeply uninteresting parochial business logic. I could happily compromise on some of those things (and every job will have its downsides, which is OK). But I can't find even one job that doesn't require compromising all of those things.
One friend of mine switched from being a developer to a DBA and said everything except being on-call more was a huge improvement: much higher pay, more respect and if he did well it made a difference: his previous two jobs were such that he would get in no way rewarded more (not even someone else noticing) for quality code versus code that implemented only 60% of the features with tons of bugs. Everyone notices if the database goes down.
I would be happy to work more as a DBA, and already have done some work in that capacity on a consulting project that focused heavily on SQLAlchemy and some database metaprogramming.
Unfortunately, my flexibility on the topical nature of my next job has not helped me much. Everywhere I interview still wants to hire people who fundamentally cut corners and de-prioritize quality, all while working for reduced wage and unacceptable benefits.
In all of my job searching, the only consistent thing I've seen is that companies want to enforce downward pressure on wages and benefits. It does not seem to matter how much quality degrades or how much market share they need to give up, so long as wages do not grow in any manner proportional to the value that a person's skills or experience can add.
The best thing you could do to improve your employability is get together with some other unemployed developers and come up with a really slick tool that leverages ML/statistics and open data to do something cool on the web.
A major influence on me is a 2011 Cato Unbound article called "Who Cares About Forecast Accuracy" by Robin Hanson [0].
I actually only found the article a few months ago, but it crystallizes the experiences I've lived through for the past 5 years perfectly.
Since the social status concerns of executives are typically the primary concerns within any organization, even if a better forecast can add value in some ways, if it risks taking value away in the social status realm, everyone will veto it.
As a result, people are often hired purely for credential (this has happened to me before). Then you are asked to do things that are directly damaging to your own career because they take you so far away from your area of expertise or your specialization that they make it harder for you to be hired by anyone else.
I have not yet encountered any organization which actually engages in fundamental research about improving forecast accuracy or truly solving problems with A.I. or machine learning.
Instead, I've only encountered places that put a thick wrapper of marketing around a thin kernel of simplistic technology, and then spend all their efforts on scaling up the marketing and sales apparatuses around the fundamentally-simple thing. (Simple things are often perfectly fine, of course. But in these situations, the simplicity was not chosen because it was a cost-effective way to meet business needs. Rather, it was chosen because it was cheap and easier to politically control. And after making the choice, no further effort was ever spent investigating whether any type of alternative could improve upon the simple thing in a cost-saving way.)
For a long time, I was very bitter about this. But then I realized that if this is the only mode of employment for machine learning researchers and engineers, it might still be OK. If the other aspects of the job afford you ways of meeting other goals, then perhaps it's OK to sacrifice the aspect of intellectual curiosity.
Unfortunately for me, the other goals I have include having adequate privacy at work, focusing on software craftsmanship and being proud of things that I make professionally, and maintaining a healthy work/life balance. And it seems these things, like intellectual curiosity, are also mostly impossible to find in modern technology jobs.
At any rate, I am not convinced that it would be a wise investment of time to conduct this type of side project. One bad outcome would be for it to embody sort of cargo-cult data journalism practices, like mining for t-stats or something and milking a misrepresentative headline for a bunch of blog hits. An even worse outcome would be to actually solve a problem really well, parlay it into a job offer, and find you have been hired purely as office decoration, to produce demoware or increase value for a potential acqui-hire or whatever.
[0] http://www.cato-unbound.org/2011/07/13/robin-hanson/who-care...
Is this an insoluble problem? Probably not, but I think you might find that a bit of flexibility will not only improve your chances of being hired, but also improve your chances of finding a place that will motivate you and bring you some level of satisfaction. Compromise on some things in the short term to join an organization where you _could_ one day be happy. Then, bring change in a positive way so that your co-workers embrace it. It has to start small and it takes time, but one day, you just might look back and realize that you were actually satisfied with what you were working on, with whom you were working, and why you were doing it. Sadly, that is probably at around the time, or after you decide you need more and then you can leave and do it again.
I found myself in a similar situation with short stints on my resume just like yours. I eventually learned that there was no perfect job, but that I had created a pretty good one over time.
For example, I remember the joy I felt when I first read the passage of Paul Graham's essay "Great Hackers" in which he describes the notion of placing software developers in a noisy open area as similar to making painters work in a factory where the air is full of soot. For me personally, working in a noisy, non-private space is simply not possible. I'd like to compromise, but it's borderline physical pathology with me. Wearing headphones doesn't work. The most significant thing I need in order to provide the business with the productivity they hired me to give them is privacy.
But at the same time, I don't like working from home. I love collaborating and having an "open door policy" to talk with and assist coworkers with anything I can. I just need a door that shuts so I can protect some of my time to use for the more thought-intensive and creativity-focused aspects of my work.
For me, there's no way to compromise on this. I've worked in cubicle banks, open-plan shared desks, shared bullpens, etc. etc. Even though I still get high marks from my boss, it drives me mad and causes me to inescapably resent showing up at work every day. Almost to the point where I'm starting to wonder if I have misophonia.
Anyway, this singular issue (finding a private office) seems like it is the single hardest thing in the software industry. I speculate this is because businesses want to confer lowered status on the activities of engineers, but whatever it is, it makes the process of interviewing pretty maddening.
I've had several places get to the point of making me a job offer, and then tell me that it is impossible for them to negotiate a private office for me. They have offered to raise the pay, add more vacation, and many other things. But I have to turn them down, for my own health. But it really begs the question... why from a business point of view is it so hard to provide some employees (who plausibly need it) with privacy. Why is employment for our field so overwhelmingly dominated by open-plan spaces, that people are willing to offer more money to attract someone but not at all willing to compromise on the type of physical space you can inhabit while working?
Anyway, for me is suggests that it's not so easy as "just compromise." There's a lot that I'd be happy to compromise on. I'll work in a wide range of languages. I can work in Windows. I can work long hours. I'm willing to live pretty much anywhere. But what I can't compromise is the need for private space ... which seems to be the one thing that essentially no one provides.
The reason I suggest a side project is not so you can develop a better machine learning technique, but rather so you can demonstrate:
1.) your ability to use raw data and ML/AI techniques to tell a useful or interesting story.
2.) your ability to plan and implement a functional, cohesive tool as part of a team, ideally in a leadership role.
You're unemployed, so aside from aggressively networking with people who have the sort of job you want to have, building a side project is the best possible use of your time.
As for the things you value being impossible to find at modern tech jobs, if you are valuable enough, almost all organizations will do things like put up partitions for you, let you work flexible hours, and so forth. You don't see CEOs sitting in a bullpen or being chided because they weren't at work at 8:30 on the dot. You probably just need to spend some more time developing yourself and "paying your dues" as it were.
My personal feelings about academia are only tangential to this thread, and far too verbose to describe here. But my general conclusion was that essentially no one should do post-docs. They are mostly only value-destructive, career-wise, and force you to delay even having proper health insurance for far too long in life. The Economist's article "The Disposable Academic" is a reasonable summary approximation of what I learned about academia.
> aside from aggressively networking with people who have the sort of job you want to have
In some sense, this is very difficult for me. I don't care that much what the topical content of a job is -- I care much more about the manner in which I work. I have many contacts who work professionally in data science, but I wouldn't want their job. Most of them just seem to be naturally more capable of making compromises about noisy workspaces and projects that don't use ML to help the business.
> As for the things you value being impossible to find at modern tech jobs, if you are valuable enough, almost all organizations will do things like put up partitions for you, let you work flexible hours, and so forth.
My experience across 4 organizations is very different. Even when a manager would tell me I was an indispensable asset to the team and they were thrilled with my efforts, they would simultaneously tell me they could not make even the tiniest concession about physical workspaces or other arrangements. I've often found that if you do not negotiate these things heavily, up-front, and get them in writing, then it's simply impossible to ever get them.
The problem of wealth inequality was predicted to occur within capitalism long ago by a man named Karl Marx. So compelling were his predictions that entire societies have been constructed (and overthrown) in attempt to create a system that is more fair. Russia and China are the two biggest failed experiments in trying to solve this problem. Karl Marx may have been wrong about communism, but he is most famous for being right about wealth inequality and capitalism.
What Karl Marx said about inequality is this: It happens because of the private ownership of the means of production. As long as the lifeblood of the economy, its vital resources, factories and technology are owned by a tiny group of individuals, as long as production is governed by the law of capital accumulation and not the needs of society as a whole, poverty, injustice and capitalist crisis will remain and ultimately grow. Workers produce everything; the ruling class produces nothing, yet the ruling class own everything.
Take for example your typical Tech corporation: Apple. Apple makes $1,865,306 of revenue per employee. The top senior software engineer gets paid around $222,550, which is a number that is far higher than the average salary of an apple employee.
What is evident is this: The people who do almost all of the work in apple (engineers) are only getting paid less than a fifth of the revenue they generate. Where does the rest of this revenue go?
It is used to increase the incomes of a few people who do much less or no work. C-level executives pad their salaries with a huge portion of this money. Tim Cook alone got 10.2 million in 2015 and that's not including stock options. While tim cook is a very capable individual, his salary indicates that his work output is equivalent to the work output of 50 engineers. Can a single human produce output equivalent to 50 engineers? I would argue no.
The rest of the money is either distributed as dividends or reinvested back into the company in the form of infrastructure improvements or just money in the bank. Who directly benefits from this? The Shareholders or in karl marx's words: The people who own the means of production. These are the people attending davos. They are the real problem, it has nothing to do with AI.
One man can outperform 50 engineers (even 10 000) and often do.
Look at Microsoft market cap after Satya Nadella became CEO. Their market cap raises about ~100 billions dollars. You could argue that he did nothing significant and it's all about engineers but after Steve Ballmer board members didn't change engineers - they change executives.
Probably 50 engineers of Apple have bigger impact on Notes app than Tim Cook, but Apple don't earn so much only because that app. It's all about vision and direction, and this is why good CEO will always earn much more than engineer
Not, it's not them. It you remove all of them, they will be replaced by other people (as nomenclatura and bureaucrats replaced the previous elite in USSR).
It's inside the system. We use money, hierarchy and laws because we do not understand each other.
If you want more justice you need to do the Judgment of Solomon:
https://en.wikipedia.org/wiki/Judgment_of_Solomon
It requires smart people and it doesn't scale.
In capitalism you get 1% of the population who own 50% of the worlds wealth which you characterized as:
"The problem with US used to be some people were richer the others"
I think the problem is bigger than that.
The paper is not a definitive declaration of what will happen, rather it is a survey of CxOs and HR managers. These people have expertise but their expertise is not predicting the outcome of AI.
That's exactly what a "robot butler/maid" is for. http://p-fst1.pixstatic.com/54652b06697ab07b10007e4a/_w.540_...
That said I do a lot of the right things for the "wrong" reasons: I recycle because I don't like wasting perfectly good resources, I take the train because that means I can code on my way to work (and I don't enjoy driving when tired) etc.
From an economic perspective, 1500 private jets (with crew and maintenance and fuel etc etc) are much better than a couple of developers and sysadmins to run Skype.
I'll believe there's a crisis with the people who say there's a crisis act like there's a crisis."
Some of them already are. You might want to pay closer attention:
http://www.theguardian.com/public-leaders-network/2015/jan/2...
Abstaining and vowing an ascetic life to practice what you preach only puts one at a disadvantage to the opposition.
Of course, the powerful who don't actually care about they solution are going to fly on jets no matter what.
Four Years is just one election cycle, so buckle-up and enjoy the ride, make sure you're ahead of the tide.
I think the history will continue to repeat itself until someone explain it with something like science (non-cultural).
I see the structure of the society like an electromagnetic field or like gravity. Something people don't know the existence.
Agents infuence the system and are themselves changed by the result. We can see that in politics.
Mankind deseperatly needs this kind of tool to survive. Philosophy or principles are not shared enough to avoid conflicts and wars. Something neutral could help.
Thank you again :-)
As long as the internet is held together by duct-tapes - I do not think there is going to be a shortage of jobs.
I always find it amusing when tech people think they can make someone else unemployed with a script.
Oh gosh - this package manager is awesome - said no one ever.
The real reason for job losses is with income inequality caused by dysfunctional govts.
http://www.theatlantic.com/business/archive/2015/07/america-...
I also find it amusing you compared the US and Russia for social mobility when the Gini coefficient is actually worse for the US than Russia at present (granted, more of an indicator of economic disparity but almost nobody thinks that high income inequality is what happens with higher social mobility)
http://www.telegraph.co.uk/sponsored/rbth/business/8046055/G...
But anyone with half a brain can start going to local libraries to get interested in subjects, attend community college, apply for new jobs, learn a trade, etc etc
And there are 10s of millions who want to lose weight. Is there a significant social issue preventing the loss of weight, is losing weight an achievable American dream... or does want alone not achieve a dream? The problem isn't the lack of desire, it is the lack of discipline.
Even if we made becoming filthy rich as easy as losing weight why would people do the former when they still can't do the latter today?
Think about it like this. No single individual can produce a billion dollars worth of output with his own bare hands. If I "choose" to be a billionaire I have to employ many people and pay them less money than their worth so I can skim the extra resources off the top. If everyone "chose" to be a billionaire who would be left to do the actual work?
Billionaires are only billionaires because they employ other people.
To be wealthy by definition means other people must be poorer than you and thus by definition if you choose to be wealthy you are choosing for other people to be poorer than you. If you chose for other people to be poor than those people never had a choice. Thus in capitalism there are Plenty of people who do not have the freedom of choice.
To some degree true (once we forget about traders, inherited fortunes etc.)
The rest of your post is bad framing I think: I think there are a lot of nice people who runs businesses.
Of course the opposite exists as well but just because someone is well off doesn't mean they are greedy; some of them worked hard, some where lucky and some played their cards well against bigger companies.