shas3

People in the software industry find it hard to comprehend the fact that many old-school companies actually don't require all their employees working at 150% thrust to remain super-profitable. Slow product life-cycles, manufacturing prowess, regulated markets, etc. mean that in many companies, you can have a fruitful career and not burn yourself out with 80 hour weeks. This is usually more true with non-software/non-service companies where white-collar labor isn't the most critical asset. In the same companies, employees in manufacturing, sales, etc. have it significantly harder than developers, R&D, etc.

One possible downside is that not everything you work on makes it to the market and you have to be proactive in keeping your skills and knowledge up-to-date and to remain employable for your next job.

Uh ok. You can slice up 90% in many ways. But why choose to cut it such that you lose 100% of other industries? That sounds quite effed up.

Yes, everyone wants to keep their costs low. Especially software companies because most of their expenditure is on personnel. However, in meat-space companies like pharma, chemicals, etc. plant, hard goods, etc. cost more than personnel. A senior-level manager often cannot pay $30k extra just to match Si valley salaries in an auction because they don't have so much room on personnel.

These norms are not set by individuals, but by a system of financial analysts, shareholders (think large pension funds, etc.), etc.

"vast majority" still excludes tens of thousands of people. These are real people with real families and real jobs in real companies, not just some statistical data points.

Yeah this fixation on Silicon Valley is dumb. The worst affected will really be other industries. Small wonder that Lofgren who is a Bay-Area Representative is the one sponsoring the auction system. Many Si valley jobs for which H1Bs are hired are generalist in nature. And it is for generic programmer jobs that you have potential for wage abuse. But outside Si valley, for much lesser wages, companies rely on highly specialized skills. So specialized that often these are jobs-for-life without much transferability. Think about skills in microbiology, materials science, etc. relevant to specific products in specific companies like P&G, Dow, etc. These employers and employees will be majorly shafted if legislation is made with just Silicon Valley in mind.

The whole notion of pay big $$s if the specialized skills are worth so much is wrong because these companies have smaller budgets for personnel than plant.

There also LightOn by Igor Carron and co. LightOn.io

I'm reasonably well plugged into that community. All the applications there are 'academic' applications. There are efforts to build hardware based on this- startups, etc. However, the translation is still slow and nowhere near as fast as what we saw with deep learning (just comparing apples and apples with apples defined as step-change jumps in research).

This is a very good and timely compilation of all the important topics!

I ask this earnest question because I have a deep interest in randomized linear algebra, random projections, 'sketching'/sampling, compressive sensing, etc.:

Do any of you use it in industry applications? If so, at a high level, how do you use it?

I know I'm asking a "I have a hammer and that is a nail"-type question, but I am interested in seeing "deployable" applications of these topics. I don't have any to report, other than academic ones.

Wtf is "culture exists"? Like "culture of Indians"? You're digging yourself deeper in a hole. My comment about feathers and dots was ironic to highlight your dog-whistle. India is a country of 1.2 billion people with "many cultures". In any case, tarring people of certain ethnicities or race with "cultures" IS the problem with racism.

Indian: feathers or dots?

Seriously, what's the point of mentioning their ethnicity and gender, exactly? I'm thankful that I'm not your co-worker.

John Urschel is a legit full-time mathematician. Stewart's coauthorship seems to have more to do with her skills as a director than computer scientist.

Eh? Nothing precludes grooming skilled workers! Also, obviously, new hires can be highly skilled. Someone already skilled may fit better in one company than another, etc. etc.

Bay Area is not just software. There are big pharma companies, financial firms, non-software roles in software companies, etc.

"if the salary isn't really high..."

How high? Why is 100k a magic number? 100k isn't even a big deal in SF or NYC. While a high skilled scientist working in say, Corning, NY at Corning Corp or Niskayuna, NY at GE's research labs, 100k enables a king's life. How would you adjust for experience? Perhaps a company sees immense potential and cultural fit in an employee and wants to groom them for future roles? The wages of a 30-year industry veteran would kick the promising young star out of the H1B pool, etc. etc.

Setting a wage threshold without considering other factors is unfairly simplistic.

Besides, companies pay wages based on industry and market averages. You can't suddenly expect them to peg wages to outbid competitors on H1B. Why would Procter and Gamble or Dow or Caterpillar pay university, Facebook, McKinsey, or hedge fund wages? Why should they?

Let's assume that such a person passes a triage for O1 (there's no guarantee! Suppose there are two companies using some very specific technology, first company in the US poaches an employee from the Shanghai office of the second company for his narrow skill in this technology- would he necessarily be an individual with extraordinary ability? Unique ability, yes. Extraordinary "genius"? Hard to say).

There is a significant difference between H1B and O1, which is the burden for evidence. The burden for O1 is quite onerous compared to H1B. You have to present several reference letters, meet strict criteria for excellence, and even then it is not guaranteed. There is a fair amount of inconsistency in how O1 (or EB1) cases are adjudicated.

I'm not sure about the bidding war. I've been in situations where companies are looking for people with specific skills (e.g. working with specific microbiology techniques) but would never be able to outbid what Facebook or someone in Bay Area would pay for generalist skills in another field. There is nowhere close to 100% elasticity in wages to make an auction system fair.

Also, this would handicap companies in low-cost-of-living regions of the US. This includes a massive chunk of the economy in solid economies with low costs of living like Michigan, Minnesota, Ohio, Texas, Arizona, North Carolina, and Georgia.

I think there's a big misconception here on this thread, thanks to the poorly written WSJ blog post. Rep. Issa's bill only applies to companies which have more than 15% of their workforce of 50 or more on visas. That number is calibrated to affect outsourcing companies like Infosys, TCS, HCL, etc. So, while it's fun to jump on the "foreigners shouldn't work in America" bandwagon on this thread, this bill is not what you are looking for.

https://www.americanbazaaronline.com/2017/01/06/new-h-1b-vis...

However, Rep. Zoe Lofgren's bill is a little different and does two things, replace lottery by a bidding type system and eliminate country caps, which, by introducing an artificial 8-12 year wait time for Indians, makes even non-outsourcing company employees de facto indentured labor. (So, if you are one of five experts in the world on say, mining safety, you still have to wait 10 years for a permanent residency just by virtue of where you were born.)

The answer is right there in the article. Applied and numerical mathematicians at UCLA math department have for many years been helping out Hollywood with these problems. I'm sure their students, collaborators in other universities, etc. the academic community provides a gateway to get into this kind of work.

One thing we fail to consider when talking about automation is that, at the higher end, tasks are complex and (man+machine) is much much better than machine alone. Further, the time-growth of (man+machine) will asymptotically dominate (machine). This will mean people with newer skills to master (man+machine) will be hired rather than jobs just going away. So predicting a particular job going away is a futile exercise at the higher end. The lower end is different, jobs there are simple enough to fully be automated away.

Many large company jobs are de facto jobs-for-life. But things sometimes get boring mid-way into careers.

I am honestly shocked by the "today" in "today you can take..." I grew up in India. In the 1990s a very common career path for people who didn't major in CS/EE/math/etc. was to attend non-university training in computers and programming. There were these institutes in every neighborhood and I recall at least half a dozen national-chains who'd advertise regularly in the newspapers, etc. There's at least one publicly traded company, NIIT that has been doing this for 30 years now https://en.wikipedia.org/wiki/NIIT

I don't know if it was substantially different in the US.

That's stretching it. I think I should have qualified domain specific expertise as quantitative-domain specific expertise. Afaik, MD is more about human biology and meant to help you become a practicing doctor. PhD-biomedicine, MPH/MS-bioinformatics, or MD-PhD programs tend to have more substantial quantitative components than just an MD. Caveat: these are gross generalizations. I know some MDs who are competent enough in quantitative analysis to give any engineer a run for their money!

Would an English MA or PhD, for example couple well with an ML-MOOC for natural language processing jobs? Possibly, but would involve a lot of effort to build the bridge. However, if someone has a mechanical engineering degree or a PhD in something similar, a MOOC-ML can work well to position them for a bunch of jobs in large mechanical engineering companies looking to jump on the ML bandwagon or ones traditionally strong in ML-type mechanical engineering like UTC.

But that does mean that you have pretty good C programming skills! I think your comment reinforces my claim about pairing MOOC-depth ML with other skills.

(Edit: I'm not saying MOOC-depth derisively. Serious learners and autodidacts can go pretty deep with just MOOCs for guidance. I just mean prima facie, based on content of some of the lighter and more popular MOOCs like Andrew Ng's. Abu-Mostafa's on EdX is meatier.)

I loosely used the term to refer to computer vision, which overlaps a bit with vision science (especially color, perception, etc.). But yeah, computer vision.

Frankly, MOOC-type machine learning- being able to do plain vanilla logistic regression or black-box deep learning techniques is not enough to get a job. This knowledge and experience has to be paired with one or more strengths: excellence in programming, grad-level math/stat/numerical skills/theoretical machine learning, domain specific expertise or experience (e.g. vision, audio, natural language, networks, geophysics, biomed, finance, etc.), proven ability to learn and adapt very fast.

I think the following interview questions can help screen candidates (not an exhaustive list):

1. Asking a candidate to cast a non-standard problem of classification or estimation into a tractable optimization problem. (this is a very valuable skill that someone who has done good studies in numerical linear algebra/machine learning/stat/information theory/control systems/signal processing/math/etc. should be able to do)

2. Asking them to take an algorithm they have used and explain every step in deriving the algorithm. (It will help interviewer calibrate the level of learning in the interviewee. Also helps screen for indisciplined black-box users.)

3. Presenting challenging machine learning scenarios: using customized ensemble learning approaches, imbalanced data sets, noisy labels, multiple instances, different error metrics, etc. and seeing how interviewee approaches the problem from first principles (real world problems almost always involve some of these issues)

4. Testing their intuitions in "feature-engineering" for different types of data. (with the partial exception of cases where rigorous research/successful products show the utility of deep learning, one has to almost necessarily do a fair bit of feature-engineering)

Surely it would be guarded as a trade secret, as it usually happens in large companies.

How exactly Nvidia and ARM are competitors of Intel is an interesting question in itself. Nvidia and ARM are fabless and compete against a subset of Intel's IP and products. Intel may take multiple routes to reacting against superior competition without necessarily even making better GPGPUs.

This is the first time I'm hearing that computer scientists deride engineering as not-real-computing. Any references? It doesn't check out based on my background. In fact anecdotally, I've heard the reverse. I've heard EE and stat algorithms folks criticize CS ML/CV researchers for using algorithms as black boxes compared to the rigorous standards of EE/Stat (aka reviewer standards in IEEE Transactions in Information Theory, IEEE Trans Sig Proc, majority of stat journals). Now obviously there are a vast number of CS researchers who are competent at stat-level rigor. But there are also a large number of ML researchers (applied ML?) in CS departments who use algorithms as black boxes.

Totally on point, regarding multivariate relationships! The problem is not that they do T-test (and other Stat-101-jargon blackbox stuff), but that they stop at it. To many of them, even the existence of multivariate effects is beyond their imagination.

So, inference to many biomedical folks is just 1-dimensional. Big data has a long way to go to penetrate fields where people cannot think in more than 1 dimension!