Academics, we need to talk
matt-welsh.blogspot.com
matt-welsh.blogspot.com
I'm getting kind of sick of this tactic, where this fake concession is made before going on about how academia isn't designed well enough to deliver to industry.
If authors of articles like this didn't care about the efficiency of academia to deliver them research, what's the big deal if there are a bunch of people somewhere running around in circles? Hey, at least the CS academics occasionally produce something useful to industry, which is more than I can say about other groups running in circles. Sure it could be about burning through money, or a real desire to improve the state of CS academia. But if that was the case I'd expect these articles to have categorically different discussion and calls to action. Or - at the very least - address goals of acedemia other than performing research for industry.
I've never been an academic, and never want to be, but even I gave up reading after the "(I know, crazy, right?)" line. At that point, I knew for 100% sure, that the target audience of this article is not academics. Nobody is that much of a condescending prick to the person they're trying to persuade. This is not a "we need to talk" conversation this is a "I need to talk at you, so I can show off to other people."
At the risk of sounding a little prickly with my comment - isn't it a little presumptuous to write off a certain amount of research as "wrong"? There are plenty of examples of research that didn't have a clear purpose leading to breakthroughs - safety glass, microwave ovens, and Teflon all spring to mind.
Also, focusing on efficiency doesn't really align with academia's purpose, which (to my mind) has more to do with fundamental research. This article about Xerox PARC springs to mind:
http://www.fastcodesign.com/3046437/5-steps-to-recreate-xero...
> It drives me insane to see papers that claim that some problem is "unsolved" when most of the industry players have already solved it, but they didn't happen to write an NSDI or SIGCOMM paper about it.
If it was easy to know when you make incorrect assumptions, nobody would.
I find this point rather condescending. What you term "incorrect assumptions" are often necessary simplifications that reveal the core of the problem at hand. The point is to try and understand what makes problems hard and to develop new ways of solving them; not to deliver to companies like Google ready-made solutions that cater to their every operational need. Don't like that? Too bad. Fund your own research.
In my opinion the scientific authors doing the work are in the best position to judge what assumptions are and are not appropriate for their work. If the science is bad we expect the community to pick up on that via peer review or in a subsequent publication.
That sort of contextualizes your post as a long winded statement of "My job is hard".
This dogma is one of worst thing happening to research.
I don't think anyone would disagree with you when you say "We should rather find and implement a model that leads to best research results possible." Viable alternative models are what I am looking for.
Basic research cannot be so easily evaluated, often it is decades, sometimes centuries before the practical use is found, if ever. Basic research, by its definition, has unclear significance for anything except researcher's curiosity and intellectual fulfillment. The results in the form of a paper are often too remote from the needs of society and often so intellectually involved nobody except few people competent in the field can judge whether the work is even meaningful, far from judging its usefulness.
The current evaluation process for basic research is that grant committees don't even try to evaluate the usefulness of the research requestors have done. They use superficial indicators like existing publication score, academic rank and history of the requestors, relation of the work to other high-profile trending topics, association with popular research groups and political considerations.
That's why I think people who want to do basic research should not be funded by their colleagues in field directly from tax money. It's too opaque and ridden with corruption.
In this case, please trying to take shortcuts towards their goal of improving the amount of publications and grant approvals. There is reward in the system for this kind of behaviour. You being a reviewer/jury position unfortunately do not have the luxury of a filter.
Is there a way to early catch this , by looking at past trends ?
Slight detour is that this is one of my rationale for spending time reviewing papers for journals/conferences. In average, only 10 to 20% of papers I review really stands out or appeal to me, which is correlated to acceptance rate of a top journal/conference.
I like your post because I think it is the duty of academics to communicate their results for maximum positive effect and to represent real world impact accurately, but I also have to ask what can industry do to expose itself to excellent research which is sitting unused?
That is not the point of academia, and indeed not the point of why academics are offered tenure with their institutions. Please consider Galileo and Copernicus and the Church of the 15th and 16th centuries. Please consider history.
Google as a business may care about research being aligned with its business, or what 'society' or certain segments of society care about; Google really should not care what academia and academics want to work on. That is the point of, for better or worse, academia: independent research by some of our smartest people.
If Google, and industry, care about what gets research, they should fund it. They then can choose. Please leave academia and academics to be just that, academic.
Without government funding, support, and indeed the wider academia 'ecosystem', my father would have never been able to be a historian of the Scottish enlightenment, and producing the seminal book on Adam Smith. And if you don't think that the Scottish Enlightenment is important to our modern understanding of the Universe, then I suppose that you then have to rethink the contributions of these notable academics, scientists, engineers, and philosophers: David Hume, McLaren, Taylor, James Watt, Telford, Napier--off the top of my head.
My point is that my fathers book may not itself be a major contribution to a current revolutionary idea, but it will likely be part of some future realisation about economics, since Adam Smith is a major cornerstone of our current understanding of economic thought.
Click "Threads" and you will be presented with a list of your recent comments. Anything that is still editable will have an "edit" link. I am not sure how long this link is available but its definitely longer than an hour.
It is kind of similar to what YC and VC's do: give a little bit of funding to a small number of businesses, but hope a (very) small number will pay off big.
Academia is a small investment to a number of ideas, with the hopes of big pay-offs in a very small number. The technology behind the Web was invented by Tim Berners-Lee while at CERN in the early nineties. You might not think that funding in high energy and particle physics would be worthwhile, but yet something completely different and revolutionary for all of humanity came out of it.
We just do not know where these revolutionary ideas come from, but what we do know is that they tend to come from some of the very brightest people.
Much like how YC often says that the best startup founders aren't necessarily the smartest people in a room. Occasionally, being the very brightest can even be counter-productive because in the end the people who are less smart and realize it will strive to work harder.
And that's what research really is, really hard laborious work. Not this fantasy of someone bright sitting in a chair thinking up discoveries. That's why research is hard.
[Citation needed]
There are such things as talent, insight and experience but intelligence makes everything easier, from learning new fields and tools to evaluating and formulating new ideas. Ceteris Paribus more intelligence is better.
I believe you presented a logical leap between having all the tools one could ever dream of <-> delivering scientific impact.
Here's a citation: "scientific impact is a decelerating function of grant funding size".
http://journals.plos.org/plosone/article?id=10.1371/journal....
The whole point of my blog post is this: Most academics are trying to do work that is relevant to industry, but many of them are going about it the wrong way. Nobody is saying you have to work on industry-relevant research, but if you're going to try, at least do it right.
That is certainly not what most academics think.
Under this premise, investing in the arts is a waste of money, when you could invest in UX studies. Investing in the humanities is a waste of money, when you could invest in data analysis. Hell, why even have academia at all? Wouldn't it be better if education was managed and provided directly by the industry?
This fetish for optimisation only ends up hurting the sense of humanity in all of us. Not everything has to be done with the purpose of being disruptive, or in a frenzy for efficiency. Some people are just goddamn curious about bees and will study honey bees (as useless as it may sound) because understanding nature is a goddamn delight in its own right, even if it doesn't yield the next billion-dollar idea.
That innate delight and curiosity is where science is born. Let's appreciate that. It applies to STEM too. Maybe some academics don't care about your problems or in fact solving any problems at all, and decided to be in academia not for the "potentially-groundbreaking research" but just the thrill of finding new stuff and uncovering the beauty in the universe, don't you think?
"Industry focused" doesn't necessarily mean it's looking to push the needle of industry much yet. Often the first paper academics and students do in an area will really be about trying to take an industry problem into the lab, to see if they can prod it and discover what's interesting about it. Not yet about trying to push their research out into industry. Little experiments are how we get to look at new problems for the first time when we're not close enough to buy you a beer. And little papers are how we get to start discussing what we're doing with our peers at other universities. In a sense, where start-ups have Minimum Viable Products, PhD students and academics have Smallest Publishable Activities to start exploring a new area ;)
Sabbaticals and internships are a lovely idea, but it's increasingly hard to get universities to give leave to do that. (Backfilling an academic's teaching can be surprisingly hard to arrange, and then there's the academic's service roles for the university, their PhD, masters, and honours students, etc) So I expect most academics will always have many more research interests than sabbaticals / placements.
Ok, those are the constraints - on to the value:
Most universities do focus fairly heavily on industries in their area. (eg, agriculture in regional areas). But it's not a good idea for all their research to focus on that. If we teach in an area, we want to be at least somewhat research-active in it, as that's how we keep teaching from stagnating. And in lots of countries, students find it too expensive to move far from home, so they study at a uni they're in the catchment for. So unless we want to shrink the future pool of employees to children who grew up in academic-beer-buying-reach of a company in that industry, we really do need some research happening at universities outside beer-buying distance. Which means you'll get a bunch of "bad industry research" papers as they first explore the area.
You might not want a new multi-hop algorithm. But a graduating masters student who's designed their own multi-hop algorithm with different characteristics, and undergrads having had some exposure to the problem...
And sure, the interaction design student might only have developed their app for one mobile platform, not several, and only tested it with a dozen people not a million. But really that's because we figure it's probably sensible to find out if it was even worth writing for one platform, before we spend so much of that government-funded time on writing it to work across several...
There's quite a few things industry, such as Google, could do to help if you're keen to make the match between research and industry closer. One is to disseminate your problems, not just your libraries.
If Google emailed "here's a good small problem and project we'd be interested in" (a different problem to each university), I doubt there is a CS department in the world that would refuse to offer it as a project for their honours students. (Which means the faculty who supervise the students would be looking at the problem with interest too.) The academics would just need to know it's a unique problem for them -- that they're not being asked to compete with an unknown number of other honours students and academics around the world on the same problem (which really would be inefficient as well as unfair on the students).
If you want to work on problems irrelevant to industry, go nuts.
If you want to work on problems relevant to the industry, it helps to double-check that your problems are relevant to the industry.
The desire to work on problems relevant to industry is coming from academia, not the author.
The department where I am studying is big on graph theory and other theoretical CS concepts, and I always feel a bit uneasy when we write or read about applications (on a grant proposal or in a conference paper, usually) and yet none of the authors really cares about applications. They usually care because "it was an interesting problem that some people studied in the past and we can do it better than them".
Yet, as far as I know, nobody can really write that sentence unless it is such a fundamental problem that its usefulness or importance goes without saying. So we are now in a situation where everyone (theoretical CS, applied theoretical CS, and practical CS) pretends their work have actual applications, and I feel Matt is calling them out on it.
The point the article tries to make is that if you're trying to research something relevent to industry, it's probably not nearly as valuable academically as something more exotic would be, so you'd better make a good effort to make it practically valuable to compensate.
Nobody is that much of a condescending prick to the person they're trying to persuade.
In context the line didn't appear condescending at all as I read it. A little sarcastic, that's all. The author made it clear that he doesn't think academics need to produce something that's actually used by consumers. And the author was/is an academic. If you'll allow me to speculate, it seems that you don't have the highest opinion of academics and so it seems possible that you might further think that in order to be valuable they need to produce a product that is economically valuable. If you read the "I know, crazy, right?" line with that attitude, I can see how it might appear much more condescending than it actually is.
This is not a "we need to talk" conversation this is a "I need to talk at you, so I can show off to other people."
This is a non sequitur. "This article isn't trying to persuade academics, therefore it is trying to promote the author's social standing at academics' expense". The author identified a problem and offered a potential solution, does that mean he also has to persuade? Your statement presents a false dichotomy, that he has to either persuade academics or shame academics to promote himself. I say that he doesn't make an attempt to persuade beyond identifying problem and solution because he doesn't want to bore the reader with rhetoric, nothing more nefarious.
(I work in an industry research lab, write code for a real product, and publish academic computer science papers.)
This +1, er +1,000,000.
I would firmly state that academia by definition is not supposed to 'deliver to industry', and not even to society. The whole point of academia is to give the very smartest people (whatever that means) the freedom to explore ideas, so that a some point humanity may benefit from their ideas, experiments, and research.
My point is that it could be any number of generations of research from the current generation of (what seems like quack) research (to the establishment) will turn into being instrumental to our understanding of the universe. Galileo Galilei (and Copernicus) for instance. Their ideas (and research) did not sit well with the establishment of the Church (which was pretty much the central authority of everything at the time), but their ideas are now central to our understanding of our Solar system.
edit: formatting.
Students spend their money and time to learn something and frequently they get worked like a racehorse and nobody cares about if what they learn is transferable after the academic system no longer needs them.
If you are not going to be relevant to society the only thing I can do is vote for somebody who is going to cut your research budget.
Relevance is a very broad term, with industry a subset.
That's very insightful. It's easy to lose sight of things and be reduced to a particular worldview. The Higgs boson doesn't have to have commercialization potential for us to justify spending a few billion and a few decades discovering it. :) It's an end in itself.
Government and business and society should not be fettering independent research, just the same as they should not be interfering with law courts. The point of independent research is for the independent researchers to have the freedom to follow their research even though it might not be popular with the local Wood Pulp mill pumping dioxins into the local river, and that is pollution is going to cause cancer. This research may not be popular with many in the community because it will impact their jobs. This research may not be popular with the government, and elected officials because it will affect their tax receipts and donations. This research my not be popular with you because it is going to put your family out jobs, but it is important research, we can ultimately make informed decisions.
That is the point of research, information, and knowledge--so we can make informed and reasoned decisions.
I just can not believe the attitude and entitlement of some who think that just because they pay tax they think everything that is not obviously useful to themselves is useless.
Having organized education that is relevent to the real world is an exellent idea. However, reforming schools is not the appropriate approach. We live in a capitalist society, if they dont like whats around, they should compete. Im confident people are more than willing to pay 20k a year for a guarenteed job. Code boot camps charge that much for inadequate preparation and people are still clamboring to get in there.
Edit: i agree with you, im just so passionate right now. I need to relax
Having schools that don't teach for the sake of fostering knowledge and critical thinking but rather just train you to serve in a specific industry/company would be the final blow to higher education in the US.
That, and also, academia is several thousand years older than industry.
There aren't 2 different kinds of research, kind A that is never gonna go anywhere and kind B that is visionary. The vast majority of the advancement of knowledge has been what academia has always valued: probably never gonna go anywhere, but might push the envelope by an inch or a millimeter. The whole reason you have to research a thing is you don't know whether it's gonna go anywhere.
Which isn't how we think in industry, not even in industry R&D. We have to know it'll probably to go somewhere, or at least have a small chance of going really far. That's the only way to get a profit in expectation.
But work that will probably go nowhere or even if it does go somewhere, won't get very far, like most of scientific progress? The point of systems like tenure is to foster that, since industry can't.
But I do think we can separate "kind that is incremental improvement on existing systems" and "kind that is visionary change of existing systems". This is particularly true in computer systems research. It's common to see a paper where people tweak a small part of an existing system; I consider that incremental research, and it is necessary. But the value of such incremental research is dependent on the assumptions made by the researchers. Often, those assumptions are informed by impressions of what is important to the wider field of computing, and the author is saying those impressions are often misguided.
You can argue, quite rightly perhaps, that focusing on impact/industry applications is a bad idea, and that we will miss out on important epochal discoveries, but that is a problem with society, not the author of the blog post.
Academics are in an insanely competitive environment, where what is rewarded is bringing in grants/high impact publications. There are a very select few academics that are so brilliant and have such sterling reputations they can afford to not play this game (like Matt Welsh's former advisor) but most young researchers don't have this luxury.
For example, Peter Higgs, the Nobel prize winner who postulated the existence of his namesake Boson, flat out said that: "I wouldn't be productive enough for today's academic system" (http://www.theguardian.com/science/2013/dec/06/peter-higgs-b...). He spent several years of quiet research without publishing anything to develop his theory - a young professor doing the same now is unthinkable. The most highly successful young scientists I know now are incredibly career driven and optimize ruthlessly for the kind of output that tenure committees are looking for.
Basically, if you want researchers to incorporate best practices (tests, version control, well commented code, etc) and to actually attempt ambitious longterm research programmes, make sure that's what you reward, and remember you cannot just reward success. By definition, something ambitious has a high possibility of failure - if failing means that your career is destroyed, then people won't do it.
https://news.ycombinator.com/item?id=10141736
I had heard about BS in academia where only paper output, grants, citations, and so on count. That Anti had to fight to get them to care about his work being implemented speaks wonders about how this works out in practice. Had he only cared about academic success, he could've dropped some light technical details and graphs in the paper then been done with it while the idea collected dust. Many benefited from him fighting the tide on it to produce good papers and an implementation.
I'm not in academic circles but I bet many face the same battle. With the pressure, it might be impossible for them to produce the desired output on their own within their constraints. Or so difficult many give up. Perhaps we should encourage them to have one good line of research they string out over years for quality and ensuring delivery while doing lots of nice papers in between to keep institutions happy. Think that might work?
Of course, if your goal is that Google adopt your new system in their data centers, then you need to know what they already do. But the problem with that model of research is the initial goal, not the way it's currently executed.
For example, I'd love to improve search relevance, but w/o having access to Google's search engine to build on, it's pretty hard. That's my suggestion. :-)
Google recently open sourced its TensorFlow plstform specifically to enable researchers (and others) to build upon and improve it -- trying to avoid the problem with MapReduce (where a bunch of clones came out that were, at least initially, inferior to the original).
And again, these are all academic projects that aim at being practical. The point is supported by a number of academics that incorporate real-world information and constraints into their work to produce deliverables that advance state-of-the-art and are useful. Examples that come to mind are: Haskell/Ocaml/Racket languages, CompCert C compiler, Microsoft's SLAM for drivers, SWIFT auto-partitioning for web apps, Sector/Sphere filesystem, the old SASD storage project (principles became EMC), old Beowulf clusters, ABC HW synthesis, RISC-V work, and so on. So many examples of academics keeping their head in the real-world instead of the clouds to making a name for themselves with awesome stuff with immediate and long-lasting benefits. I'm not Matt but I'm guessing he'd rather see more examples like this than, say, a TCP/IP improvement that breaks compatibility with all existing Tier 1-3 stacks and whose goal is to improve overall Web experience. Yes, there are people working on those. ;)
It's aimed at a real world problem but their solution is not good.
A couple of days ago, someone asked where the verification infrastructure was on https://news.ycombinator.com/item?id=10831601 . So I took another look around and found it was pretty much unchanged from when I looked last time. There is almost nothing there. It is not up to industry standards, to put it lightly.
It's not just the verification aspect that is weak either. On the design side, they only have docs on the ISA. For SOC work, you are essentially given no docs. Then in another slap in the face, the alternative is to look for code to read but the code is in Scala. Basically only helping those who went to Berkley or something.
It is something that seems relevant but if you were to try using it most engineers would have a pretty hard time.
Them not having every industrial tool available doesn't change the fact that the research, from ISA design to tools developed, was quite practical and with high potential for adoption in industry. An industry that rejects almost everything out of academia if we're talking replacing x86 or ARM. Some support for my hypothesis comes from the fact that all kinds of academics are building on it and major industry players just committed support.
Is it ideal? No. I usually recommend Gaisler's SPARC work, Oracle/Fujitsu/IBM for high-end, Cavium's Octeons for RISC + accelerators, and some others as more ideal. Yet, it was a smart start that could easily become those and with some components made already. Also progressing faster on that than anything else.
They haven't followed engineering practices which is one of the issues mdwelsh was talking about.
If they've synthesized to 45nm and 28nm, where's all their synthesis stuff - constraints etc.?
They have no back end stuff, very little docs, almost no tests, almost no verification infrastructure.
Any link you have where people mention these and any other issues?
Someone name fmarch asked on https://news.ycombinator.com/item?id=10831601 about verification against the ISA model.
It possibly can be done via a torture tester apparently, https://github.com/ucb-bar/riscv-torture , but taking a quick look I don't think it handles loops, interrupts, floating point instructions etc.
Doesn't seem nearly as thorough as what I've read in ASIC papers on verification. They did (co-simulation?), equivalence, gate-level testing, all kinds of stuff. Plus, you did it for a living so I take your word there. I do hope they have some other stuff somewhere if they're doing tapeouts at 28nm. Hard to imagine unless they just really trust the synthesis and formal verification tools.
The flow is here:
http://www.cs.berkeley.edu/~yunsup/papers/riscv-esscirc2014....
Are those tools and techniques good enough to get first pass if the Chisel output was good enough to start with? Would it work in normal cases until it hits corner cases or has physical failures?
The torture test page said it still needed support for floating point instructions. That kinda says, they did no torture tests of floating point instructions. I wouldn't be happy with that. Same goes for loops. Etc.
You have to think about physical failures as well: the paper mentions various RAMs in the 45 nm processor. You should have BIST for those and Design For Test module/s. Otherwise you have no way to test for defects.
An example would be putting Racket to use on industrial scale projects with groups of programmers from different backgrounds. These would discover any pain points of language/tooling plus opportunities for improvement. Doesn't have to be Google: just practical, diverse, and outside Racket's normal sphere.
The reason I used Racket as an example is that they already do some of that at least in their forums. Maybe some commercial sector but I lack data on that. They've evolved to be constantly more useful for both academic and practical stuff through such feedback.
If you doubt that or that they're purely academic in a bubble, then feel free to share why. You may have data I dont have but Ive seen them adapt based on external feedback and usefulness in specific cases. Not a Racket user or insider, though.
But what you're suggesting requires persuading a large group of developers to adopt a new language -- if you have a recipe for doing that, lots of people including me would love to learn it.
"But what you're suggesting requires persuading a large group of developers to adopt a new language -- if you have a recipe for doing that, lots of people including me would love to learn it."
What I'm suggesting is a group of people interested in trying something and reporting the results try something and report the results. You don't have to convince anyone of anything as you're responsible for you, not them. :)
All you'd have to do is make sure the tutorials/guides, tooling, core libraries, and distribution are in order. Optionally a little bit of evangelism for awareness. Random person on forum or HN: "Hey, I'd like to try to write some business code or some service in a new language. What should I use?" Drop a Racket link, guide, and something on macros & live updates (if racket has it). I remember loving those when I played with LISP back in the day. I wouldn't expect any more from the Racket team.
Now, I'll drop Racket references in these tangents if you're saying you all sat around in Plato's Cave with no exposure to real programming past its shadows in academic papers and just came up with everything in Racket on your own. Just seems like there's some feedback loops in there from useful projects that caused improvements that make it more useful in practice. If you say no, I'll concede I'm wrong given you're part of the core team. Then be mystified at its evolution.
And far be it from me to encourage you to stop mentioning Racket! But I think fewer of the academic projects you mentioned than you think were developed by people based on industry needs. Instead, we Racketeers are all software developers, and we make Racket the language we want to program in. The most significant Racket application at the beginning (and maybe still) is DrRacket, the IDE. Developing that has led to everything from FFI improvements to contract systems, just as an example. I expect the same to be true for many other real working systems developed by academics.
So, I issue a retraction that's the opposite of my prior claims: Rackets features, libraries, and tooling were all developed by academics or Racket community with feedback or justification from real-world projects (eg web servers) or use in industry. Purely Racket community working within the Racket community on day-to-day, academic or leisurely needs.
I'll revisit the others on the list to see which of them might be the same.
"And far be it from me to encourage you to stop mentioning Racket!"
I wouldn't anyway. You all have earned mention with the right mix of attributes in the project. :)
"Instead, we Racketeers are all software developers, and we make Racket the language we want to program in. "
That makes sense. Scratching an itch as the old FOSS motto goes. I did the same thing with a 4GL a long time ago. I understand the motivation. Staying with such a project for many years is an angle I apparently haven't caught up to. Regrettably. ;)
"The most significant Racket application at the beginning (and maybe still) is DrRacket, the IDE. Developing that has led to everything from FFI improvements to contract systems,"
That makes sense. It looks like a very complex program. It would stretch the language into programming-in-the-large and robustness territory by itself.
"I expect the same to be true for many other real working systems developed by academics."
I'll keep it in mind. Thanks for your time and information on the Racket project.
http://research.microsoft.com/en-us/um/people/simonpj/papers...
Also, failure to address the general case is not so bad--it just means that the next part of the general case has to be addressed by the next researcher.
Finally, I think the real issue is academics who have an idea, and cloak it in pseudo-relevance to industry to sell it. A program analysis framework isn't suddenly industry-relevant now that it applied to JavaScript, and we should just be ok with not chasing the latest industry fad.
Over time, fields of study become industrialized. There was a time when doing research in computer vision, machine learning, and speech processing was risky because the field was new, difficult to enter, and the prospects for commercialization were slim. That time has passed. Those 20 people working at Google are the people that helped that time pass. One could argue that the place for this work is now in industry - the motivations are all right and the resources and data are aligned to carry the work forward at a rapid pace.
This happens in other fields. For example, there's some word on the street that DARPA is going to stop funding so much basic research into applied robotics. Industry, they say, has got this covered. You can argue that they're right. The commercial sector is starting to get real thirsty for robots. Amazon talks about automated drone delivery. Everyone talks about self driving cars. The military wants to buy automated planes as a purchase, not as a research project. The time for basic research, it seems, is over.
As far as I can tell, this happened with systems about fifteen years ago, so the academic activity you see in systems is what is left over after all of the researchers that could do things moved into applying their research in industry. You no longer need to have weird hair and be buried in a basement to think about 20 computers talking to each other in parallel - you can go work at any technology company and think about two million computers talking to each other in parallel, and get paid two orders of magnitude more money. So the people doing systems research in academia are the people that cannot take their systems research into industry. If they could get internships, they would, and then they would get jobs. They haven't.
This couldn't be farther from the truth. Your idea's are generally credited to the company, which in turn is credited to the CEO or some other high up. On collaboration, it's only more collaborative within a given company, and not always even then. Between companies it's outright hostile to collaboration by definition.
Depending on the managers and team leads involved, that kind of thing can also happen when promotions come around. At every place I've worked, a common complaint is that the TL for the project got promoted despite not doing much because they were TL. Of course that's not supposed to happen, but it happens all the time.
The post seems to compare the worst case in academia vs. the best case in industry. You could just as easily flip things around and make industry sound bad.
This is like when Tim Bray mentioned that Amazon is great because he hasn't experienced the same problems that have gotten a lot of press lately. Of course he's treated well! He's Tim Bray!
Matt Welsh is exactly the kind of big name that isn't going to lose credit on something. Of course he gets credit! He's Matt Welsh!
It's unfortunate that a "lessons learned" paper summarizing a sabbatical in industry doing customer-facing work would not be publishable. Surely it's far more useful to other academics than most papers. It'd definitely be more broadly relevant.
My wife is a professor in a practical field and I'm always sad to hear what counts as a "good" paper or a publishable article. The big journals in these fields drive the notion of what is and isn't legitimate research. That's the point where what constitutes career-advancing "academic output" has to be changed. But I'm not enough of an insider to have any idea of how to go about doing that.
You have mentioned elsewhere that you are looking into how Google can help the academic world focus more on the right industrial questions, and you personally recommend such sabbaticals/internships -- so this seems like a natural step.
But maybe that's not the general case, but as someone coming from the tech sector to get a degree (and not the other way round) this sounds all too similar :)
> It drives me insane to see papers that claim that some problem is "unsolved" when most of the industry players have already solved it, but they didn't happen to write an NSDI or SIGCOMM paper about it.
I've seen many examples of industry "solutions" that aren't documented, aren't published, and aren't even validated. There's a place for papers like these. I'm not quite your typical CS researcher (I do applied math and software for medical imaging), so YMMV, but I think this criticism is too harsh.
"Open" is a different league from proprietary. It doesn't matter that they are behind proprietary. but it matters when they are working on the wrong problems.
Having been bitten by the 'but we already know how to do that' comments, I find this particular aspect of industry to be very irritating. There are 3 possibilities:
The unpublished solution is brilliant.
The solution fits for very limited constraints applicable only to that situation.
The solution is a half-assed hack that only looks like it works.
In only one of those situations is the 'naive' comment valid.
There aren't that many professor jobs out there. It's unbelievably greedy to be taking one up to do industry's dirty work.
You can always take an afternoon off a semester here and there to be an adjunct and teach a SE class or give a guest lecture.
Many capitulate altogether, trying to do industry work from their academic position. It is these last people who should get out.
The creation of the internet worked wonderfully. It was invented by academics based on government grants when it was basic research, and then refined and turned into something practical and workable and, most importantly, profitable, by industry. Exactly as it's supposed to be.
The companies wouldn't have done this work, at least outside of their research labs, because the solution's theory is too far from the need of any one problem. But the result -- a more memory-efficient hash table design -- turns out to be broadly useful.
And yes, I do consider this to be "industry-relevant" research. I'm not going to solve their problems for them -- but there can be great synergies between industry and academia for having broad impact through adoption.
(full disclosure: I'm an academic on sabbatical at Google for the year. It's likely I'm a little biased in my belief that both have value. But this isn't a bias unique to me; systems as a general area is close to industry, and most of my colleagues rotate in and out of industry periodically via sabbaticals or startups.)
My PhD advisor never seemed to care particularly about publishing papers; rather, he wanted to move the needle for the field, and he did (multiple times).
Racking up publications is fine, but if you want to have impact on the real world, there's a lot more you can do.
Other academics, for example those doing "stuff going way beyond where industry is focused today" as the author explicitly states, can safely ignore it.
In my experience, this is only true until there is money to be made. Or more specifically, that industry was more than willing to share credit, but ownership was theirs.
Anyone who has ever been through a conference/journal submission process knows this pain. You can usually tell from the comments which of the reviewers is working in your field and wants to shut you out.
Therefore, I don't think Academics should stop doing what they do (i.e. wander around) and have a laser focus on Industry's product-based researches.
Are there a bunch of protocols that I don't know about?
Are you maybe referring also to centralized path finding algorithms? This would explain the comment.
Apparently the author was unable to break that habit.
Imagine being someone who thinks that Capital could decide what is a good problem to work on...
But it's not just the hard & expensive stuff that they are resistant to. Even the "easy" stuff (like adopting a programming language designed by the professionals instead of the amateurs) they won't do.
I build interactive proof assistants. But I'm not pushing their applicability to industry, and I don't expect them to be relevant to industry (for a long time at least). Why? Because it's too expensive. Formal verification in type theory MAKES NO ECONOMIC SENSE; ask anyone who's actually ever done any industrial verification, and you will find out what tools they are using, and it has nothing at all to do with the area of research I'm involved in. This is because there are inherent trade-offs in every technique, and industrial use-cases tend to prefer a certain set of trade-offs, and I prefer a different one.
But it's a fascinating topic, and something that I'm preparing to devote the next several years of my life to. And I can safely say that a meteorite will more likely destroy Manhattan than will any of my computer science research be of widespread relevance to industry.
So, no, I totally disagree with everything you have said.
Judging academia from your experience on program committees is like judging the entertainment industry from watching Britain's Got Talent.
Alternatively: if you are doing work that attempts to have industry relevance you should have some idea of what problems are actually relevant to industry. In particular, just because you think something is an interesting and challenging problem that just has to be affecting industry players does not mean it actually is. It may have been solved already, or you may have made some poor assumptions on conceiving the problem which, if corrected, make the problem disappear entirely (perhaps replaced by a different one that would've been a more valuable research target).
If you're trying to do forward-looking invent things that aren't even a twinkle in industry's eye yet, that's absolutely fine too, and the author explicitly calls out that the only important thing here is to recognize when your work isn't likely to be applicable in the near term. Nowhere does Matt state that this makes this sort of research less valuable, and honestly in many cases academia is the only place it can reasonably happen due to funding incentives.
No f-ing way.
Cornell University has many departments that are good, but when I look at the agriculture and vet school they are beyond anybody else.
Ag schools do research which is relevant to the technological and business problems of their industry. They do plenty of work on genetic engineering, chemistry and work closely with the likes of Monsanto. They are doing a lot for big ag. They also do research to help small farmers beat pests without pesticides, produce and market (delicious!) Halal meat, even help householders same money and have a better lawn. "Organic" and "Alternative" innovations diffuse into the mainstream. Pesticides are expensive to buy and to apply; if there is a cultural tweak that's cheaper, they'll do it in a flash. When corn prices got high, Cornell promoted dairy farmers to plant cabbages and other crops as an alternative forage.
They are always beta testing new crops in our area; Cornell and UNH are finding variants of plants that perform well in the cold Northeast climate, have expanded Wine production and are commercializing new fruits such as the Paw Paw.
Their research is relevant, and it is also communicated directly to the public and industry. Cornell Agricultural Extension has an office in every county of the state that you can walk up to and call and get questions answered, go to an event, etc. They work with trade publications, local government.
And it is not just New York, they do research on tropical agriculture and run a program to get access to agricultural literature to anyone in poor countries that need it.
I would point to that as being a much more real "technology transfer" than the people who are concerned about copyrights and patents.
This is all much different from a software researcher who can just work from his/her lab (or even bedroom) at all times. It's easy to lose sight of the greater industry when you don't need its (direct) help to complete your research.
Please don't make unsubstantive, divisive generalizations like this in HN comments. They make the threads worse, including your otherwise fine comment.
I also don't feel academia is obliged to any particular promise of delivery, but that's kind of independent.
Instead it's saying roughly: much of the work coming out of academia attempts solve problems applicable to industry, but in actuality industry is not actually suffering from the problems solved (either because underlying assumptions are wrong and industry is plagued by a different problem or because the problem has already be adequately solved by existing work).
"My first piece of advice: do a sabbatical or internship in industry."
"you have to work on a real product team"
"hold yourself to a higher standard"
"keep an open mind"
> Now, I don't think all academic research has to be relevant to industry. In some sense, the best research (albeit the riskiest and often hardest to fund) is stuff going way beyond where industry is focused today. Many academics kind of kid themselves about how forward-thinking their work is, though. Working on biomolecular computation? That's far out. Working on building a faster version of MapReduce? Not so much. I'd argue most academics work on the latter kind of problem -- and that's fine! -- but don't pretend you're immune from industry relevance just because you're in a university.
This paragraph makes this point abundantly clear: there's plenty of research that is focussed on expanding the field and is discovering problems that will exist once industry catches up. Fantastic. It's risky to try and find the future, but someone has to do it, and I think (based on this and his other writing that I've encountered) Matt would agree that time spent in industry is of dubious utility to those folks[0].
For research that is attempting to tackle extant problems encountered by industry, though, the researcher would be well served by ensuring that the problem they're attempting to solve is actually extant and that the assumptions they're making in trying to solve it don't simply sound reasonable but actually reasonably embody assumptions underlying real (aka industry) deployments. Otherwise they may get a lovely paper which solves a problem that nobody actually has or presents a solution that is impractical because it assumes things which are, in real deployments, incorrect.
[0]: On one hand, it might give them a better feel for in which direction a tractable future lies, but by the same token it might prevent them from exploring potentially fruitful avenues by constraining their thinking.