How to Criticize Computer Scientists (2001)
cs.purdue.edu
cs.purdue.edu
On the odd occasion that they have evaluated the framework you named, you can simply try again a few minutes later with a different framework. If you are challenged, use condescension to imply that the victim did not try hard enough.
This tactic works even for situations where the victim has used a first-principles approach to completely demolish a problem to the satisfaction of all stakeholders. 'Reinventing the wheel' is a type of sin. The mere suggestion of it will stick like shit to a blanket.
People fit frighteningly complicated models with millions-to-billions of parameters. They throw in hairy regularization schemes justified by fancy math.
When it comes time to evaluate the output, however: "Our model is better because this one number is bigger than their number [once, on this one test set]."
Of course, over the years, the game developed dozens of other rules.
[1] https://clauswilke.com/dataviz/boxplots-violins.html
[2] https://www.sciencedirect.com/topics/mathematics/highest-den...
It is super weird that an field devoted to doing inference somehow just...doesn't when it comes to evaluating their/our own work.
Is it now past the peak hype cycle, or do people still love it?
The dev has now left but he would always insist that the existing solutions didn't meet every need. There was always one feature that required him to build some over engineered version of something. "Good enough" was never an option for him.
1) try Foo because of all the Xes all of us have tried it was the only one which didn't cause manic depression,
2) continue the discussion until the most ruthless/charismatic/stubborn person "wins", or
3) try new shiny Bar, because all the cool kids are using it.
I wonder if this is part of the reason so many projects go for the new shiny thing: in a group of reasonable people a lot of the time all of them will agree that all existing solutions suck.
This made me chuckle, and then I read this:
> systems researchers will light up when telling you that they have constructed a system that is twice as fast, half the size, and more powerful than its predecessor
and thought "Oh God, that is me"! (I'm not in research) I almost don't want to read the rest.
On a (possibly) less personal topic, I've found that non-computer science researchers who program will dismiss all help with "it doesn't need to be run by anyone else", (which is kind of scary considering the problems with reproducibility) or, if they are open to help but they don't understand what you've suggested/written/submitted etc, will try hard to drop it quietly. I think it's so they don't have to admit any lack of understanding, which is weird - I can barely understand code I wrote 6 months ago and I've written plenty of Perl in the past, too. The idea that code should be so easily understood that to ask a question would make one seem inadequate strikes me as a fanciful dream.
Edit: don't want to mislead, I'm not a researcher, it just sounds like me :/ :)
It wasn't so much about understanding, I guess, but the apparent loss of ownership. Like if bits of the research get automated, then it isn't no longer their work, but the computer's.
It looks to them like if they can just add one more floor to the house of cards it will be over, why bother explaining the whole project to someone else?
I read that, chuckled and felt offended because IT ME.
> systems researchers will light up when telling you that they have constructed a system that is twice as fast, half the size, and more powerful than its predecessor
Then I read this and I _also_ thought IT ME.
I am not sure whether this means I have the skills of both sides or the foibles.
I used to work with bioinformaticians, and often wasn't impressed by a lot of the code that they churned out. But to be fair a lot of the work was just to produce a one off graph / heatmap to prove or disprove something, so most of the time it wasn't so important.
> I can barely understand code I wrote 6 months ago and I've written plenty of Perl in the past, too. The idea that code should be so easily understood that to ask a question would make one seem inadequate strikes me as a fanciful dream.
This was a big learning experience for me. I wrote and maintained a Django based system for four and a half years. When I couldn't understand my own code a few months later it was time to refactor. Ask yourself why you don't understand it and how you would expect it to be if it was written in an easier to understand way. It will save you time in the long run. Ex-colleagues commented that they found my code / database design fairly logical after I had left that job.
Cosmology: Use the word "cosmology" interchangeably with "astronomy."
Simulations: "Did you include dust?" If they say yes then ask about magnetic fields. Either way, claim that they tuned their physical hyperparameters to achieve their results.
Observations: "Did you follow up with <X telescope at different wavelength>?" Pick one that's especially competitive to request observations, such as Hubble or SOFIA or ALMA, so that if the answer is no they'll feel extra bad about their rejected Hubble/SOFIA/ALMA observing proposal.
Or by adult professionals on Hacker News.
The worse form of this is when they: 1) make poor choices faster than you can catch up
2) have a less senior team (in ability, not title) that can’t see more than a couple commits ahead to keep the damage in check.
Explain it in your exit interview.
No such thing.
This reminds me of some papers that got their proofs wrong by applying some fancy theorems. The fact that their results are correct clearly suggests that those theorems were added as a second thought.
There isn't clear distinction between the engineering aspects and the theoretical aspects, as the article said, it looks as if we are trying to get approval by mathematicians so papers become a convoluted amalgamation of different ideas and more often than not actually provide the worst of both worlds.
Yes, there may be solid math behind it that says “if your problem is of type T, this algorithm will get within Foo of the optimal solution in O(n log n) time”, but the problem is that nobody can tell whether a given real-world problem is of type T. Yet, people happily run the algorithm and if it works, it works.
The only problem was that under real conditions his code behaved like the stack was infinite; you know like it is in a theoretical computer.
This is my favorite, not because I've heard this particular one, but because I've heard this vein of low-effort comment after nearly every talk I've ever seen. "Did you consider this specific aspect of niche-thing-only-I'm-working-on?" Some people seem to always look for the opportunity to show how much they know about a topic rather than actually discuss in good faith the topic at hand.
I've seen 2 kinds of people that usually make this comment. Some do because niche-thing is the one they currently know better, and they have to make some comment, so they ask about what they know better. The others are the more interesting kind, they ask because they want to use niche-thing in some new way, so if you did for a chance consider it, they are very interested in it, and if you didn't, it may be an opportunity for them.
Also, if you are ever involved in academic research and you hear the words "that's just engineering" be very wary. It's a strong indicator that 1) the idea is not practical 2) they don't understand what they are doing. 3) you are going to have to make it work, often as an unacknowledged side project "that should only take a few days". I often spent years figuring out these "that's just engineering" side projects. Even worse, we'd build a "proof of concept" that ignored all the engineering, and then try to get other researchers or companies interested in the system as though it was complete.
Yet another thing I discovered is that for engineers one of the most important words they use in discussions is "no". Listen to two engineers discuss a problem and almost every other sentence will start with "no", "no that won't work because..." It's an important part of how they figure out how to make things work, by figuring out what does not work. CS researchers take it as an insult however, as in "no, you're an idiot and here's why". Perhaps because lead researchers are treated as infallible by their grad students they get the idea they can't be wrong, so they are not used to being contradicted. It's a huge problem in getting anything done. It can take months or years and a lot of wasted work to lead them around the circular path back to the original bad decision and try to get them to reconsider it (they are often very proud of it which makes it even more difficult). Saying "no" is somewhat like the mindset that is necessary for computer security work, you have to be able to attack systems or ideas from a ferocious point of view, seeking any weakness, without feeling that attacks on ideas are attacks on you. It's one of the most productive parts of discussion and is not taught directly, only by example and many CS researchers are so intent on building their reputations they will not tolerate it and are highly insulted by it and will defend their ideas to the point of absurdity. Framing a contradiction as a question helps here somewhat, though you still have to be careful in framing the question so it only leads to the contradiction rather than stating it outright.
Also I'm surprised that you had academics shopping "products" to you. I've definitely shopped ideas to companies before, but always with the explicit understanding that I'm doing "research" - i.e. you're spending a bunch of money on something that may go nowhere, and you're not going to get a product out of it.
That misunderstanding often makes the conversations end right there.
As a counter, I've found that many engineers will shut down ideas before you can even get started working on them because "no that won't work". It can be very frustrating, as the technical arguments they offer are stiff as a board. And not always as technical as they think.
Granted I work with GPU hardware research, so it could just be a lack of products in this area. Maybe viz? ML especially probably? I get the impression ML profs are a load of shit tbh.
The field has exploded over the recent years and the new people are somewhat filtered for the ability to hype up hiring committees.
It's because once something fails, in engineering the work doesn't stop. You have to root cause it and understand the failure. That's a valuable exercise in itself.
This is far from unique to academic circles...
This is how you should work in science anyways: "we tried to refute a theory, failed at that and are now forced to assume that it is valid to some degree."
Does CS have a problem with that in general?
Generally, I would argue most researchers are very similar to what you attribute to engineers, e.g. they often answer "no" first (I have heard that commented on by outside observers multiple times). In fact researchers (academic or otherwise) are trained to find flaws in systems quickly, which typically involves saying "no that doesn't work because ...".
The flip side of the coin is that managers (and Professors or group leaders are essentially managers) don't like to be told that something can't be done. Especially if it was their idea. This is the same in many industry settings. So they really dislike to be told "no".
The irony is that at some point in the transition from researcher in the trenches (PhD, postdoc ...) to Professor/group leader many start to think that the PhD students/postdocs say know because they don't want to do the work. It's really weird, because pretty much everyone who is in academia is strongly self-motivated as they should know from their own experience.
This also applies to industry. This is basically managing up 101 for engineers. If the boss thinks it's their idea, it'll get approved and supported. If they think it's someone else's idea, depending on environment, it'll get blocked or sabotaged.
That's what authorship is for. What's the point in supporting academia if you can't get authorship?
This paragraph is a great insight - knowing how someone else is going to receive your (as you perceive it) constructive criticism cuts down miles to the destination of achieving the common goal. Speaking so your listeners will be able to "hear" you without perceiving an attack is critical in cross-discipline groups.
An eminent computer scientist (Jeff Mogul I think) once pointed out that computer systems research is basically engineering.
I concur with this assessment and suggest that it's something that systems researchers should be proud of. After all, engineering means you're actually designing and building something that could potentially work.
It should have been better concluded and summarized by a decision graph.
:)
As someone who's mostly self-taught in CS (EE degree), I'd love to listen to something like this if anyone knows of a recording somewhere.
"There's no mathematical rigor" --> "That's a strength, it's simple, performant, and therefore easy to verify for our use cases"
"This is unnecessarily complex, and anyway you've ignored the constants. Our N is small." --> "Asymptotic performance just let's us sleep at night knowing it'll never be that bad. Here's a plot of the predicted and actual cost over our sized N's, you can see they agree well".
Perhaps the moral of the story is, be an engineer for your use cases, and a theorist for scaling.
This is true about all fields that involve substantial amount of mathematics, e.g., physics and economics. In physics (and to a lesser extent, economics) you see a sort of "rift" between experimentalists and theorists very often, with both groups thinking that they are better than the other. Even among theoretical physicists, you tend to see this sort of petty rivalry: high-energy theorists thinking that they are better because they are after the fundamental laws of the universe, condensed matter (both hard and soft) theorists who feel that they are the better lot since their theories can be compared with experiments, hard-condensed matter theorists often look down upon soft-condensed matter theory as being a "classical" discipline invented to bring in more grant money, etc. While there's some truth to all these beliefs, rather than engage in this petty rivalry, it would actually do a lot more good if people just did honest work in their own fields.
LOL UR PPR SUX JK ;)
https://www.brainpickings.org/2014/03/28/daniel-dennett-rapo...
"Oh look, another dev surprise."
"They improved the code... To run in O(2^n) time."
"I'm sure it worked in dev."
"I can't believe it passed QA!"
Despite all the equations, it seems to me that your work didn't require any real mathematical sophistication. Did I miss something? (This is an especially good ploy if you observe others struggling to understand the talk because they will not want to admit to that after you imply it was easy.)
Isn't this just a straightforward extension of an old result by Hartmanis? (Not even Hartmanis remembers all the theorems Hartmanis proved, but everyone else will assume you remember something they have forgotten.)
Am I missing something here? Can you identify any deep mathematical content in this work? (Once again, audience members who found the talk difficult to understand will be unwilling to admit it.)
Wasn't all this done years ago at Xerox PARC? (No one remembers what was really done at PARC, but everyone else will assume you remember something they don't.)
Have you tested this on the chip Intel got running last week in their lab? (No one knows what
chip Intel got running last week, but everyone will assume you do.)
Am I missing something? Isn't it obvious that there's a bottleneck in the system that prevents scaling to arbitrary size? (This is safe because there's a bottleneck in every system that prevents arbitrary scaling.)
If the author did not intend for it to be a joke, you can make the conclusion that the author has psychopathic tendencies.
I read it like the guy meant it because he felt unfairly sidelined and had some departmental spat with computer scientists...i don't care about that spat, i just thought he was being mean. Maybe it is a meant as a joke... But they say in every joke there's truth, and a lot of comics are the most angry people inside. Humor is their sublimation
Edit: i find this one funnier
https://www.cs.purdue.edu/homes/dec/essay.jargon.html
But it definitely has an edge of bitterness within the cs department, as i suspected initially. The themes i read into the first essay are clearly present in the second. I think i was right that this guy has an axe to grind. Just because he writes some supposedly satirical essays doesn't mean he isn't a toxoc person inside or possibly out...
Despite all the equations, it seems to me that your work didn't require any real mathematical sophistication. Did I miss something? (This is an especially good ploy if you observe others struggling to understand the talk because they will not want to admit to that after you imply it was easy.)
Isn't this just a straightforward extension of an old result by Hartmanis? (Not even Hartmanis remembers all the theorems Hartmanis proved, but everyone else will assume you remember something they have forgotten.)
Am I missing something here? Can you identify any deep mathematical content in this work? (Once again, audience members who found the talk difficult to understand will be unwilling to admit it.)
Wasn't all this done years ago at Xerox PARC? (No one remembers what was really done at PARC, but everyone else will assume you remember something they don't.)
Have you tested this on the chip Intel got running last week in their lab? (No one knows what
chip Intel got running last week, but everyone will assume you do.)
Am I missing something? Isn't it obvious that there's a bottleneck in the system that prevents scaling to arbitrary size? (This is safe because there's a bottleneck in every system that prevents arbitrary scaling.)Reaction to this guy issuing instructions on how to be cruel.
Now this de_nied guy is objecting to someone calling out bullying uses the schoolyard bullying tactic of repeating the words back to you. What's wrong with you? Unless you're a bully... Or is the de_nied guy the bully who was bullied?
"Isn't that same approach suggested and funded by Jeffrey Epstein?"
Google Translate produces a better translation (which is of course still awful), but I have a feeling that the one in question was made by an earlier version of Google Translate as well.
The primary title of this document is a thinly veiled attempt of shrouding the author's true intent. It's clear from their final words on the topic that they mean to insult, not criticize.
For good general advice on productive criticism, I recommend How to Criticize with Kindness, by the philosopher Daniel Dennett. [1]
[0] https://github.com/Droogans/unmaintainable-code
[1] https://www.brainpickings.org/2014/03/28/daniel-dennett-rapo...
> How To Criticize Computer Scientists > or > Avoiding Ineffective Deprecation And > Making Insults More Pointed
Am I missing something? Isn't your failure to read the full title a result of your desire to insult the author, rather than to criticize them? Wasn't this done already at Xerox PARC?