Why can't you guys comment your fucking code?
reddit.com
reddit.com
I've always thought that academic writing is designed to impress rather than educate. I haven't read much academic code, but it seems the trait has carried over.
I really can't stand code that is designed to make the programmer look smart. I stopped messing about with CodeWars for this reason. All this dick-measuring about how few instructions you can use to implement a problem eyeroll. It misses the important thing: when some poor sap comes back to maintain the code in 2 years' time, can they?
First of all, coding isn't exactly a high class activity in most of academia. Any implementation that's not an uncommented mathematica notebook or matlab mess is in the top 1% of academic code.
Second of all, no one reads the code. So while there are incentives to intelli-obfuscate papers to trick naive reviewers, those incentives don't exist with code. (Oh, and it doesn't really work out well in papers either most of the time.)
Plus the code is still fresh in your mind so you haven't forgotten why did it that way, and discarded trade-offs.
https://github.com/facebookresearch/end-to-end-negotiator/bl...
The code tells what the program does. The comments should state what it is supposed to be doing and why it should do it. When they don't match you have found a bug.
I forget who that quote is attributed to.
Funny you're saying this, because it took me a while to understand what you meant by "DL" (which is required to understand what code you're complaining about).
See "Comments are Lies" http://www.codingblocks.net/podcast/clean-code-comments-are-...
Changing, say, a function should prompt the IDE to somehow "tag" the associated comment so that you remember to check its accuracy, at the bare minimum. Other nice-to-haves would be having references to code, e.g., if you talk about variable x in your comment and then rename it, the comment reference should be updated.
You can feel the disdain for code commenting in the industry by looking at the anemic tool support for comments.
If you've been a developer for any length of time, I'm sure you will have encountered a situation where you've read some code, thought to yourself "wow, that's terrible, I'm going to change that". Then, several hours later, realised why it was done the way it was and actually, you can't really improve it after all. That is why we comment; to save the next person who enters the cave (and, if you're the kind of dev who spends several hours trying to understand something and doesn't retrospectively comment what you've found, then I'm afraid we can never be friends).
And please, don't assume the next person has the same knowledge of the system that you do.
Declare each variable and argument on separate lines, and add a one-liner comment next to each. If you are afraid that you will fall into comment verbosity hell, you can set up a rule to force yourself to use only single-line comments.
The best compromise is to make your code readable and only comment about the "why", not the "how".
What we don't know is why it was done in the first place, or why it was done a particular way, or why I shouldn't change that line that looks wrong but is really correct.
The fact that the code is unreadable is because it's written by a ML expert. I'd say it's actually higher quality than you'd usually get from academia, but nowhere near what you'd see outside of that. Academic code serves a very different purpose than production code. It's meant to prove a point and usually not used after. There simply isn't incentive to write well tested maintainable code.
Why is code coming out of research labs/universities so bad?
1. DON'T WANT TO WRITE CLEAR CODE
Different incentives between academic research (publications count, citation count, fear of being scooped if research too clear...) and industry (code maintainability, clarity, robustness, handling corner cases, performance...).
2. DON'T SEE WHY CLEAR CODE MATTERS
Academic projects are one-offs, not grounded in a wider context. Even if the researcher would like to build something long-term useful and robust, they don't have the requisite domain knowledge to go that deep. This external knowledge, the subject matter expertise, is hard to come by.
In lieu of that, researchers start solving artificial problems on artificial datasets. The proper project cycle is broken, there's no feedback from other people using the fruits of your labour.
3. DON'T KNOW HOW TO WRITE CLEAR CODE
Lack of programming experience. Choosing the right abstraction boundaries and expressing them clearly and succinctly in code is HARD. Naming things is hard, structuring things is hard -- clarity in your head first, the code follows naturally.
But it's a skill like any other. When we put students to work on real ML projects, they are shocked. Many have never seen a properly designed piece of code in their life, don't know any tools, how to share or collaborate (git, SSH...). It's not like they'd sprinkle some extra comments here and there and be done -- the whole code structure and how they attain clarity of thought is different.
The GOOD NEWS is, it doesn't cost any more time to write good code than bad code.
So once they learn, everyone wins. That's the main purpose of our Incubator programme. It's not like researchers write crap code on purpose, no need to assume bad faith. Seeing the fruits of your labour useful and used by others is immensely rewarding!
The first condition is of course a problem that one has to deal with if the person writing code finds things self-explanatory that to you are highly complicated.
The second condition is of course a problem for those people who are not able to explain things well in human language.
edited: for clarity (prior actually my really clever example of people who are not able to explain things well in human language)
That being said, if I’m doing some obscure calculations or something ambiguous, I’ll usually comment on it (for future me to understand the context).
Error: Not enough emotional investment to fix a single line, but enough to write several paragraphs of rant.
Nobody gets a free lunch, not even those trying to get a free lunch from free academic software.
Academics are not rewarded for good software engineering. The incentives are not properly aligned. And more-over, the that's not necessarily a bad thing! Research is research, not engineering.
Stop expecting production quality code out research groups. The purpose of research is to explore ideas, not implement some epsilon of your product.
If you want a perfect exposition in addition to a high quality implementation, there are ways to get it. Pay the consulting fees. Or pay the tuition. Or hire the students, Or if you can't afford that, wait for the thesis/book.
> Is pseudo-code a fucking premium?
In a paper? Well... yes. Categorically, emphatically, yes. If you want to read an exposition where space does not come at an extreme premium, you'll have to wait for the thesis/journal article.
Space comes at a premium for stupid reasons (artificial page limits) but also for good reasons (expositions cost time, which costs money and grad student time, and fuck if I'm doing a 10 year Ph.D. at $7/hr because you're irritated at some python code.
> Can you at least try to give some intuition before showering the reader with equations?
Either 1) no because you're not the intended audience, in which wait for the thesis/TR/dissemination blog post where I'll work every gory detail; or 2) no because the equations are hard for all of us to understand and no one has solved the exposition problem yet.
> How the fuck do you dare to release a paper without source code?
So all the products you work on all totally open source, eh? Either fully commit to Stallmanism while living off less than minimum hourly wage or get off your high horse.
> Why the fuck do you never ever add comments to you code When naming things, are you charged by the character? Do you get a bonus for acronyms? Do you realize that OpenAI having needed to release a "baseline" TRPO implementation is a fucking disgrace to your profession? Jesus christ, who decided to name a tensor concatenation function cat?
Because they're being paid 30k a year at best for 80 hour weeks under intense pressure to implement new ideas and the prop firms and Googles are knocking with 200k comp packages and honest to god 5 day work weeks. Time is (SERIOUS) money when you're slogging through grad school. Again, walk a mile.
And again, really, the point is to test ideas. For every publicly release line of code coming out a research group there's 5x or 10x unreleased code implementing dead ends.
If you want all this, stop bitching at over worked and absurdly under-paid grad students, and start calling your congressman to support higher NSF budgets so research groups can hire more permanent research support staff.
> Shouldn't incentives be better aligned. Should this type of complaining be celebrated?
IMO it's not as clear cut as some people think. Honestly, is the cost worth the cost to tax payers if the super high quality implementation is going to be out-of-date in 3 months?
It depends on the research project and how fast the field is moving.
At base, I wonder if the current system isn't reasonable. If research can't even stay fresh long enough to make it from conference paper to journal paper, maybe we shouldn't be throwing hundreds of thousands of engineering dollars at quality implementations.
All we're talking about is better quality code. This isn't a huge cost. It's usually just a matter of a small change in mindset while coding, and a few minutes refactoring and commenting once the code is working. Those minutes are repaid (and more!) the very next time the author has to revisit that code again. Good quality code saves money, it doesn't cost it.
You can leave error conditions unmanaged, leave the UI as a complete mess, don't worry about internationalisation, none of the things that add serious cost to a commercial project. This is not that.
If research coders aren't aware of code quality currently, maybe it's time that they got taught it? Writing good code will help researchers who write code, not hinder them.
Been there, done that, no. Refactoring a crappy implementation into a high quality, documented code base takes an enormous amount of time. And literally no one in academia cares about it. Industry doesn't care about it either because there's a presumption that any software created while in academia is inferior quality.
If you advise students to do this type of work with the expectation of any type of personal reward, you're a liar. They're basically doing software engineering that academia won't reward them for, and their year of software engineering won't count as job experience if they go into industry. And they aren't being paid software engineering salaries. It's just exploitative.
> All we're talking about is better quality code. This isn't a huge cost
This flies in the face of what the market bears in tech salaries. But no matter; I'll just never pay my software devs more than 30k, and also only give them maybe 20% of their time for actual software development... I'm sure I'll get high quality code! /s
> Those minutes are repaid (and more!) the very next time the author has to revisit that code again.
Which for 99% of research code will literally never happen.
> If research coders aren't aware of code quality currently
Tons of grad students leave academia after their phd, take very high paying jobs in industry, and become valued members of software engineering teams.
Research jobs are not software jobs. The goals and incentives are different. Sometimes it makes sense to create a maintainable research software system, but usually it makes no sense because the code is throw-away for testing an idea. That's OK. No one owes you an epsilon of your product.
I'm talking about Person A just not writing bad code in the first place.
Okay - so grad students have very little incentive to write readable code... fine, understandable. They are usually lucky if they can write an algorithm which isn't finely tuned to a very narrow dataset. In that sense, most Ph.D papers are outright fabrications lying about capabilities written by authors seeking paychecks. If people were able to investigate the underlying source code, that would be fairly apparent.
That said, why do so many professional research developers also suck as writing comprehensible code? They have fat paychecks, what's the problem now? There absolutely exist research developers who write very high quality software. They may still use very terse nomenclature (due to high cognitive load), but their code can be logically comprehended by those who do not understand the field. There are a lot of reasons (I think) for this; no incentive to improve, poor understanding of their own discipline, no acknowledgment of the problem, expectation that such work is for "engineers".
In reality - a lot of research developers are simply incompetent, but they find a way to regularly convince others they are not.. often through obfuscation. They're stringing together implementations from various publications (which themselves might be total fabricated lies) with relatively little comprehension.
That's okay, a lot of software engineers are incompetent too.
Own it.
I'm not conflating. The vast majority of research software is written by grad students.
> why do so many professional research developers also suck as writing comprehensible code?
Because incentives aren't aligned. And it's not even clear they should be aligned.
> There absolutely exist research developers who write very high quality software... In reality - a lot of research developers are simply incompetent
There absolutely exist software developers who write very high quality software... In reality - a lot of software developers are simply incompetent.
> If people were able to investigate the underlying source code, that would be fairly apparent... Own it
1. A huge amount of research software is open-sourced and does work as advertised.
2. You don't get it both ways. Either no open source demands, or no expectation that grad students should "own their work" -- and then making demands that they literally don't own their work.
Of course if it doesn't matter if the code runs correctly, by all means don't comment it.