Hacker News Reader

Comment by throwaway57023 | Hacker News Reader

Writing code in research is quite different from writing code in industry. For starters, in research, absolutely no one will read your code. Not your boss, not your peer reviewers, not your colleagues, not your tech-savvy users. No one. They just care about getting the right results and will complain if they don't, which most of the time steers you into producing correct code.

Also, most programs usually have a maintainer team that's exactly one (1) person strong, and that person is usually an underpaid grad student or postdoc with a billion other tasks at hand and not much time left for that issue you opened three months ago.

On the other hand, hey, you never have to fear the dreaded 'code review', so it's not all bad.

View on HN

> Writing code in research is quite different from writing code in industry

Hah, not always.

A short while back I was tech lead for an AI project, the goal of which was to reduce the weight (and, ergo, cost) of large steel structures.

The megacorp consultancy I work for decided to staff this project only with AI people, about half of which were fairly fresh graduates. Now, they all had a great grasp of AI, both old-skool (neural networks, genetic algorithms, particle swarm optimisation etc) and more recent innovations. But these were not developers.

The customer was a Microsoft shop, and mandated that we use C# (which was cool, it's my favourite language!), but it was immediately apparent that the "devs" had only basic training in Java. The code was a fcking mess, and we had numerous issues around software engineering concerns such as source control, DevOps and processes - the customer eventually canned the project due to the bugginess of the platform. They really did have some brilliant minds on the project, but those were not the minds of software engineers.

On a more recent AI project, unusually I was consulted first on staffing, and I insisted on a mix of AI research types and* software engineers - unsurprisingly, this project was far more successful.

Seems like your formatting was messed up by putting an asterisk in "f*cking", which caused everything to be rendered in italics between that word and the "and" which you actually wanted to be in italics.

Bah, on mobile, apologies :/

I just finished an internship at a research lab and was completely dazzled by the lack of control on the code people write.

The reasoning is that since it's research code that is used to get results only and not be used in a product, doing tests is not useful. I disagree completely: being sure that your implementation of your model or equations is exactly doing what is should is very important to evaluate your model, and yield correct results.

I imagine you said that as tongue in cheek, but good "code reviews" should not be feared. If it is the case it either means that environment you are coding is toxic, or that you have not made clear with your peers that code review is not about judging.

Code review should not be about being right or wrong, but as a team to provide the best work possible. People should accept that everybody make mistakes, and the goal of the environment/society is to minimize the consequences of those inevitable mistakes, not judge individuals based on arbitrary metrics.

Code reviews delay gratification. Often by long enough that the visceral relevance drops off a cliff. While (good) code reviews definitely improve the software quality for a project, they can be very jarring breaks in flow. It should be fairly understandable why some of us would regard them as about as pleasant as eating a fibre supplement.

Problem is, we don't want the results the author thinks are right; we want the results that are accurate.

The problem is, it's impossible to know if they are accurate without doing everything the computer was coded to do for you.

This is not entirely true. When I implement a simulation, I specifically look for specific properties of the simulated system that can be checked, for example time evolution of total energy or (angular) momentum. If you have a decent set of properties with non-linear relationships between them, it is actually quite hard to have a misbehaving simulation that still produces corrrect values for these properties. In fact, these checks have led me to the discovery of bugs that would otherwise have been impossible to find because the sim output was just plausible enough.

This really rings true for me. I've also found that looking for several "known" properties within a complex model's behavior to be an effective way of rooting out subtle bugs. If the model isn't keying in on the obvious, it hardly has a chance of keying in on subtle unknown relationships. I've even gone so far as to optimize hyperparameters based off context specific properties, e.g., hunting for the model that has the most coherent output behavior with regard to a range of inputs (assuming that outputs are expected to have continuous behavior).

https://sproutling.ai/blog/harvest-simulations?jm

https://sproutling.ai/blog/growth-simulations?jm

Well, the premise stated above is that whatever testing you, the "author", judge sufficient doesn't matter; accuracy comes only at the hands of the sacred code review!

I dare you to review dense code for numerical computations and actually spot bugs. This is really hard! Unit tests are actually much more reliable but they are limited to deterministic algorithms and models that have reasonable complexity, that is, it is viable to compute expected results by alternate means.

Code review is more about determining if you have the correct test cases to cover the algorithm, and a solid architecture for maintenance, than it is about algorithmic correctness.

Code review is a tool to push back on your manager ignoring testing: “Steve requested I add X tests.”

That's not necessarily true. We could start building correctness or equivalence proofs for the building blocks of research software, and maybe some day we could prove some meaningful equivalence between how the software is described, and how it actually works.

If only research paid as well as violating the privacy of the American people to trick them into clicking on advertisements there might be more parity in code quality control between research and industry. But we’ve instead allowed google to take all the money away from the scientists, who are increasingly doing societies heavy lifting for free.

Generating clickbait ("engagement") and playing roulette ("trading") have become the greatest pasttimes of the STEM talent pool.

A grad student faces an existential career risk for every day they spend on anything other than finishing their degree. That's because events beyond their control can throw them out onto the street. This includes loss of funding, loss of their faculty advisor, health issues, visa problems, and so forth.

They may also be the only qualified programmer in their group. A lot of research, while officially housed in some specific department, is quite multidisciplinary.

I work in an R&D group in industry, and our team invited a software engineer to join our group and teach us better practices.

>Writing code in research is quite different from writing code in industry.

Depends on which industry. For example, writing code in research is actually a lot like writing Excel macros in Finance.

> They just care about getting the right results and will complain if they don't

That sounds like a group of people willing to cut corners to get approval from their boss... which isn’t all that different from the incentives in “industry”.

Reply on news.ycombinator.com