Hacker News Reader

Comment by kylecazar | Hacker News Reader

They want you to just submit a PR with a test that, if passed, would indicate the problem for you is fixed.

I kind of like this. It's a more formal proof of concept. You prove the bug exists by writing a failing test. If they cannot construct a failing test then it's either too hard to mock or reproduce (and therefore maybe not even worth fixing, for a free tool), or it's impossible because it's not a bug. Frees up maintainer time from dealing with reports that aren't bugs.

> If they cannot construct a failing test then it's either too hard to mock or reproduce (…), or it's impossible because it's not a bug.

Or, you know, the user is not a developer. Or is unfamiliar with Python, or their test suite, or git, or…

It is perfectly possible to be good at reporting bugs but be incapable of submitting pull requests.

The problem with popular tools is that they have more bugs that can be fixed. So bug reports are pretty much worthless: You know that there are 1000 bugs out there, but you only have resources to fix 10 of them.

By asking users to provide reproducible test cases, you can massively reduce the amount of work you have to do. Of course that means 90% of bugs will never be reported. But since you don't have the resources to fix them anyway, why not just focus on the bugs that can be reproduced and come with a test case...

I don't think it's necessarily about fixing those bugs, but I think a lot of times it's more about at least having those bugs be documented in order to raise awareness on (probable) issues down the line for whoever would want to use that project in the future.

You missed the point entirely.

It’s your prerogative if and how you want to limit the amount of people who can contribute, but I was explicitly replying to someone claiming that a person’s inability to code is in any way related to the validity or importance of the bug.

What happens if you don’t know Python? Python is a relatively easy language to learn but no way I’m gonna learn Python just to report an issue

Good chance you wouldn't be writing good bug reports either, then. Github issues have enough noise that a first-pass filter like this feels like a good idea, even if it has some false positives.

I fail to see the logic in your comment. Just another case of Goodhart's law.

This isn't really a metric though. It's a formal existence proof that the bug exists. The key difference IMO is that you have to create a test which A) looks (to the maintainer) like it should pass, while simultaneously B) not passing. It's much harder to game.

There are other cases where Goodharts Law fails as well: consider quant firms, where the "metric" used to judge a trader is basically how much money you pull in. Seems to be working fine for them

If you can’t describe your bug in a test, then you probably can’t describe it sufficiently in English either.

Seems to make sense

This in no way aligns with reality. I frequently interact with users who can’t code at all but make good bug reports. One of the best ways to ensure success is to have a form (GitHub allows creating those) which describe exactly what is necessary and guide people in the right direction.

What you're saying is even worse, since you’re implying someone could be an expert computer programmer or power user, but because they’re unfamiliar with the specific language this project chose, they are incapable of making good bug reports. That makes no sense.

The same thing that happens if the author of the OSS you use doesn't know English.

Then you don’t get to contribute bug reports.

Perfectly fine rule for a maintainer to have.

If the bug is egregious enough, somebody else will find it. If the bug is important enough to you but esoteric, then ask on a forum or enlist the help of someone you know who does know Python.

How do you currently submit bug reports on e.g. MS Word or Adobe Photoshop? This way is certainly more open than those commonly-deployed software.

Did you (or anyone) in this thread look to see exactly what they are looking for with their provided examples?

https://github.com/soimort/you-get/pull/2680/commits/313b8d2...

You do not need to know Python deeply to construct what they are expecting. They’re not actually looking for a unit test or something.

> Did you (or anyone) in this thread look to see exactly what they are looking for with their provided examples?

I did. And I looked at all examples of “good commits”, not just the trivial ones.

https://github.com/soimort/you-get/pull/2685/files

That’s already complex for someone unfamiliar with the software (which might nonetheless be able to open a competent bug report).

That's exactly it. They put up a gate that blocks low-effort issues that only add busywork. I like it!

Reply on news.ycombinator.com