Hacker News Reader

Comment by cs702 | Hacker News Reader

Notably, of those bug reports, fewer than 1% (only 3 bugs) were specific to the Linux version of the game. That is, over 99% of the bugs reported by Linux gamers also affected players in other platforms. Moreover (quoting from the OP):

> The report quality [from Linux users] is stellar. I mean we have all seen bug reports like: “it crashes for me after a few hours”. Do you know what a developer can do with such a report? Feel sorry at best. You can’t really fix any bug unless you can replicate it, see it with your own eyes, peek inside and finally see that it’s fixed. And with bug reports from Linux players is just something else. You get all the software/os versions, all the logs, you get core dumps and you get replication steps. Sometimes I got with the player over discord and we quickly iterated a few versions with progressive fixes to isolate the problem. You just don’t get that kind of engagement from anyone else.

View on HN

Indeed, although my kneejerk reaction to the subject line was to expect another sad article about how it's not worth it to support Linux, this author seems very happy with the decision to do so.

> Worth it? Oh, yes - at least for me. Not for the extra sales - although it’s nice. It’s worth it to get the massive feedback boost and free, hundred-people strong QA team on your side. An invaluable asset for an independent game studio.

Maybe there's a better term to throw at this. "Even though just 5.8% of userbase, Linux users contribute 55% of helpful QA feedback"

Makes total sense. You're unlikely to use Linux if you're not something of a power user, at least. I would guess there's a lot more developer type people who use Linux, and they are the people who appreciate a good bug report.

Same people also have a better idea of what might be causing the bug, how to get more information, logs, and so on.

They're also probably more used to the development process. With proprietary software, if you don't pay for a support contract, you basically can't submit a bug report - not really. You just ignore the bug and hope it's fixed in later versions.

This applies to big AAA games too, so games aren't necessarily special.

But indie games are in a weird spot where:

1. They're clearly artists. Nobody who want to make tons of cash becomes an indie game developer. If you have the aptitude to grind through the docs and make a game, you have the aptitude to grind leetcode and get a comfortable job somewhere that pays benefits and cash. And since they're artists, and they're making games, people empathize deeply with them.

2. They get a bit of a pass on the whole free software zealotry thing. I'm sure some people are still hard-line about it, but like, if I had to rank software in terms of freedom-threat-if-proprietary, something like Office or a compiler and towards the top, and Valheim is rather closer to the bottom.

3. They really have to listen to their players to survive, so they end up operating a lot more in the open.

So they end up looking a lot more like an Open/Free project than you would expect.

Reminds me of a tweet from Yann LeCun a while back where he bragged that they take down more reported content than other social sites. While at the face of it, this seems good, what it really means is that their system is worse at proactively taking stuff down before it gets reported. If they took everything down that violated their terms of service before it was noticed by someone else, they'd have a 0% follow through on these take down requests.

I think the title was intentionally a little clickbaity but I'm not exactly bothered by it. Clickbaity is a negative when the bait-and-switch robs you of the pay-off. In this case, I think it was overall a more interesting story and so for me subverting my expectations was actually a pleasant experience.

OTOH, it could have a negative effect if people only read the title and not the actual content.

Fair point. Unfortunately, there's no helping some people :)

I didn't read the content, but I trust the HN voting process to get the juicy comments on the top :)

Multicapitalism might be the more academic term.

A developer invested time to support linux not necessarily for financial return but social capital returns in the form of more helpful QA feedback.

Wow what an interesting term. Any idea where I can learn more?

There's a few forms I've bookmarked:

- Follow Bill Baue and the Reporting 3.0 folks - https://twitter.com/bbaue/status/1342101339120279552 - https://www.r3-0.org/wp-content/uploads/2020/12/r3-0-White-P...

- Joe Brewer on Multicapitalism and producing ecological flow: https://earth-regenerators.mn.co/posts/the-hidden-flow-of-re... skip to 16minutes for this model, references "the atlas of economic complexity" - https://atlas.cid.harvard.edu/ (bias: I sorta collaborate with Brewer on Earth Regenerators)

- Ethan Roland & Gregory Landua on 8 forms of capital : http://www.appleseedpermaculture.com/8-forms-of-capital/

The worst part? I've seen developers and companies actually complain about this. Just 6% of sales yet look at the huge amount of work they cause us!

Huge amount of work they do for us

Marketing/PR depts never see it that way. They open up new tickets, and that's the metric they care about. Not justifying it, but marketing depts rarely get it right.

Marketing of Facebook was of no use when Facebook was laying flat not so long ago. It shows which department should really have a definitive voice in product metrics.

> marketing depts rarely get it right.

Depends on whether you feel selling people stuff they don’t need is ‘not right’.

This is the case with people selling business software, meaning bugs would mean wrong accounting or worse outcomes for the customers. Yet all layers in the company would treat these reports as an annoyance.

Are there any good articles on writing helpful bug reports? Whenever I submit a bug I try to give as much detail and be as specific as possible but I always imagine some dev somewhere reading it rolling their eyes at the unnecessary paragraphs I’ve submitted.

I can recommend https://www.chiark.greenend.org.uk/~sgtatham/bugs.html

Thank you! I’m definitely guilty of mongoose-ing and diagnosing in my bug reports so this is very helpful!

> diagnosing in my bug reports

It's probably fine, if you put it in a separate section, clearing indicating that you are guessing.

At work we have an optional "diagnostic" section in our ticket template, mainly intended for the team, but I'd be happy to see a user try to fill it. At worse it's harmless, at best it gives the actual reason, somewhere in the middle it can give ideas even if it is wrong.

> At worse it's harmless

I wish. It happens very often that such speculation derails the whole investigation. It really shouldn’t, but people are people. If the title of the bug report says that the foobar is broken it might take weeks until someone realises that they were wrong and actually the problem is in the frobnicator which feeds the foobar.

Especially when because of this we assing the investigation to the wrong team. The foobar people don’t want to appear as if they don’t take the bug seriously, but all they check everything looks normal on their side.

It's interesting to compare that guide "How to report bugs effectively with ESR's "How to ask questions the smart way" [1]. They both cover similar material, but the styles are extremely different. The latter is rather hostile to the reader: "If ... then you are one of the idiots we are talking about." "If you decide to come to us for help, you don't want to be one of the losers." It's also heavy on "us vs them", how we are experts and you need to treat us properly.

You know, it just occurred to me that the "How to ask questions" document is targeted as much at hackers and how they should maintain their "standards" than at users who are asking questions. For example, the document has approving examples of "logically impeccable but dismissive" hacker answers; these make more sense as instructions on how a hacker should respond than as something relevant to a user.

I guess my point is that I had read the "How to ask questions" document for decades and viewed it as an objective document, not realizing how arrogant and "gatekeeping" it is.

[1] http://www.catb.org/~esr/faqs/smart-questions.html

I agree, that "how to ask questions the smart way" article always left a bad taste in my mouth. Perhaps there should be a "how to answer questions" companion piece.

esr seems to have a pretty big ego, just based on his writings. (e.g. at one point he declared that he can tell if someone is smart or not just by looking at them)

Strange, he's old enough to have watched the first TV run of Lt. Columbo ...

He actually at one point wrote that he was a reincarnation of the god Pan, so he has/had a very literal god complex, too. It's such a shame that he was the person to interpret hacker culture for those of us in the future, because he's hardly a good lens

I don’t disagree with your sentiment, but that wasn’t meant as a literal “I am the god Pan” — it was figurative, hyperbole. At least, that’s my recollection from when I read it a couple of decades ago.

I've read it more recently, and he did mean it literally. I've cut this together because it's far too long to quote en masse, but the full thing is at [1]

    Desperate for something to feed my jones, I snaffled my other sister's 
    abandoned flute. And wow! I was a natural...immediately better with it than 
    with the guitar I'd been hacking at for months. [...] This was delightful but 
    mystifying. All I'd had to do was learn to play a scale, and this amazing river 
    of music poured forth with barely an effort on my part. It seemed almost as 
    though my hands and lips had always known what to do, had been waiting for me 
    to pick up the flute. [...] I got these stunning rushes of pure timeless joy, 
    when my consciousness seemed to expand outwards from the limits of my skin to 
    fill the universe and I could no longer tell whether I was playing the music or 
    the music was playing me. Nor were these effects just going on inside my own 
    head. [...] I was walking home, idly puzzling over this peculiar incident, and 
    damn near fell over when I finally got it. That girl had been trying to cope 
    with a theophany; she had looked at me and seen a god. A particular god. And I 
    knew, suddenly, with utter shattering certainty, which one it was. And that it 
    probably was not the first time I had inadvertently triggered such an 
    experience, and would almost certainly not be the last. [...] Not that I took 
    any of it seriously as a description of the real world. It was an intellectual 
    chew-toy, perhaps at best a way of understanding the pathologies that prevented 
    human beings from living the infinitely more desirable life of reason and 
    science. Until I realized, finally, belatedly, what had been happening to me. 
    Until the Great God Pan reached out of my hindbrain and thundered "YOU!". And 
    his gift is music and his chosen instruments the pipes and flutes. And his, too 
    the power of joy; magic so strong that when it flowed out of me, even before I 
    knew what I was doing, it amazed people into awe and incoherence and poetry. 
    [...] (And, oh, yes. The first time I handled a set of pan-pipes I could play 
    them. Fluently. Effortlessly. And knew I could before I touched them.)

[1]: http://www.catb.org/esr/writings/dancing.html

The https://WhatHaveYouTried.com guy backtracked after a few years in the linked follow-up article, ashamed of giving the geek world another way to gatekeep and tell unworthy people to get lost.

Gemmell made a mistake in assuming the shame and guilt of the people who abused his article. The article is just a tool. We should hold the abusers responsible for their gatekeeping and telling others to get lost.

Use this template:

1. When I do this

2. Here is what I expect to happen

3. But instead, this happens

...4. And this is my software version and Linux distro.

Also useful to run the app from the terminal, replicate the bug, and see if any logs pop up.

Ugh no thanks, I'll just send them a screenshot and say "it's broken, please fix this!"

If it's good enough for my colleagues to submit tickets like this, it's good enough for me!

One more tidbit: In the "When I do this", I find it useful to omit stuff that isn't interesting. Does the thing fail if I do something immediately? Or does it only occur if the application has been dormant for a few minutes? Does it only occur using the mouse, or can I use the keyboard?

Get a good set of steps for reproducing the problem, but wander a little off those steps to see what is actually necessary. I've been surprised more than once when something that I thought was incredibly complex only required a few steps. And also been surprised when a complex repro was actually complex (e.g., if I drag quickly it works, but if I drag slowly it doesn't? what?).

It's probably better to err on the side of having too many steps rather than missing cruicial steps to reproduce the problem.

bug reports and communication in general: If you've got a good summary/abstract more detail is always better. I don't know if I've ever had to talk to someone on my team about how they "over communicated" (not the same as TMI!). Make the receiver responsible for saying "OK, that's enough, got it!"

Conversely I think you should generally write the smallest amount of text you can. It maximizes the chance someone else will actually read it.

I sometimes try to follow the journalistic reporting way of writing. (Not the clickbait style mind you.)

So I start it off with a few brief paragraphs of text explaining what I did, what happened and what I expected to happen. Each paragraph only a couple of sentences long.

Then below that I fill in with more details.

Of course this still means that if you glance at it you still see a lot of text and may not bother reading. But my hope is that if by chance the recipient does read the first few paragraphs then it will provide the necessary motivation to either read on or to begin investigating on their own.

It's best to do something like

1. config and versions

2. the bug (failure to see expected results

3. how to reproduce in as simple terms as possible

4. any data such as screen shots, core dumps, logs, etc

https://community.kde.org/Get_Involved/Issue_Reporting This one is pretty good

Think about what you would want to know if someone was reporting the bug to you. Provide at least that much information.

"Just the facts, Ma'am"

https://www.youtube.com/watch?v=v4LPkmGO5Cc

From "Post-surgical deaths in Scotland drop by a third, attributed to a checklist" https://westurner.github.io/hnlog/#comment-19684376 https://news.ycombinator.com/item?id=19686470 :

> GitHub and GitLab support task checklists in Markdown and also project boards [...]

> GitHub and GitLab support (multiple) Issue and Pull Request templates:

> Default: /.github/ISSUE_TEMPLATE.md || Configure in web interface

> /.github/ISSUE_TEMPLATE/Name.md || /.gitlab/issue_templates/Name.md

> Default: /.github/PULL_REQUEST_TEMPLATE.md || Configure in web interface

> /.github/PULL_REQUEST_TEMPLATE/Name.md || /.gitlab/merge_request_templates/Name.md

> There are template templates in awesome-github-templates [1] and checklist template templates in github-issue-templates [2].

> [1] https://github.com/devspace/awesome-github-templates

> [2] https://github.com/stevemao/github-issue-templates

As a dev, I think

STEPS TO REPRODUCE Open foo click bar

EXPECTED RESULT saves bar

ACTUAL RESULT crashes with error x912344

MISC <any other logs, details here>

is the best. It gives just the right amount of detail to get started, in a good, actionable format.

Yeah, that was basically most of the bug template we were filling when I used to work QA at Warner Bros. Games.

    Summary :
    Steps to reproduce :
    Expected result :
    Actual result : 
    Build/Version :
    And a few more fields about platform, severity, etc. on Jira.

Lucky you to have that standard enforced within the org! We don't so I pretty much invariably have to respond to "bug reports" ("foo doesn't work pls help") with "can you please format your bug report like this?" haha

Out of curiosity, what does deving for Warner Bros mean? Like working on their internal systems and stuff? Netflix integrations or something? Sounds cool, bet you get to see movies before they come out too!

Sorry for the delay!

I was working at their Montreal Studio. It's exclusively a video game studio. I was mostly working in QA on Middle-earth: Shadow of War.

Even if the game is technically developed by Monolith Productions, Warner Bros. Interactive Entertainment (WBIE) helps with their ressources (Artists, QA, Devs).

There's also alot of internal tooling for Builds automation, Automated testing / QA Aids tools, etc that is handled by WBIE as the game editor.

We didn't see movies before they got out but we had access to a 30-50 persons theater room that I was able to "rent" for free outside of business hours to watch movies with friends and show them the non-secure areas of the office. (Cafeteria (Free drinks!), gaming room, etc.))

I'm not really into commics but we had a room with thousands of comic books that we could take and read anytime we wanted (WB owns DC Entertainment).

I worked there 6-8 months when I was out of school before getting my first dev job.

No sweat, I'm glad to see the reply at all :) those perks do indeed sound really cool haha

Very important to also ask: Which application. Get an URL if at all possible. If someone says 'the calendar program is down', I have at least 5 candidates, and 50/50 chance something I've never seen pops up.

Users tend to rename programs, even give them names of other programs. Our entire finance department can't wrap their heads around the fact that 'Oracle' is not a bookkeeping application. Worse, they are now convinced the new version is called 'fusion'. Oracle financials? Never heard of.

Usually sending just "help" and no other data is preferred. Don't send screenshots, don't make use of the ingame reporting tools (pressing F11), don't send steps to replicate.

I’ve often thought that game developers, esp indy studios, would benefit from doing first beta releases on Linux exclusively - wringing out a bunch of bugs, then doing a wider release.

maybe alpha, beta level you gotta be on all platforms.

Well that might be one way to _accelerate_ your dev process.

>"we have all seen bug reports like: “it crashes for me after a few hours"

Here is a gem from my collection of user "bug reports". The email with this exact words and nothing else - "something is wrong, what do I do?". I was biting my fingers trying not to reply with the first thing that came into my head.

I don't hold myself and frankly answer that my crystal ball is on scheduled maintenance today.

So when will your crystal ball be ready, could you then asap adress the issue? Also keeping spare crystal balls would be really helpful.

We are a small company supporting a free opensource product, we can't afford spare crystal balls. Even the one we have was bought on a secondary market, warranty expired in late 14th century.

"We are sorry to read that. Please clearly describe your problem to us so we can help you. Feel free to call us on this phone number: XXX-XXX-XXX"

People don't always understand our perspective, we should guide them into doing the right thing and everybody will be happy.

Save your generic answer so you don't waste your time doing so, but being pedagogical should be part of our work in my opinion.

>"We are sorry to read that. Please clearly describe your problem to us so we can ..."

What makes you think I do not do those things?

I don't :-)

I was speaking to everyone. Apparently I got your last sentence right.

Interpreted literally it’s a reasonable question: “What process should I follow when I encounter a problem?”

Well try to visit car repair shop and ask the same question as in the original email and then explain them what you really meant.

Conclusion: want good testing and bug reports of your game? Support linux.

Possibly, but how many of the bugs are rare edge cases (i.e., attempting to break the game) versus problems players actually face in normal gameplay? Without knowing what priority or severity these bugs take, it is difficult to say how useful the increased volume of feedback is.

Even if someone is not attempting to break the game, they might accidentaly hit those rare edge cases.

Of course, you could infer the usefulness of the increased volume of feedback from the fact that a game author is extremely happy about it.

Overall, I don't see any particular problems with having reported issues. We definitely don't pay cloud storage per open issue or anything like that. Initial assessment of a bug report is not particularly difficult either.

I'm not sure if your concerns translate to actual negative impact.

FTA it sounds like basically all of them were normal gameplay that affected all platforms. only 3 were linux specific.

I'm not a gamer, but from what I've read, the whole industry has been transformed by the incredible lengths people go to, to cheat.

So I would expect that edge cases might be really relevant to mainstream use because everybody is trying to break the game.

Depending on what kind of game you have, those exotic edge cases can come to be the dominant behavior in games, particularly multi-player where slight advantages can lead to some players dominating every one else.

Or have proper analytics. If a user emails you and says "It crashes after a few hours" rather than feeling bad for them, you ask them what their steam/game id is and you look them up in the crash reporter system and see exactly what happens rather than relying on a few programmer consumers doing the work for you.

But not analytics that are too good, or certain people will write a lot of inflammatory articles about how your software is nothing but evil spyware.

Seriously, though, having a robust and automatic crash reporting system very much helps track down bugs you're never going to get a good report on or be able to directly reproduce.

It's not really that hard to do this: ask bug reporters to attach their logs. Don't just take them preemptively "just in case".

If I asked users to attach a bug report file, most of them would complain and tell me to stop waisting their time and get the problem fixed. I have many times explained to users that I found their issue in our reporting and have not ever got a negative response.

The average user does not give two shits that you can see exactly what line of code their instance of your game crashed at. They want to play a video game and not play find the bug with you.

The vast majority of users would not even be as nice as to email you saying there was a crash. They would just immediately complain on twitter or refund the product if it wasn’t resolved without any interaction.

You're in a thread talking about Linux users who file good bug reports.

The average user also has no idea what/how much information is contained in those automated reports. If you are just sending a build ID and a stack trace, sure. If you are sending memory dumps (even partial) or associating those stack dumps with identifying information (e.g. steam IDs) then you really should be asking for informed consent before you do that. And no, it being a common practice to just collect as much information as you can does not make it an OK practice.

Yep, it’s a great point to raise in support of releasing software for Linux. Adept users who can produce high quality bug reports and help solve bugs across all platforms provide a lot of value, perhaps enough to justify “only” a ~5% market share.

Just from the title, I went in expecting that percentage to be much higher, but still expected I'd get a chance to complain how report quality was not taken into account (I expected Linux users to submit better reports)!

Pleasantly surprised on all accounts :D

Sometimes I forget my detailed bug reports with steps I took, even if I can't reproduce it--since it might provide a hint to someone with insight into the programs internals--, are a weird outlier.

more likely s/linux/developers, which i suspect represents the majority of linux users

God damnit... Clickbating is ruining any shred of useful information...

I am not sure why would one expect more linux bugs in a game made using a game engine (I couldn't find any info and I am assuming this game is one, correct me if I am wrong). If it is not working in linux, then it must be an engine bug, not a game one. If I am wrong then props to devs for having such a consistent game engine

Perhaps other option is using cpp, and having UB that behaves differently in different compilers. But I doubt that as well

OP was not saying that 38% of the bugs are Linux-specific, he was saying that Linux users report many more bugs per capita than windows users (and he sees that as a good thing).

Reply on news.ycombinator.com