Hacker News Reader

Reminds me of Apple's Jim Reekes "engineers are retarded".

https://m.youtube.com/watch?v=C5d151lqJsA

(The annecdote really starts at 2:30).

Haha, what a great video.

To some extent he's onto something. Sometimes you are in this zone when everything will work if you just do this one quick little hack, and then the hack ends up not getting replaced with a proper solution, and in the end you end up with a harddrive right next to a speaker >.<

There actually seem to be people whose default mode of operation seems to lie entirely in that zone though. I'm pretty sure everyone arrives at a similar conclusion if they realise that attempts to fix a certain person's code always end in a express trip to the 6th stage of debugging [1]...

[1]: http://plasmasturm.org/log/6debug/

Definitely. And that "quick hack by default" person will often end up being the favorite of management because they get so many things "done."

So it's important to figure out what kind of manager you have, and try not to be the slow guy on a team full of quick-hack people. You won't look good and nobody will be happy.

> the slow guy on a team full of quick-hack people

isn't that called 'cultural fit'?

That assessment is best made by the People and Culture team, not you.

> So it's important to figure out what kind of manager you have, and try not to be the slow guy on a team full of quick-hack people. You won't look good and nobody will be happy.

I'd also add that it's important to understand the context in which your team is working.

If I were managing this hypothetical team in a pre-product-market-fit startup I could see the 'slow' person as a potentially greater risk than the others. Not because of speed per se, but because he may be investing too much time in directions that don't help the company learn about their market, and he may be building grand architectural visions that are hard to delete when we realise the product is going in a direction that person didn't anticipate.

On the other hand if I were managing the same hypothetical team in a highly defined context, for example a mature product or an open source library with a large userbase, the 'quick hack' people would need to change their ways.

Obviously this analysis is incredibly shallow and would need a ton of conversation and observation; I'm just making the point that different phases of product market fit require very different approaches & it's worth being aware.

Kent Beck talks about this need for context awareness a lot in his 3x (Explore, Expand, Extract).

https://www.facebook.com/notes/kent-beck/the-product-develop...

https://www.facebook.com/notes/kent-beck/comparing-explore-e...

Beck could have saved time by following tech strategy legend Simon Wardley ;)

http://blog.gardeviance.org/2012/06/pioneers-settlers-and-to...

Also note that Wardley was pushing past PaaS into serverless and per-function billing back in 2005 with Zimki... lightyears ahead of AWS, until it was designated 'non core' by Canon.

We do ourselves a kindness when we skip to step 4.

Except step 5 needs two variants, then: "Oh, I see why it happens" and "Oh, I see that it doesn't, so let's find a way to communicate better with the user".

There are a lot of ways that things can go, but starting out with the disposition that either the bug report is wrong or the other machine is somehow made of worse silicon than our own, is rarely helpful.

Agreed. The direction I was going is this: If there's a bug report, then there is a problem. The problem might be the bug that was reported, or it might be that you surprised the user/tester in a way that they perceived as being a bug. Obviously something needs to be improved, even if it's just communication with the user, rather than changing the behavior of the program.

> starting out with the disposition [...] is rarely helpful.

Of course, I fully agree. And yet I have witnessed enough developers stating the first stage almost verbatim as a response to a complaint that I grew really quite fond of the link.

It is, actually, quite useful in that regard - it allows you to point out the absurdity of the statement in a humoristic way.

That, and it wouldn't be a good parody of the Kübler-Ross model if it didn't start with Denial :-)

However, investigating every bug report while a worthy cause is something not practical for most shops, especially when the underlying cause for these 'only sometimes and only on my machine' issues is some bizarre bug in some underlying dependency or even windows.

Now fixing these issues can often lead to really lean mean software that flies, but if you're in the all too typical situation of overambitious deadlines then you're already in triage mode and long hard fixes are just about the bottom of the pile

That was amazing. I love crotchety grizzled engineers more than anyone I think. This was classic and hilarious. I can't wait to share with everyone at work.

Well, he would know about engineers being "retarded". I found him extremely difficult to work with, unwilling to accept that any behavior of his code was a bug, and an all around obnoxious guy.

I thought Windows had a fancy interrupt priority system that should, in principle, allow sound playback to preempt network interrupts? AMD64 added a fancy feature (CR8 access to the task priority register) just to accelerate interrupt prioritization.

(This is all very vague memory. I know how this stuff works on Linux. Linux does not have interrupt priorities.)

I could be wrong, but my understanding of some sound cards is that they have essentially a single memory buffer that they are reading from when instructed to play a sound. Most sound cards let the OS split the buffer into two halves and raise an interrupt when one half of the buffer completes playing.

Interrupt prioritization doesn't help much, because the sound data is likely being generated from user mode, and the playback complete interrupt is being handled in kernel mode, which would require a transition back for further processing. When receiving data from a network card, no transfer back to user mode is required and will have implicit priority over sound generation. Therefore, network processing is likely to starve out sound generation barring choices like those described.

(This is somewhat hearsay, corrections welcome :))

Needing to wake up usermode is not necessarily an insurmountable obstacle; under Linux you can use a PREEMPT config. Storming a network card having unwanted consequences can also be mitigated: under Linux you have some network device driver that use an hybrid interrupt/polling approach with the NAPI. Combining both should theoretically allow a system to generate sound smoothly in the described conditions.

IMO you shouldn't really need to wake user mode that often. Just let user code buffer the output samples well ahead of the hardware needing them and have the kernel do the copies when needed.

Of course if you can do that, it's really good. However, sometimes you need low latency.

Windows does not really have interrupt priorities IIRC. It kind of sort of have some, but the applicative code (device driver included) can't chose at which level they run their ISR, so its quite random whether or not the ISR you prefer should preempt others.

Plus typically general purpose OSes don't map hw interrupt priorities mechanisms to their own priorities (and very soon the hw interrupt controllers do not even know whether an ISR is running or not). I believe neither Linux nor Windows can benefit from hw assistance for interrupt priorities (and they mainly don't care at sw level anyway)

Windows interrupt priorities are called IRQLs, see https://blogs.msdn.microsoft.com/doronh/2010/02/02/what-is-i... for a discussion.

Mostly PnP decides the levels, unless you have multiple IRQs and you need to specify the priority.

That very page makes me think device drivers don't choose their DIRQL. Any more precise pointer to how they can do that?

Yeah thanks Microsoft. They apparently never thought of someone playing mp3's from a network drive!! It drove me nuts for months, until I figured out exactly what you just posted.

Thanks for the links and reading material.

It's amazing how important the scheduler is in an OS. I'm no expert on OS-level programming, but couldn't they have just used a simple round robin scheduler?

In this case the scheduler isn't as much of a problem as the interrupt service routine, which by its very nature interrupts the normal thread-level scheduler to do some work with the hardware.

Both the Network Interface Card (or wireless chip) and the Audio hardware generate interrupts for a variety of reasons. Network cards generate interrupts when data is finished sending or receiving. Audio hardware generally generates its own interrupts when its buffers have emptied out, to signal the OS to provide it more audio to play.

In this case, it sounds like Vista's network interrupt was running for so long that it was taking priority over other interrupts (audio) and causing them to not be serviced quickly enough. This is definitely a problem, but the fix should not have been to limit the amount of work done per interrupt, but instead to delegate that work to some other process with lower priority.

It's weird to think of audio as higher priority than networking traffic, but in this case that actually makes a ton of sense. Audio generation only requires ~88.2 KB of data per second, but latency is much lower and less tolerable by a human listener. Modern network connections are working with multiple MB per second, and are (relatively) more tolerant of slight delays. Same with mouse and keyboard input, it's why your mouse tends to move smoothly on most Windows machines even when the rest of the OS is obviously struggling with some heavy task.

On Windows, like other modern operating systems, a driver is supposed to do only the minimum amount of work possible in the interrupt handler, to avoid this problem. The real work is supposed to happen later when it can be scheduled.

While it's true that networking device drivers on Windows defer all meaningful work to post-ISR, it only moves the goal posts a little bit. On Windows, a lot of network traffic processing (e.g. tcp/ip acknowledgment, virtual switch packet forwarding) is performed in a DPC[1] queued in response to the device's interrupt. A DPC generally runs at dispatch level, which gives it more-or-less exclusive access to the CPU until completion. This preempts the thread scheduler and can result in user-visible stutters in audio and video when the network load outpaces the CPU's ability to quickly service the network traffic.

[1] https://docs.microsoft.com/en-us/windows-hardware/drivers/ke...

These are hardly complex tasks, they could be done by computers from the 70s. It's quite possible that Windows makes a mess of it anyway, of course.

But the top comment was about how interrupts from the network card interfere with interrupts from the sound card so it can't keep its buffer filled. That shouldn't happen.

Unless the CPU just isn't fast enough to do networking and decode or generate the audio at the same time, but then it's never going to work properly.

Just look at the process you have running, 90% of them are doing nothing. Round Robin is a waste for an OS.

Processes that are waiting on some event (like polling some file descriptors) will never be scheduled until they're woken up. It doesn't matter what type of scheduler you are using. The algorithm just decides which to run within the set of programs actually running.