Hacker News Reader

Comment by jacquesm | Hacker News Reader

That's because our operating systems treat user input as a soft real time problem when actually it should be a hard real time problem. That's why your window manager sometimes can be unresponsive. I've used a hard real time OS for a desktop for some years and the difference with consumer stuff was extreme. You get used to the computer instantly doing what you tell it to do. None of these half second or longer delays between command and result. It's like magic.

View on HN

This is one of the things I still miss about BeOS. On late 90s hardware, the OS wasn't quite hard real-time like QNX but the worst-latency latency was much better than Windows 10 or MacOS on the latest hardware, even in the presence of heavy resource contention — I remember simultaneously surfing the web, downloading video from a camcorder with a Firewire interface which did not buffer, and compiling Mozilla. The UI latency didn't change at all, and the DV transfer didn't drop a packet — everything else got throttled back as necessary, of course, but that meant that, say, GCC took longer to run rather than impacting what I was doing in the foreground.

> I remember simultaneously surfing the web, downloading video from a camcorder with a Firewire interface which did not buffer, and compiling Mozilla

Did you forget to run the OpenGL teapot?

BeOS was magical. I wish I could use Haiku in my desktop today.

It wasn't magic, it just starved everything else whenever you waved the mouse around. In this respect it was no different from Mac System 7, even though the design wasn't the same.

That's not true, classic Mac OS used a cooperative multitasking model, where processes had to be friendly enough to yield control back to the system / other processes to allow them to do work. A poorly programmed or greedy process could easily hog the CPU.

System 7 checked for mouse motion every fraction of a second and if the mouse was moving it would drop everything and draw. It didn't need apps to yield for this to happen.

Are you comparing that shit when you pulled down a menu in System 7 could bring a whole network down, with Be? Such as a good joke...

The CPU and the whole net by holding a menu item down.

That was arguably an application specific issue, though the OS was painful to code for. I actually coded some classic Mac OS networking code and it was insanely easy to mess things up, but eventually it ended up being rock solid. The trick was all level interrupts where supposed to only do almost nothing. You couldn’t allocate memory for example which was a major hint of just how minimal those functions needed to be.

Worse with cooperative multitasking any running application could mess things up.

Then do not talk about Classic Mac as something magical because compared to Be they were utterly ridiculous.

For its era, System 7 was good, but later, it was worse than even Windows 98.

Be tried to do things right, it was ~1995 after all.

I see where java gui coder took their inspiration from /s

And that's exactly how a desktop operation system should behave. The most important priority of a user-facing OS should be the user after all.

Is it possible to do this with Linux? Giving the UI very high priority etc.

Yes, with the RT-Preempt: https://wiki.archlinux.org/index.php/Realtime_kernel_patchse...

It’s been a while since I tested it (pre-Wayland). The main problems I saw back then was that the window managers tended to do lots of file I/O for things like moving windows or opening menus so it was critical to tune the I/O scheduler (and NFS home directories were game over), and since rendering was up to the app it was common to have, say, the chrome respond relatively well but everything inside the window would be chunky on the main thread.

I’d hope this is much better now given things like Wayland and the increased use of parallelism.

Linux used to do that out of the box. But there was a series of "quasi-real time has too much maintenance costs", "quasi-real time isn't even real enough for the real gains", and "our computers are fast now, we don't have to trade so much throughput for latency" decisions that were all very sensible but let us here.

My 5 year old uses our first iPad which is over 10 years old now. The keyboard UI from then is still faster than my iPad pro keyboard UI because Steve Jobs valued the priority of that in the stack :)

They also ran surprisingly long, particularly the die-shrunk iPad 2: https://i.imgur.com/ELyFwFu.png

Indeed, I remember being particularly impressed by iPad battery life in the early days. I would recharge maybe once a week of light usage. Now I’ve got to plug in nearly every single night.

I remember that too.

But I also remember not having background processes. It's a trade of.

I charge my close to 5 years old Pixel C tablet about once a week. Light usage, mostly some streaming up to an hour per day. Not sure if your light usage means something else though.

I think Steve Jobs have the same latency sensitive condition as some of us do. Getting irritated when things aren't the speed they are suppose to be.

I’d be pretty surprised, because I feel like the UI in OS X has always had more latency than Windows. Especially noticeable with the mouse with acceleration, but also throughout the rest of the UI. This was my biggest gripe in switching to OS X.

Yeah. Mouse was always way better in Windows and Linux. Finder browsing files has also always been laggy. Now with the latest macos versions it's ridiculous, if you take a screenshot, it takes about a minute for the file to appear.

What the heck are you taking about lol

Screen shots appear instantly in macOS, I'm not sure what machine you're on.

Finder browsing files is much faster in macOS, Windows scans the entire directory with defender every time you click on it before it allows you to sort it or interact with it. That can take several minutes on some of my directories.

The mouse tracking model in Windows is somewhat different, and a bit more responsive, but there are multiple third party extensions to macOS the make its model exactly the same as the Windows model.

I have an old Panasonic “smart” TV with a daft animated but glacially slow UI.

You can tell the execs that approved that were probably impressed watching someone else demo it but probably never used it themselves.

Jobs would have spotted it was rubbish immediately.

OK so this is a partial tangent - but I have a washing machine that annoys the hell out of me for this exact same reason (I know washing machine UI is not a hot topic usually but it damn well should be!)

It has a dial on the front to choose settings. Potentially a great idea to skip quick to the one I want - BUT the damn thing can only detect 1 step of change about every half second. So if you fast-turn it 4 clicks, it still only moves one step. So you have to stand there, slowly turning it, click, click, click, click....

The dial is the biggest design feature on there. Massive. Right in the middle. Bright lights all around it. But they couldn't even be bothered to make it solve the one problem it was there to do.

I'm getting angry even reading about it.

It's the kind of thing that'd make you reject purchasing it if only you'd thought to have tested that specific bit of it before buying it.

I knew hacker news was the right place to share this story. Only here people would understand the pain I am in!

I am almost a day late... But yes.

Honestly, I wish that they would make appliances with physical controls as opposed to digital. At least those are easier to fix/mod on your own.

I always think that about modern SLR cameras. They took a series of brilliant, instantly accessible physical dials (around the lens for aperture, on the top for exposure) and replaced them with menus and buttons on a tiny screen. WHY? How is that progress?

I think if someone did a kickstarter for a very simple physical buttoned digital camera it would do very well.

Or there was even a demo unit in the first place...

Open it up, take the PCB out, figure out how the dial sensor works (the dial is probably just repeatedly nudging a contact out of the way), and thennn... program an Arduino to be a giant input buffer (that slowly replays), and wire the Arduino between the dial and the rest of the machine. :D

Caveat: said Arduino may need to sit between the rest of the buttons as well, in case inputs must be given in sequence.

Possible benefit: switch the Arduino for something from the ESP family, and you could feed instructions to the machine over Wi-Fi or Bluetooth. (Which could take the form of a dedicated wireless button panel above the machine with macros for frequently-used options.)

(Hmm. I can tell there's some over-optimization in here, but I'm not sure _where_.)

Yes, if you switch your kernel configuration to preempt. Have a look at https://liquorix.net/

On Arch Linux this is available as linux-zen

There is also realtime-linux - https://wiki.archlinux.org/index.php/Realtime_kernel_patchse...

> Have a look at https://liquorix.net/

This looks super fun to play with, thanks!

what are the tradeoffs here? I assume there is a reason why it isn't default or even in the main repo?

That's not up to kernel. Most of the slug is from the UI itself doing I/O (UI there is a wooly stuff almost nobody understands completely anyways).

I remember that about Amiga. Everything was just smooth. I still have fond memories of Cygnus editor being super-responsive to anything I could throw at it, even with SAS C running in the background.

I was taken aback by how slow Windows 95 felt after I finally jumped the wagon and retired my beloved Amiga in favor of a PC, circa 1995. Just moving the cursor was incredibly jerky in comparison and things would randomly freeze for seconds for no apparent reason. Windows NT 4 was much smoother though.

Even today, in Visual Studio, I regularly have multi-second pauses between a key press and the character showing-up on the screen, though this probably has more to do with Resharper than either Visual Studio or Windows.

> and the DV transfer didn't drop a packet

Considering how FireWire worked in comparison to USB that's not particularly surprising. The host machine was more or less out of the way while the FW interfaces did their thing instead of requiring constant hand-holding.

Kind of: it used DMA, but the buffer size wasn’t infinite. At the time, Windows 9x and Mac OS would lose data if you ran anything demanding. Windows NT and Linux might, depending on drivers and the exact nature of the other process’ activity.

> but the worst-latency latency was much better than Windows 10 or MacOS on the latest hardware

Citation needed. Do you have any numbers on that?

I'm skeptical that BeOS has better latency than modern OSes on modern hardware.

One of the funniest instances of software intentionally kneecapping itself are compositors. As far as I know compositors universally treat all monitors together as a single surface and just assume that they're synchronized. On Windows, this causes all sorts of hilarious stuttering and hitching. On Linux, compositors just vsync on the slowest display.

From what I understand, on linux wayland makes it possible to do this properly but I'm not sure that any environment on top of it is doing it. I think it's in a "nice to have" holding pattern for them because there's still so much to do to getting a stable and usable desktop environment there to begin with.

Should be fixed in the upcoming GNOME 3.38: https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1285

I wrote a gtk program in Rust that did all its work in the wrong place. It locked up the ability to move ANY windows on the desktop for several seconds. That's a compositor design issue if ever there was one. Is that going to get fixed?

If it was an X11 app, there are several things that can cause that that are just unfixable, unfortunately (e.g. XLockDisplay). If it was a Wayland app, that sounds like a bug in the compositor. If you still have a repro case, please file it.

It was gnome on wayland.

This is changing, I believe that the next release of mutter (GNOME) will remove the global clock on Wayland so it can drive each monitor at it's native frequency[1].

IIUC input is still driven at 60Hz but changing this is in discussion [2].

[1] https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1285 [2] https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/168

How does mutter behave for X?

Mutter is a compositor for Wayland iirc, so it's non-applicable to X.

?? If so what's the X gnome compositor?

Did a little more research, Mutter is used on X as well it looks like, or at least used to be [0]. Sorry for the inaccuracy.

https://en.wikipedia.org/wiki/Mutter_(window_manager)

Edit: I missed the part where you were asking specifically about GNOME. Removed everything unrelated to that.

> IIUC input is still driven at 60Hz but changing this is in discussion [2].

Why are inputs synchronized to the frame in Wayland? What the f---? That's just... wow.

The logic is that it doesn't make sense to deliver multiple batches of input in a single frame since the application will only get the chance to render once, maybe as well batch up a chunk of input once per frame.

The logic isn't perfect and the implementation is flawed but I see where the decision originated.

The app only gets to present one frame per compositor frame, but that doesn't restrict the app to rendering only one frame. This purely seems like a power-saving feature; it might not be entirely unreasonable in battery-powered devices, but everywhere else adding yet-another frame of input lag to systems that are generally already noticeably worse than the competition - not good.

Since the compositor is already using vsync, it's generally pointless to run graphical applications under a compositor with vsync as well, since that only adds latency. [2] Hypothetically that can result in tearing, however in practice that doesn't matter, because the scanout of the compositor is basically a copy in GPU memory, so you'd have to be rather unlucky to see tearing, let alone consistent tearing. [1]

Of course, OpenGL doesn't support triple buffering (since it has fixed definitions of exactly one GL_FRONT and exactly one GL_BACK buffer per framebuffer/output surface), and it seems like at least nVidia on Linux doesn't support it for Vulkan, either (no mailbox present mode, only FIFO).

In any case, on X11 there is a huge difference between running a 3D UI on a compositor with (A) VSync enabled, (B) running without VSync (which introduces unncessarily high GPU loads, since UIs are generally super-quick to render and if your app architecture is correct, you should be seeing thousands of frames per second), (C) running with a frame limiter (careful choice of limiting FPS to avoid common multiples of 30), (D) rendering a frame as soon as at least one event has been posted, which requires some explicit synchronization work and thought (worker threads have to post events to wake the UI, animations need a timer). (D) is what QML does and what I'm implementing currently. It works well, latency seems to be indistinguishable from B) and C), without the unnecessary power use. X11 only seems to deliver events at a rate of somewhere around 250-400 per second, so an explicit FPS limiter is unnecessary.

[1] If you are rendering to a monitor, scanout happens approx 95 % of the time, so tearing is very likely. Let's say your app's window is 1451x1400 pixels big, then its framebuffer will be about 8 MB. The "scanout" will take something like 15 µs, so at 60 Hz scanout happens about 0.1 % of the time.

[2] My angle here is primarily applications that happen to use <insert 3D API here> as their UI rendering backend. Games have slightly different considerations and should generally just run full-tilt with triple buffering.

Absolutely. I have a laptop with a 1080p screen that's like 15-inches and a 27-inch 1080p monitor that I connect to it.

When I scroll in MS Word, there is this insane black tearing across the white surface when I scroll, but only on the laptop monitor.

It's A WORD PROCESSOR! It's just a word processor... It's.... weeps

I have a similar setup, but experience no such issues. maybe it's specific to your hardware/drivers?

Maybe, but I have drivers for the monitors and everything else factory default windows install. It’s some ASUS predator gaming laptop that I inherited, but it’s reasonably powerful.

You may need a driver for the laptop's video card. The 'tearing' you describe is very characteristic of Microsoft's generic display drivers, perhaps for some reason the OS was unable to auto-install the driver?

Or the graphics card isn't being used for some reason? E.g., if your laptop has NVidia's Optimus[1], the graphics card may be inactive while Word's running. There's usually a way to force applications to use the GPU with Optimus.

[1] https://www.nvidia.com/en-us/geforce/technologies/optimus/

Definitely have actual Nvidia drivers installed.

Predator is an Acer line, not ASUS. Are you sure Word is using the nVidia gpu and not the Intel (on processor) gpu?

If you see this, a video of that would be interesting to see. (I'm not entirely certain where would be a good spot to post it in a support context though...)

Windows 10 has improved this earlier this year (2004 = 2020 April): https://www.reddit.com/r/Windows10/comments/f57tk4/dwm_multi...

I just yesterday tried current stable debian with kde and after installing nvidia drivers it started vsyncing to my fastest display, not the slowest. (I checked with glxgears).

This is why I still have a soft spot for the original MacOS (before OSX). The hardware was 1000x slower but it felt faster in that it responded instantly to user inputs. And that's because the OS was built from scratch as a nearly hard realtime OS. They built it that way because they had to: The RAM was so small and the hardware so limited they didn't have the luxury of seventeen layers of API cruft between every mouse click and something changing on screen.

People put up with slow responses today because many programmers are simply unaware that instantaneous response architectures are possible.

I remember the file copy process completely stalling, when I just pressed a mouse button. Is that what you mean? :)

Most operating systems and software throw tons of junk into the critical path. I don't believe they're anywhere near soft real time, and I bet soft real time would work pretty well. You can miss a couple deadlines by a couple million cycles each and still have instant response.

(And a real-time scheduler isn't enough by itself, the many layers of architecture around handling input events can still be bad even if you're meeting every deadline there.)

Oh this is so frustrating with today's OSes. While it is easy to see latency as being an issue with media centric applications (gaming, music ..), a mundane impact has been the increase in interactions where you do something like click on a menu item and the menu item item refreshes before receiving the click and then takes it as happening on a different item.

The human is still faster than the computer!

I've long been of the same opinion. There's nothing that would make me happier as a computer user. Delays in feedback are unacceptable.

I feel like even "soft real time" is too generous for Android.

Yeah just try using your phone while Google's updating some crap in the background again... Doesn't depend on the newness of the phone either — this happened on every Android phone I've owned since 2011.

Not limited to Android. On iOS for example Safari loading some ads or whatever causes so much input lag that the keyboard can't keep up and mangles the input completely. And on some websites that one just shouldn't visit without rigorous blockers (ehrm reddit) the device becomes so sluggish to respond that taps on the close button are not registered consistently any more.

What, you don't enjoy the back-button+muscle-memory keyboard/navigation tango?

I feel like your comment gives voice to what I wanted to say in the thread about why doctors hate their computers: https://news.ycombinator.com/item?id=24336039

https://64.ms/

For the lazy:

"""In an interactive software application, any user action SHOULD result in a noticeable change within 16ms, actionable information within 32ms, and at least one full screen of content within 64ms."""

click Feed - Not Found

click About - Not Found

to be fair, the empty white not found page did open probably under 64ms...

Came here to say this :)

Yeah my main site is at apitman.com. I'm in the process of overhauling some plumbing. Haven't decided how to handle situations like this where I have a single post which has it's own domain. Here's the actual post: https://apitman.com/13/#64-ms

If there's only one post, and there is low immediate probability to change that anytime soon, "can add additional posts" feels like premature specialization, and I'd probably go have fun with a custom design instead.

</subjective_opinion>

>I've used a hard real time OS for a desktop for some years

Sounds interesting enough to try. Which operating system is this ?

I’d like to know too

This seems to be the answer: https://news.ycombinator.com/item?id=24346411

What hard realtime OS can be used as a desktop?

this sounds like a really cool way to operate and it would be really interesting if you could expand on your experiences.

I worked on a project that used QnX for large volume message transmission via custom telex interfaces. Millions of messages on a good Monday morning. Because I like my development machine to run the same OS as the servers I ended up using it for my daily driver for quite a few years. In the end I liked it so much that when Quantum Software dawdled on their 32 bit implementation that I wrote my own version of the OS.

One really neat demo was 250 windows running the size of poststamps with a little bouncing line demo inside them. All still received their 'fair share' of time, all could still be moved around and expanded as though the machine was otherwise idle.

About as good as the Amiga.

Would you have that OS you wrote up on github or anything, would you? Big fan of QNX; like to look at OS code.

Ok. see https://jacquesmattheij.com/task.cc

That's the scheduler. There are a lot of moving parts to running this code, I may make a little project out of restoring it to life under a VM at some point.

Happy reading.

Edit: reading it myself, wow, my English was cringe worthy back then. Moving to Canada certainly fixed that.

Awesome. Thanks. FWIW, I think there's still a place for 32-bit OSes if you never port to 64-bit. Small devices, IoT, or this mc68030 thing I've been building. Lot's of opportunity for fun.

Sure, but if I'm going to spend that much effort I might as well make it future proof. The first iteration of that project cost me two years of my life, and that was when I was 27. To do this again today would be on a relative scale a much bigger investment in the time that remains.

And what about IO ? What will you do when everything is stuck waiting for that spinning rust to position itself under the drive head ?

That's just another user process. So everything else will continue. You have to let go of your macro kernel mindset if you want to make sense of the timing in a micro kernel environment. Blocking threads is fine. Just make sure that you don't do that sort of stuff in your UI thread and you won't even realize there is such a thing as disk or network IO.

Mouse, keyboard, screen. Those are the things that should run at high priority. Everything else can - quite literally - wait.

> Mouse, keyboard, screen. Those are the things that should run at high priority. Everything else can - quite literally - wait.

I might add audio to this list.

Ah yes, of course. Sorry, I wasn't thinking clearly, just had the usual UI loop and regular interaction with a computer in mind, you are 100% right, audio should have a very high priority. Nothing more annoying than dropouts. Incidentally, the way most OSs deal with that is by having very large audio buffers which in turn will give you terrible latency. On a hard real time OS you could reduce the size of those buffers quite a bit because you can guarantee they are filled (and emptied) regularly.

This is not straightforwardly true. Using small buffers and real-time scheduling works, but it gives terrible power efficiency on a modern high-performance CPU. What you actually want is a scheme with large buffers that can throw out those buffers if something requiring low latency happens.

A lot of usecases require the audio to be low latency all the time. And I don't just mean professional recording studios or musicians, super mainstream things like gaming (especially VR) are way more immersive with low latency audio.

And video conferencing also gets a lot easier, echo cancellation is so hard when you have a lot of data in flight because you will have to do auto correlation on a much larger amount of data.

Video conferencing also fundamentally wants low latency.

Listening to music, on the other hand, does not require low latency except in response to user input.

Well, consider search-as-you-type - you're typing, this goes into some kind of a query into a database. How should the whole UI react ? Display entered text, without updating search results ? Reflow it with best match when results arrive ?

Those are two different processes. The display of the field should be high priority, the lookup should be at least one priority lower.

I remember reading iOS have been handling touch input interrupts with high priority (vs normal priority in android) along with HW accelerated UI Kit ~ 5 years ago in topics such as 'Why iOS is more responsive than android'; I presume android now does the same or has improved in other ways as the latency in even low-end phones aren't drastically slow when compared with iPhones.

I believe the original Pixel had very high priority for input. It felt so much better than any other non-iPhone I had used before. They were working on it around 2016 for sure. https://www.androidpolice.com/2016/04/04/google-open-sources...

And in addition to that, there's the problem of some apps doing too much work on the UI thread.

That should affect the app, but I have seen it cripple gnome desktop too. That should not happen with Wayland.

I mean, on macOS, when something's using too much CPU, or when there's not enough RAM, the foreground app sometimes freezes as you type and loses half of your keystrokes. That annoys the hell out of me.

> That's because our operating systems treat user input as a soft real time problem when actually it should be a hard real time problem.

Please expand on your reasoning here. Maybe hard realtime is optimal for a kiosk or appliance, but a general purpose OS treating "user input" as a hard interrupt seems problematic. I see merit in your position if we had a sub-category of "interactive OS" for general purpose end-user devices.

Which OS? What is your setup?

Probably QNX or VxWorks.

Also RTLinux kernel: https://en.wikipedia.org/wiki/RTLinux

RTLinux is soft real time, FWIW.

Can you elaborate? The wiki says it combines hard-realtime with soft-realtime. It sounds like it depends how you configure it but this is the first I’ve read of it so I could be mistaken.

They're just mistaken, RTLinux is hard realtime. It works by running a tiny real time OS on the bare metal, and then running a Linux kernel as a (lightly) paravirtualized lowest priority task. You write your hard realtime code to run on the RTOS, and then your soft/non realtime code runs on Linux.

In this model, your compositor, input pipeline and (at least the UI latency sensitive part of) your applications would have to be ported to the RTOS, which makes this pretty infeasible. But it works really well if you have a some hard-realtime control loop talking to a bunch of non-realtime network IO or UI code. Would be a fun way to build a wifi controlled quadcopter.

These days you probably want to use Xenomai, or possibly RTAI.

Yes, I was mistaken. I was thinking of the preempt_rt patches.

Thanks for the great writeup correcting me!

RTLinux is a special supervisor below the kernel that allows you to run "hard-realtime" programs outside of the Linux environment with a lot of restrictions. Everything within the Linux environment is soft. As far as I can tell, it's not feasible to run the GUI stack in the more primitive environment.

QnX is correct.

Is it feasible for mortals to get hold of and run QnX as a desktop OS?

Apparently not really: https://news.ycombinator.com/item?id=24346411

You might find an old archived copy of it but nothing that would be very useful if you intend to run a modern browser and other tools like that. For curiosity's sake you could.

There used to be a one-floppy installer with the desktop on it, I'm sure you could get that to work in a VM.

Lots of screenshots:

https://www.operating-system.org/betriebssystem/_english/bs-...

Docs:

https://www.mikecramer.com/qnx/qnx_6.1_docs/sysadmin/intro.h...

Unfortunately I can't seem to locate any disk images, rumor has it there was a VMWare image floating around.

Absolutely. The thing you need to be asking for is a hobbyist license, which might not always be advertised but is very much a thing!

I emailed QNX a couple years ago about hobbyist or non-commercial licenses and they replied that they no longer offer them.

It was once. I still have a machine in the closet with QNX 6.1 and the Photon GUI. It has a version of Firefox so old it's called Firebird. But after Blackberry bought QNX, they made it closed source and no longer self-hosting. Today you have to cross-compile from Windows.

Grr.

You can find QNX 6.5 and suitable license keys by digging around on the web a bit. 6.5 is probably the last version suitable for use as a desktop OS—I believe 6.6 is where they started to remove the ability to self-host, and the GUI was completely removed by QNX 7. (It's a shame, because IMO the Photon GUI is really clever.)

Unfortunately QNX 6.5 is pretty old and not too well-supported (QNX is fully POSIX-compliant so you can sometimes get things working on it without too much trouble however) but it's damn fantastic to play around with in a VM. It's actually so fun.

What is holding Apple / Microsoft / Google back from implementing this in their OSs?

Is Fuchsia hard real time?

edit: looks like Fuchsia is based on the Wikipedia Overview https://en.wikipedia.org/wiki/Google_Fuchsia#Overview

Usually with real time OSes in my experience on Linux (preempt rt), throughput performance tends to drop a bit because now the processor is responding to inputs ASAP and so other tasks need to wait. Inputs can be network/file I/O, and user inputs.

If the processor is rendering a frame and a network packet comes in, now it needs to respond to the network first, and render the frame after processing the packet. That is a very simplified example on one processor, but there are trade-offs depending on workload/inputs/context-switching is what I'm saying, and measuring latency is step one in trying to optimize for it.

Which OS was that that you used?

What OS do you use?

And does the mouse being a ps/2 input vs USB make a difference?

Right now I use Linux, unfortunately a series of corporate games left QnX without a window manager and mostly useful for embedded work. But if someone were to do a 64 bit port of it it would be an awesome workstation OS even today. And that's just the UI, under the hood it was very impressive as well. Extremely elegant and well thought out.

Are there any open source real-time operating systems?

None with a GUI. I never released mine, and it's so far behind the times now it would take major work to get it ported to the 64 bit era and even then it would still be a C based OS.

This would be one way in which the Rust crowd could really make a difference.

Redox seems like a big effort that has at least some of the properties you're looking for:

https://www.redox-os.org/

But Redox is not a real time OS is it? I've not seen it written as part of the design or as a future goal.

It's a microkernel with even the drivers running in userspace, and has its own graphics stack. Seems the ideal setting to go for realtime requirements for the UI. I don't know if they've done anything in that direction but it seems the most well positioned to fulfill the requirements jacquesm was hoping for.

If they can see it through then that's a pretty good development. But there is more to QnX than just 'realtime' and 'microkernel', their whole philosophy about how a cluster of machines should work, the services location and so on is very well thought out. I'd be happier if Redox took some more lessons from Erlang and QnX on board than to do a re-invention. We have a lot of knowledge about systems like that lying around and re-inventing the wheel is fun but also ultimately will lead to having to re-learn those lessons the hard way.

Perhaps you should lend a hand then? :)

Not for large systems, but in the embedded space there is Nuttx (now managed by Apache foundation).

Blackberry BB10 was based on QNX. The UI "feel" was fantastic.

Yes, it was fantastic - minimal, elegant, simple, fast, had low learning curve and everything had a description - even icons in action bar[0]. As their competitors, namely iOS7+ and Android 5+ totally replaced gradients into flat design I was very welcoming the balanced UI redesign on my Z10[1]. However, autocompletion on their touch keyboard was probably the best [2], you had 3-5 suggestion that were spread around on the keyboard and to use them I had to place a finger below and flick it up. On BB Passport it was even more pleasant with physical keyboard - it had small gesture sensor and reacted on swipes in the air above the keyboard.

On the developer side, documentation was also amazing. There are tens of full, working example applications. BB10 used QT behind the scenes (Cascades UI). It had also a Python (2.x) binary on the device among other typical UNIX programs.

[0]: https://developer.blackberry.com/devzone/files/design/bb10/i...

[1]: https://developer.blackberry.com/devzone/design/bb10/10_3_vi...

[2]: https://developer.blackberry.com/devzone/design/bb10/keyboar...

I've been involved with Qt for a long time and got a Z10 as a conference freebie, but the great developer documentation is actually news to me! I prefer to just use phones, I develop for PC hardware for fun and embedded platforms for money these days. At the time, some of my coworkers helped with porting Qt to QNX. I worked on a different Qt project on QNX and I was also duly impressed by the technical quality and elegance of the OS.

The BB10 UI was built on something like QML (the language and runtime) from Qt 4 with their own UI elements and an OpenGL based backend from the acquired company The Astonishing Tribe. They had animations e.g. for slider switches running in the render thread, perfect 60 fps. Qt Quick (the UI framework based on QML, colloquially called "QML") only got an OpenGL backend in Qt 5.

Another very good Qt-based phone OS (after the Nokia N9, got one of these at a conference as well) that failed :(

By the way, the Ford Sync 3 IVI ("in-vehicle infotainment system") is also based on QNX and Qt and it received fairly good reviews. I think I made some tiny contribution to it, if only helping a coworker with something.

> And does the mouse being a ps/2 input vs USB make a difference?

The difference used to be very noticeable. Nowadays my PC doesn't have a PS/2 port.

Is the difference due to changes in the operating system or is it due to differences in hardware usb vs ps/2 inputs?

Mainly due to hardware :

- USB works by polling for changes at fixed intervals

- ps/2 works with interrupts, so the OS will know immediately when hw does something.

What about USB mice with 250-1000Hz polling rate. I can't imagine it's still a problem.

I might even argue that it’s a worse problem.

A kid who asks you 1000 times a second whether you are there yet is going to know within 1ms of when you get there, but I imagine we’d all like being the driver much more if there were no “are we there yet”s and all the kid needed was one “We’re here” exactly when you arrive.

As I understand it, USB2 drivers don’t actually poll — the host controller chip polls the mouse. On USB3, there’s a real interrupt mechanism.

There is in fact an interrupt mechanism for input devices - the host sends a request with a long timeout and the device responds to the outstanding request when it has something exciting to report, or when the timeout is about to expire. This is used frequently in low input rate devices like keyboards. Gaming mice use a high polling rate because it saves you the effort of keeping track of outstanding requests - you just report your current state immediately on every request, and the driver end can decide how often to ask you.

I may need to update my knowledge (I hope I do if you’re right), but from my understanding USB is a CPU-emulated protocol, which is to say there are no pins for “mouse button”, and the CPU has to do some “figuring out what this data means” work for each input event. Then, all this is done on a multitasking CPU that may or may not prioritize the interpretation and processing of that input.

That all being true, having a publish/subscribe model is still not the same as having a true interrupt (and I’d have to do some digging to figure out whether there wasn’t still polling somewhere in the pub/sub pipeline)

While this is intuitively obvious, it doesn't really always play out that way in practice since interrupts can be expensive to handle. For example polled IO can be significantly lower latency than waiting for interrupts.

You’ve piqued my curiosity. Please explain.

Essentially you're spinning waiting for the event that you know must come. Your latency will be on the order of a few clock cycles. Whereas if you use interrupts you are always at least a state-save and state-restore underway before you can service the next event, and likely a lot more than that.

The difference is real time control of stepper motors with crazy speeds when microstepping at 300K steps / second or more interpolating across 5 axis vs moving them at a snails pace of maybe 10K steps / second (on really nice and otherwise idle hardware, miss a single step and your goose is cooked, now you have a cumulative error).

I thought I'd seen a rather dramatic benchmark of this but I can't find it now. (I think it was in the context of io_uring, whose documentation mentions that polled IO is much lower latency and I'd trust Jens Axeboe on that one.)

This intuitively makes sense especially with SSDs and especially with NVMe SSDs: you're not going to have to wait that long, especially compared to the overhead of an interrupt. Also, if you're reading sequentially, once you get one I/O completion you're going to get a lot more in a short timeframe—it makes sense to just poll for those instead of handling every one as an interrupt. Interrupts are pretty expensive (I honestly don't have a good grasp of how Linux interrupt handling works but it can easily be on the order of microseconds.)

It also has some side benefits like preventing the CPU from going into a lower power state for only a short period of time.

Does a serial port poll or interrupt?

If it’s possible to interrupt on a serial port, are there existing examples of how to configure a mouse or keyboard to interrupt over a serial port?

It can do both, but I believe you need hardware flow control working to get proper interrupt behavior for that. I don't think any mice actually did it that way back in the day.

All serial chips that I'm familiar with since the 8 bit days would do interrupts, the only time that I worked without is when we were just using GPIO pins to emulate serial ports.

Depends on how you configure the chip. You don't actually need a chip, you can bit-bang serial just fine if you have accurate enough timing.

Typically a serial port would contain a small buffer which would fill up, upon completion of the first byte an interrupt would be generated and you'd respond to that and read out the register freeing up room for more bytes to be received. Transmit the same but reversed, as soon as a byte had left the shifter in the chip it would generate an interrupt so you could re-use that space for more bytes to send.

This works quite well. Hardware flow control can help in case the OS doesn't respond fast enough to the interrupts, so you don't lose characters.

What operating system do you use? I'd like to try it out!

You mean software vs hardware? Can you elaborate?

It has to do with the architecture of operating system. Real-time operating systems prioritize and preempt tasks based on their priority value - no matter how inefficient it may be; whereas operating systems such as Windows and Linux try to optimize throughput and speed over event based priorities.

More to read here: https://stackoverflow.com/questions/5452371/why-isnt-every-o...

Low latency > high throughput in a real time context.

I assume it means the user input handler is given a deterministic time slot to do stuff. Regardless of what the operating system is doing, it will always schedule mouse movement (or something along those lines). It's a software implementation (scheduling), but it requires hardware that can support it - usually you need fine-grain control over interrupts. So typically nowadays you see RTOSes on microcontrollers (eg ARM). Maybe more precisely you need access to that fine grain control and that often means getting hold of datasheets that are under strict NDA (eg Broadcom).

RTOSes are often found in autopilot and vehicle management systems controlling critical peripherals or safety critical software. More mundanely there are also sensors that will get upset if you don't give them undivided attention for fixed periods of time (or if you interrupt them, they will terminate the transfer). Image sensors are particularly picky about this.

You can implement hard real-time scheduling using polling instead of interrupts, which is usually faster anyway. It just changes how you measure your tolerance, since nothing is instantaneous, ever.

This! Automotive hard real time is polling based.

I think most things are trending that direction, even in soft/non real-time. How fast can an interrupt respond these days anyway, a dozen microseconds?

Nanoseconds if you can respond inside the interrupt handler ('top half' in Linux.) Microseconds to milliseconds with significant variability if you need to be scheduled ('bottom half'.)

And that only if your codepath is deterministic. Linux can do pretty funny stuff at times.

I've run a plasma cutter under Linux on a run-of-the-mill VIA SBC, it was actually pretty easy: just make a program setuid root, use that to disable interrupts after you've read all your data in and then just go wild on polled and precision timed IO until you're done. Re-enable interrupts and return as if nothing ever happened.

Yep, more that actually being able to turn interrupts off is a barrier to doing real time work on desktop OS's, for example. Even if you can disable them globally (I think on the Pi you can with some low level C), but stuff goes wonky quickly e.g. if the WiFi soc doesn't get serviced.

It's a good lesson to play with polling versus interrupts on a micro. Susprising just how many cycles it takes to jump into the ISR.

> I've used a hard real time OS for a desktop for some years

Was this a QNX variant?

I'm wondering how you'd rate iOS, especiallly with the ProMotion displays on the iPad Pro. Do they come close in terms of responsiveness?

Yes. I don't use a smartphone, I use a dumbphone.

What OS are you using?

Right now using Linux (vanilla Ubuntu LTS).

> I've used a hard real time OS for a desktop

Which one?

Already answered, but it was QnX.

Thank you. (I just noticed the thread where it was answered.)

what OS are you running as your desktop?

Reply on news.ycombinator.com