Computer input latency from 1977-2017 (2017)
danluu.com
danluu.com
My experiment got down to ~10 ms average tap latency on a 60 Hz phone with a touchscreen scanning at 240 Hz: https://twitter.com/kdrag0n/status/1291213993219039232
Source code: https://github.com/kdrag0n/touchpaint
I have to say that writing on phones really feels much more interactive and natural when the latency is so low. I don't personally feel like these ranges of latency are that big of a deal on desktop computers where input is indirect, but it feels much more significant on phones where your interactions are right underneath your finger.
[1] https://www.macworld.com/article/3402336/apple-pencil-change...
It's the same reason I'm not a big fan of mushy keyboards, or having to ssh between continents.
Alacritty used to make kind of a big deal of these "benchmarks" on it's main page, and they don't any longer because the software improvements in the terminal are likely completely overwhelmed by whatever keyboard/monitor/OS you happen to be using.
That's the point. If you really care about this stuff, the terminal isn't exactly the place to go improve stuff unless it was way worse than it should've been in the first place.
But, rendering the entire terminal buffer is slightly overkill when only a few pixels changed, and the time it takes can end up significant on weaker devices driving high resolution displays.
As a result, Alacritty doesn't have the best "time to light" performance, and there have been long debates in the issues about it.
It's a few years old now so I've no doubt that Alacritty's performance has improved, but I'm suspicious of the idea that Terminal.app is noticeably slow.
I assume the extra forced compositing/rendering/whatever being imposed by antialiasing is to blame, though I could be wrong.
Catalina: https://i.imgur.com/G66HpJ6.png
Big Sur: https://i.imgur.com/R5dlzdj.png
rdar://FB8901170 if any apple employees happen to see this :)
Low latency is important for me, because I use Vim inside Tmux inside Xterm inside Xorg and latencies add up.
Example of ROG Phone II results:
Kernel module, display running at 60 Hz: 10 ms
Android, 120 Hz display: 32 ms
Flutter, 120 Hz display: 30 ms
They're only this close because Android gets 2x the refresh rate here, but it's still 3x slower.
Running list of latency tests on many different phones: https://docs.google.com/spreadsheets/d/1mahGpTKZLgKpaBDvcNR7...
Or if Vulkan is more your jam, doable with: https://www.khronos.org/registry/vulkan/specs/1.2-extensions...
And then "no-latency" input events can be done via View#requestUnbufferedDispatch ( https://developer.android.com/reference/android/view/View?hl... )
I'm sure there's then some way to achieve front-buffer rendering in a software rendered pipeline as well to cut out the GPU latency & avoid tile-based artifacts.
I did try requestUnbufferedDispatch though, and it didn't make a noticeable visual difference, but I need to measure it to make sure.
I also find the responsiveness noticeable when typing. You'll be able to tell that the text appears faster on the screen.
I wouldn't say it's a requirement. But it does make the user experience nicer.
Though in my totally subjective experience it feels better.
Interestingly the person who did this latency test also did a keyboard latency test:
https://danluu.com/keyboard-latency/
Compared to the slowest keyboard measured it's possible to shave 45ms which if you were latency sensitive would be the biggest reduction.
That's disappointing. He isn't measuring latency nearly as much as he's measuring point of actuation.
With a clicky switch I expect it to actuate when I hear it. With a tactile switch I expect it to actuate when I feel it. With a linear switch it depends on the switch. You can get linear switches that actuate as soon as you'd like.
The more I think about this the more useless it seems. Several of the keyboards he tested come with your choice of switch, different switches with different actuation points, and so you can't just say Keyboard X has a latency of Y without mentioning what switch you were using if you're going to measure from the beginning of the keypress.
Another factor is that some (many?) 60 Hz displays buffer a whole frame themselves, and often don't have quick response times. If you go from a 10 ms response time IPS screen with a frame buffer to a 120 Hz gaming screen with 2-3 ms response time, you already got a difference of about 25 ms just in the screen itself.
8 ms is hard to notice. 50 ms less so.
The difference is pretty huge, even on systems that are much better tuned than Linux desktops (e.g. Windows 10).
That being said, while it is very nice and feels nice, it's not necessary for development work; I spend most of my days developing on a system over a VNC connection through a VPN, so the basic input lag of that setup is around 200-300 ms. Gnarly yes, but not particularly bad for text input. You get used to just do everything very slowly with the mouse.
What would you suggest, or either Windows 10 or Linux, to get the lowest latency in a terminal?
As per the linked article, the observed delay improvement isn't one frame, more like ~3 frames. It doesn't go up from 1/24 to 1/165, it goes from 2.5/24 to 2.5/165.
Computer software waits a lot more than it did in 1977. That's why 240hz displays feel much more snappier even if it's supposed to be less noticeable -- you're waiting for same 3 frames, but they pass by much faster.
Now whether you will notice that can depend what environment you are in. If your editor already has an input latency of 100ms then shaving 8 off probably is not noticeable. But going from 20 to 12 might be.
Generally, from the tests I've seen, input latency is 30-40ms on a wired keyboard on Windows.
Even at 60fps, you should be able to see the difference between 30ms and 100ms. Newer phone cameras often have a "slow motion" mode that'll give you 120+ fps video, too
SOME people can see the difference between 30ms and 100ms, but it's almost impossible to actually see it. You'll mostly end up feeling that everything sucks and not knowing how or why.
I guess it's because the latencies stack on top of each other?
It helps both slightly with latency directly, but it also gives you more points to sample from for input prediction which then helps with latency even more. Or for things like drawing applications it'll give you smoother curves assuming the drawing app looks at all intermediate touch samples.
I recently got a 165 Hz monitor for my decade-old PC (Sandy Bridge era) and with my (now old) G302 mouse, it's like having a new, much faster PC.
The same logitech mouse performs much faster through their universal wireless adapter than trhough bluetooth (it has 2 modes), they also have lightspeed adapter, but I haven't noticed much difference.
It's wild that the only hidpi 32" monitor is ~$5000, and it was released a few years ago now.
My eyeballs need that sweet crisp text.
The other problem with hidpi scaling (200%) at 4k 32", is that everything is huge, and you lose a lot of screen real estate.
Honestly I think 8k 36" is the sweet spot.
I'd be happy with like 8k at 34" or so.
The problem is that you need a font at exactly the right scale to realize this advantage. The right scale is often different for every user/monitor pair and changing the scale means redrawing the font which is a ton of work.
I'm not convinced the difference is obvious.
You should be farther from your monitor than you are from your laptop.
See eg: https://tonsky.me/blog/monitors/
The myth of 1x, 2x, and 3x being "preferred" is just an Apple-ism because their UI toolkit is pixel-based. That's a flaw in their toolkit, not an inherent technical constraint. Combined with then Apple scaling it yet again after it's rendered.
Other platforms are not nearly as badly designed.
And a GTK-ism.
Apple-ism? Please. Most plarforms and GUI libs are pixel based, and we always have photos to see and other such bitmap assets.
The largest platform, Android, is not pixel based. The largest desktop platform, Windows, also went through the effort to try and do things properly here, letting the app handle DPI instead and then providing updated toolkits to do that automatically.
All applications i use on windows support scaling natively, and provide crisp text at 150%. It's not 2010 any more, this is a solved problem. Anything that doesnt should be killed with fire.
In fact, it is common to recommend a vertical viewing angle of 30 degrees. Not more as it tends to increase eye fatigue and neck pain. If you follow that recommendation, what matters is the definition (the number of pixels), not the resolution in DPI.
So, let's run a few calculations. The "retina" resolution is based on a pixel size of 1 arc-minute, that's 20/20 vision, at 30 degrees, that's 1800 pixels. 4k is 2160 vertical, so that's about the limit of human vision. So, basically, 4k is what you want at any size.
8k is not useless but you are pushing the boundaries here. In order to notice it, you need perfect, over 20/20 vision, high luminosity and high contrast. Beyond 8k, you enter superhuman territory, with an exception: you can notice discontinuities at a much higher resolution (vernier resolution), but it only matters if you don't have anti-aliasing. And of course, high contrast, luminosity and perfect vision.
There are exception. For example there is a limit on how close a screen can be, so having 4k on a tiny smartphone screen is mostly useless. The other end of the spectrum would be VR, with fields of view over 100 degrees, 8k per eye is considered a minimum for an immersive experience.
No, because I don't want to move my 32" monitor further away to get that 30 degree viewing angle. The reason I have a 32" monitor for work is to have more screen real estate. A 30 degree viewing angle works for watching movies and stuff, but when I use it for coding, I essentially have multiple 30 degree viewing (on-screen) windows.
You might say "ok well just get two or three monitors", but that isn't the same either. Besides the space between the monitors, with one large monitor I can subdivide my screen space in any way depending on what I'm doing, where each window has a 10-30 degree viewing angle or whatever.
>you can notice discontinuities at a much higher resolution (vernier resolution), but it only matters if you don't have anti-aliasing.
That's just not true though. It does matter even with antialiasing, the difference is clear. In particular, the dell xps 15 2019 has a ~290 dpi OLED, which has high contrast.
For movies, THX recommends a 36 degrees horizontal viewing angle[2], which is about 20 degrees vertical. Recommendations vary, sometimes it is 30 degrees, sometimes it is 40, but always horizontal.
Now, no one will force you. If you prefer to have a very large screen right up your nose, that's your choice, and maybe your work environment calls for it. But it is just not what it is generally recommended and I would put it into the "exception" category. And sure, in that case, increased resolution is good.
As for antialiasing, it will not make the image sharper, quite the opposite in fact. However, if your resolution is so that it is over your visual acuity (you can't distinguish between 2 thin parallel lines and 1 thicker line), antialiasing will take care of superaccuity. That's the ability of your brain to use image processing techniques to detect jaggies that are finer than what you eye can see. If you have good vision and a 4k monitor closer to you than the recommended distance, it is normal to see the difference even with anti-aliasing turned on.
[1] https://www.viewsonic.com/library/business/best-computer-scr...
[2] https://www.thx.com/questions/thx-certified-screen-placement...
Between phone, table, and laptop, monitor, yes.
But when speaking of monitors alone, it's not that relevant an observation in practice, since most monitors, whether 27", or 32", or 24" are seen from more or less the same distance.
This is false. Misalignment of borders can be detected with a precision up to 10 times better than visual acuity [0], therefore your numbers should be multiplied by 10, meaning 40K is the optimal screen resolution, or 80K for people with particularly good vision.
[0] https://en.wikipedia.org/wiki/Hyperacuity_(scientific_term)
Anti-aliasing is the display counterpart of hyperperacuity.
Here is a paper: https://www.researchgate.net/publication/320353446_Effects_o...
Its conclusion is that with anti-aliasing, you only need half the resolution to reach the detection threshold for misalignment of borders compared to other visual acuity tests.
If Apple just sold there 27" display for 1500 bucks or something they would make a killing for folks who want a really nice pro display but don't need the overkill of the Pro Display XDR.
Why do you need 120Hz+ for productivity?
2. No one need do anything except die, but shaving a few ms of response time is nice for productivity. No one thing is critical but making sure you have a keyboard, mouse, monitor, refresh rate, and programs that aren't throwing latency out the window makes for an overall nice feeling system.
Why is this relevant? I'm seriously considering a 1440p 32:9 240hz Samsung G9 so that I can ditch multiple monitors and move to a single display with similar overall screen space just to dodge pesky multi-screen bugs like this. Just docking my laptop and using a massive screen would be SO NICE!
Only thing that's holding me back is I really want 2160p tall in that form factor. Will probably need to wait for DisplayPort2.0
I'm a huge fan of the LG27GN950. 4k 144hz glory.
Or at the size you are looking for.
Samsung 49-Inch CHG90 144Hz
It's a 49 inch, so not the same PPI. Different aspect ration too (32:9, so 5120x1440).
https://www.samsung.com/us/computing/monitors/gaming/49--chg...
Thats a PPI of something like 80, which may be fine for gaming but is not at all comparable to the high PPI monitors discussed here.
A 27" 4000x3000 display would be a dream come true.
[0] https://www.lg.com/us/monitors/lg-38gn950-b-gaming-monitor
> The interface has two main signal lines, Data and Clock. ... To transmit a byte, the device simply outputs a serial frame of data (including 8 bits of data and a parity bit) on the Data line serially as it toggles the Clock line once for each bit.
Increasing the clock rate absolutely does reduce latency on a PS/2 port.
Additionally, another thing I've noticed in the last decade is the very badly PWM-frequency-tuned LED headlights on some cars. Those engineers selected the wrong LED brightness and tuned it to terrible frequencies which leave flicker-trails when you look at or away from them.
Get those PWM frequencies above 500Hz please! Especially get above 200Hz with your LED frequencies at the very least.
Even then, the problem is that eyes don't work like cameras or monitors. Our eyes don't work with "frames". Each receptor updates on its own time. It's easy to see that, if the updates are staggered, multiple sensors could perceive higher frame-rates, even if they can't individually.
However, there's another angle to this. Disregarding input lag, which is a very real phenomenon and is greatly shortened by higher refresh, higher refresh rate monitors are able to show more 'discrete' steps for anything that's in motion. Our eyes (and brains) perceive this as movement 'smoothness', even if they can't quite make out every single frame that's displayed.
You should try that yourself. Do a blind test.
The easiest way to show this is by wiggling your mouse around quickly on a computer screen.
At 30hz - you'll see the mouse teleporting around - hopping from spot to spot, but not moving. For example, if you stare in the middle, and jerk the mouse quickly to the right, you'll see the 4 or 5 spots where it rendered.
With 60hz, you'll see the 9 or 10 spots - and have a stronger illusion of movement.
With 120hz, it might even look as smooth as a real object flying across your screen.
"The human eye can't tell the difference past 30 FPS" was literally just a thought-killing cliche repeated by console gamers getting into internet slapfights with PC gamers.
You can see the difference for yourself here on any 60hz monitor: http://www.30vs60fps.com/
In it, they have a segment on frame rate[2], where he mentions that the neurons can only fire about every 13ms, or about 75 FPS. But, the important point, they're not in sync like a computer screen is. This means the effective update rate for a group of neurons can be much less.
It's as absurd as saying it's impossible to tell the difference between a 55" 720p display compared to a 4k one at a distance of 1 foot away.
Lower framerates are less noticeable in low light, which is another reason why films look acceptable.
*Talented animators/cartoonists can get away with lower framerates
I bought all of my other high-refresh screens because wow computing feels so much better in the day-to-day desktop because of it! Not joking in the least.
I suspect there some very big fancy rendering pipeline occurring, because when I open Windows Terminal I get the nVidia overlay popup which normally only comes up when I launch games, indicating the terminal is using a GPU-based rendering engine. Which I'm sure confers some interesting benefits, but it's a heavy price to pay when, at the end of the day, it's just a terminal.
https://github.com/microsoft/terminal/blob/main/src/renderer...
The first comment in this function (DxEngine::StartPaint), for example:
// If retro terminal effects are on, we must invalidate everything for them to draw correctly.
// Yes, this will further impact the performance of retro terminal effects.
// But we're talking about running the entire display pipeline through a shader for
// cosmetic effect, so performance isn't likely the top concern with this feature. * deferred vs forward rendering (deferred adds latency)
* multithreaded vs singlethreaded
* vsync (double buffering)
It's far cry from simplified pure math people think of when they think of fps in games or refresh rate for office and typing. Software is very very lazy lately, and most of the time these issues are being fixed by throwing more hardware at it, not fixing the code.
Some things cannot be 'fixed'. It's always a trade-off. You can't expect to have all the fancy effects that rely on multiple frames and also low latency.
If there was a simple software fix, GPU manufacturers would be all over it and pushing it to all engines. It's in their interests to have the lowest latency possible to attract the more hard-core gamers (which then influence others).
Just look at all the industry cooperation that had to happen to implement adaptive sync. That goes all the way from game developers, engines, GPUs, monitors. Sure that sells more hardware(which brings other benefits), but a software-only approach would also allow companies to sell hardware, by virtue of their "optimized" drivers.
Wah? Deferred just refers to a screen space shading technique but it still happens once every frame.
> * multithreaded vs singlethreaded
Not sure what you're saying here.
And then of course, yes display buffering does have an impact.
In principle, you can push GPU pipelines to very low latencies. Continually uploading input and other state asynchronously and rendering from the most recent snapshot (with some interpolation or extrapolation as needed for smoothing out temporal jitter) can get you down to total application-induced latencies below 10ms. Even less with architectures that decouple shading and projection.
Doing this requires leaving the traditional 'CPU figures out what needs to be drawn and submits a bunch of draw calls' model, though. The GPU needs to have everything it needs to determine what to draw on its own. If using the usual graphics pipeline, that would mean all frustum/occlusion culling and draw command generation happens on the GPU, and the CPU simply submits indirect calls that tell the GPU "go draw whatever is in this other buffer that you put together".
This is something I'm working on at the moment, and the one downside is that other games that don't try to clamp down on latency now cause a subtle but continuous mild frustration.
That said, the VR space has a much tighter tolerance on input lag and there's hardware based mitigations. Oculus has a lot of techniques such as "Asynchronous Spacewarp" which will calculate intermediate frames based on head movement(an input) and movement vectors storing the velocity of each pixel. They also have APIs to mark layers as head locked or free motion etc.
I tried it a few years ago but stopped using it for reasons I don’t exactly remember at this point, and I see that it’s still pre-1.0. Wondering what people think of it these days.
Kitty is very slightly faster w.r.t. latency though, despite their benchmarks, so I use that now.
The cmd need to do a lot of things (font rendering, layout whatever) without help of other services (It need to work even without them, see the recovery mode). So it effectively bypass a lot of interactions with other services a normal program need to do in order to put the text on screen. It is basically the tty? in Windows.
But that also means it need to sacrifice a lot of things because that aren't available in recovery mode. You get no fancy emoji supports or whateve feature you would expect in a modern terminal. And the text rendering looks very bad until recently the remake the conhost.
At least until we arrange for everyone to have company provided dedicated work laptops which we can (mostly) lock down to the satisfaction of our clients and their auditors, but the switch to mostly remote working this year happened at roughly the same time as budgets were squeezed a bit (bet you can guess why!) so that hasn't happened across the board yet.
Allowing RDP from non-secured devices "for the time being" is iffy, but we are getting away with it due to the circumstances.
The threshold for me (and most people I think) is about 110ms between key strike and something appearing on the screen before something feels non-local and disembodied... which is good because I found I could get down to ~40ms local response time on a decent windows box. My 2015 macbook is about ~120ms for the terminal app.
That leaves 70ms latency budget to play with which can accommodate the the RDP overhead (~50ms) and the network hop (~10ms for me).
This whole journey started with me wondering if I should pay a small fortune for a better network connection, before I realised that most of the latency was in my keyboard and monitor!
I even built a Arduino Leonardo gadget to measure the lag systematically and some pcap software to spot the RDP packets on the wire. The code's pretty shoddy but it does the trick! You can see it and an example result with RDP here:
https://github.com/willmuldrew/lagmeter
Another big factor in the experience is jitter - so it's a good idea to ensure everything is plugged in if you can. Mice, keyboards and ethernet.
In essence, even if you have to RDP/remote into work, potentially you can still make it feel pretty local - don't give up hope!
Latency is not really an issue to be honest. I think ping is about 15ms which is much less than the lag in this article. Right now I'm using the horrible, 2012-era work laptop with just Chrome running and there are delays typing this comment into HN which is a super lean site as it is.
The main downsides are actually
* crap single-threaded performance (don't think Azure has VMs over 3ghz except maybe the highly expensive GPU ones)
* Slow IO
* Video and audio over RDP really doesn't work.. no hardware graphics accel either... reduced ability to work on multimedia, we had one webapp where WebGL wound up behaving differently which we didnt
I'd describe hp and ti as laggy and slick, respectively. The 50g is what I used in university, but now I seldom touch it as the 86/89 are so much more pleasant to use.
In mine, lack of responsiveness makes the 50g a non-starter.
Anything less than effective immediate feedback is unacceptable for interactive use.
That paradigm simply doesn't belong in a hand calculator.
I feel like you're ignoring the presence of tactile feedback, which is in fact effective on the device in question. Possibly because basically no other electronic gadgets in your life give reliable tactile feedback anymore, so you're used to visual feedback as the only indicator that your tactile input was registered.
I used the hp50g throughout university. I'm well acquainted with it and with typing faster and navigating menus faster than the screen can update.
Regardless, there's literal centimeters of distance between keyboard and CPU. There's no excuse for the calculator UI to lag noticeably relative to the keypresses.
I got the ti-89 out of sheer curiosity, mostly because it's 68k based. After using it just a little, it was obvious to me I wouldn't be suffering the 50g much anymore.
I haven't used it with slack but it works just fine with discord. You don't get voice noise cancellation with ripcord but you don't need that if you use rtxvoice anyway.
Same deal or worse with Teams, the performance of which is goddamn execrable. My laptop is an awful 2012 thing but it shouldn't take 5 seconds to switch between chats.
https://mobile.twitter.com/programmer_just/status/1253098759...
I crave some good old fashioned responsive computing, but moreover I want to fill my classroom with these devices and show a new generation that, to borrow a catchphrase from a far more noble cause, it gets better.
Sure, they are less portable and difficult to manufacture/repair. But their combination of refresh rate (without blur), image quality, and color reproduction are still second to none.
Considering the price of an nVidia 3080, which is standard PC gaming equipment, there must be a market for high end screens for gamers.
Color and raster flexibility are the two bright spots for me.
Glowing phosphors in tubes is just a great tech.
Not true. A current generation OLED (which will do 120hz) is superior to a CRT.
Even still, on many of those a good LCD has long been better than a good CRT. Color accuracy, for example. CRTs required constant fiddling & calibration to stay good at that, whereas a quality factory calibrated LCD will be much better in an "out of the box" or typical usage. Not really sure what you're calling "image quality" but it's hard to imagine that going any way but solidly in a modern high-end LCDs camp, which are brighter, wider color gamuts, higher resolution, and higher pixel densities than CRTs ever had.
> there must be a market for high end screens for gamers.
There is, which is why we have high resolution, high refresh rate, and adaptive vsync IPS monitors a plenty now. They come in all sorts of shapes & sizes, in resolutions & dimensions CRTs could only dream about.
It just doesnt feel right, and slowmo video capture confirms it.
Also V-Sync is off now.
AMD even has an "anti lag" feature which apparently does nothing.
Edit: I have a Ryzen CPU, I've expected the full AMD system to work together better.
I'm like 80% confident I've used DVI in the previous setup, but there still is a chance that I'm wrong.
I'll probably try getting a dvi->vga converter in the next couple of days and see if this is the culprit.
[1] https://www.benq.com/en-us/support/downloads-faq/products/mo...
I also experience the lag on my windows desktop while dragging windows.
Keypress responsiveness when typing is night and day, and given where I'm posting this comment, you can guess who has the advantage.
Platform: GNU(Linux)Mint Xfce
The Windows setup is convoluted, awkward, and weird sounding compared to the native app. They are using it for a reason though: it’s much much more responsive than native Linux LibreOffice.
...which is, like the rest of the DOS-based Windows from Windows/386 onwards, itself a sort of hypervisor architecture --- DOS applications effectively run in VMs thanks to V86 mode, giving native performance. The difference is very noticeable if you open something like MS-DOS EDIT on Win9x vs. NT 4/2K/XP --- the latter runs in an emulated environment and the latency is enormous.
This was posted here few times but it's still an interesting read in this context.
I have a hunch it eliminates some...
The Blackberry q10 runs Blackberry OS 10, which is based on QNX, which has a realtime microkernel.
I suspect it's just that input processing is so cheap on those early architectures, that latency is really dominated by hardware constraints and not by CPU (however slow it might be). Certainly, on the //e, I'm pretty sure the input routine did little more than copy the character to the input buffer and the screen buffer and advance the cursor. The TI's probably was not much more complicated, despite being implemented in bytecode.
Editing with EDT felt quite snappy. Amazing.
I once read that properly written Amiga software achieved around 16ms latency and would like to see that verified or debunked.
Just about every application on Apple 2e relied on keypresses being shown on the screen. Modern devices and operating systems are displaying 'windows', streaming 3d graphics etc. and running a myriad of other services at the same time.
Even in a mostly passive app like a video player, it sucks if the play/pause button has a long-delayed effect. The tendency is to press the button over and over and feel correctly that we have lost control of the app.
Secondly, not every PC in 1999 was capable of even running QIII (let alone at a high frame rate).
During the days of true low-latency gaming, actual game frame rates were in the low 20s and even consoles rarely achieved 50 or 60 fps.
If a game was playable at 20s of fps, I don't think it deserves a title of low-latency gaming.