Apple testing new external display with A13 chip
9to5mac.com
9to5mac.com
When I saw the headline, I thought maybe it had a webcam and the A13 would do things like portrait mode, before the image reached the computer.
Apple Exec: "Build an external display for the iPhone and/or iWatch."
Apple Engineers: "Ok, but we will need a GPU built into the display or it'll look like shit. It'll also need a lot of RAM/SSD to store textures, etc. Additionally, a high bandwidth, low latency wireless connection between the iDevice and the display (e.g. UWB)."
Previously to support an enormous external display the iDevice would need to have a large GPU, battery, etc that are mostly unused except for when the display is "active". With a GPU in the display, the iDevice just needs good enough data transfer rate to get textures and stuff over to the display, meanwhile events like mouse movement, or open an app, or switch apps can be very low latency instructions sent from the iDevice to the display and vice-versa.
And we know that Apple has figured out secure pairing of T2 chips for external accessories (the touchID keyboard), so they can run secure-trusted components outside of the main hardware.
So they could in theory run a WindowServer process on the display's internal Cortex chip, paired via T2 to provide verifiable security, hosting the WindowServer canvas for that display and lets the host device offload the 'flatten layers to display' problem, handle the EDR mapping, and so on. That would be a boon to laptops and also to thermally-constrained desktops, and here's why:
On my Mac, WindowServer is the highest user of CPU time in total, exceeding kernel_task, coreaudiod, everything except Zoom (which does non-accelerated H.264 encoding/decoding on x64). So if I were to plug in a second 5K display alongside my built-in, my WindowServer CPU usage would double. Fortunately, I can afford that — but maybe a laptop cannot. Offloading that WindowServer burden for managing traffic to/from the LCD, and for dealing with layer flattening and security property management (you can't screenshot certain OS dialogs with a specific security flag set!), would be a tremendous win not only for battery life of the host, but also a theoretical performance win for latency, as now macOS would only need to shuttle specific layer changes over the wire.
Of course, it could still act as a dumb display for any host that isn't ready to negotiate a T2-protected WindowServer connection, but my idea above would definitely go a long way to explaining why Apple dropped support for x64-style eGPU while still investigating hardware-accelerated monitors.
^ it used to be implemented as PDF internally, not sure if it still is or not
^^ you used to be able to kernel panic macOS by making a window so tall that, when screenshot using the Spacebar layer selection method, an out-of-bounds write would occur from within WindowServer
All this makes me think they're about to introduce wireless connection to display, akin to what is now possible with Sidekick and an iPad, especially given the recent improvements in Continuity.
I think that OS X was the first OS that did hardware compositing. How come that WindowServer consumes so much CPU instead of GPU?
In addition/as a result of these you get some usability and user experience advantages: (a) avoiding degrading laptop performance when connecting an external display without the user having to know about eGPUs (b) no additional box/cables/power mess from using an eGPU
But I still don't think it makes sense to physically merge the two.
Displays wear out far faster than the GPUs driving them, and just have much higher failure rates in general, being physical targets for thrown handheld controllers and what not. Doesn't seem wise to make the display so much more costly to replace by shoving the GPU within it.
But this is Apple... making this expensive to replace is kind of their M.O.
Is this true? This seems nearly diametric to my experience.
In fact I dare say the GPU has been the single most failure prone component of every computer I’ve owned for the past 15 years.
Which only further illustrates the point that putting something expensive and unreliable into an already expensive device seems risky.
Do they?
In my experience they both last forever until you decide to upgrade to better specs. And in that case it's quite likely you upgrade both together.
Displays don't really "wear out" these days in normal usage.
Although I guess I'm not really sure why you say they're a target for thrown handheld controllers ha... are you speaking about yourself or about kids...?
Yes.
Burn-in is a real problem [0], especially for OLEDs last I checked.
LED backlights wear out, I've personally helped multiple people replace them in external monitors and laptops over the years.
And they're obviously more fragile and exposed to the elements than a GPU within a case.
> are you speaking about yourself or about kids...?
What difference does it make? You do realize kids are a real factor, they exist, are rather common actually. And these days gaming has become just as popular among adults too.
It's as if everyone forgot the rash of broken displays when the wii came out... surely those people appreciated not having to pay for a GPU too when replacing those displays.
[0] https://en.wikipedia.org/wiki/Screen_burn-in#Plasma,_LCD,_an...
What they maybe do want is to plug a mobile computer into a big sharp hi-res monitor and get great performance. Since many graphic loads scale with resolution, building some processing horsepower directly into the monitor could be a way to deliver the experience that people want, even if their current computer doesn’t have very powerful graphics.
Most default / "auto" profiles for monitors and TVs are accurate enough for lay people out of the box. But for people who really need color accuracy, you need a calibration device. So maybe Apple is trying to obviate the need for a calibration device and have the "smart" monitor calibrate itself.
(The built-in GPU might even use data from an iPhone like the "color balance" feature Apple announced at the "Spring Loaded" keynote recently.[1] The AppleTV uses information from the IR array on the front of the iPhone to help calibrate your TV for use with AppleTV.)
If you have a couple thousand dollar display, throwing $60 in cpu, RAM, & storage on top of it seems like a pretty dead-obvious win to me. Most companies would just have nothing particularly good to offer- Android? Windows? webOS? Ok options but not great, and they are paying a lot more for less well-integrated cpus. For Apple, they have great chips made not bought, in vast volume, and can make their pick of OSXI, TV, or iOS OS software.
I believe the new iMac 27 might not have Target Display Mode, the ability to take HDMI input, which old displays had[1]. I'm not sure that kind of capability is really needed, for long. One of the things I'm looking forward to with USB4 is that there is a really good 40Gbps host-to-host connectivity build right in to the spec. This is a generic connection, not specific to video, but I can certainly imagine doing a really good low-latency remote-desktop over a high quality connection like that. So, my hope is, medium-term, that we don't need HDMI &c input on our computers for them to act as auxiliary displays. There's already a world of softwares out there for using tablets auxiliary displays, my Linux desktop runs Sway which makes adding a virtual/remote display extremely simple, all that we're missing is a way for two computers to plug in-to one another & connect! Amazingly comedically weird situation.
[1] https://www.lifewire.com/use-imac-as-monitor-with-target-dis...
My worry with embedding an A13 or whatever is that it will require an Apple device to do anything or a bunch of hackery to make it work on a non-Apple device. Previous Apple monitors were already sometimes a PITA to get running with non-Apple stuff. It just ends up with more ecosystem lock in and that's not needed from a probably $1000+ monitor.
To be honest, both Amazon and Google has gotten this right already with no touch voice controls.
https://appleinsider.com/articles/20/09/08/apple-researching...