Remote workstations for discerning artists
netflixtechblog.com
netflixtechblog.com
I really wish Microsoft would just throw a ton of money at making RDP an absolutely delightful experience. Last time I checked, RDP wasn't able to use your GPU at all unless you had a very specific, no-longer-maintained version of Windows 10 Pro (the Workstation variant).
It works on Macs and Windows. It works really well with text as well as full motion video.
For comparison with my (not particularly optimized) set up going over a fast internet connection, I get something like 150ms between pressing a key and the character starting to show up in emacs (it takes a few ms for the pixels to finish switching). 10 frames feels like a lot to me. My best guess is that without changing anything drastic or eg reducing resolution, I might be able to get that down to 100ms which is still looks like a pretty big number (eg a round trip between London and New York is something like 70ms).
Anyway, thank you for investigating this. A few frames from the network seems pretty good, especially with a reasonably large delta between frames in that video)
But of course, doing that requires application level support, or at least low-level hooks into the desktop rendering system to extract video streams and treat them differently from everything else. And if the video is being generated on the fly or is an uncompressed local preview, then you may be looking at an intermediate compression step regardless, but at least it could be compression suited for video instead of for text.
That's exactly what RDP does on Windows! I don't know the extent to which controls it supports rendering on the client, but the concept is that RDP is aware of things like text, buttons, title bars, etc., and is able to tell the client things like "draw a button at (20, 140)" instead of sending the raw pixel data. With enough engineering effort (and some standardization on the application UI side) I totally think Microsoft could make RDP the protocol of choice for virtual desktops.
It's hard to imagine that retrofitting such a serialization onto the total wild west that is desktop applications could be any more successful or consistent than what the web has already got.
The fun bit, at least as I understand it, with the move to more and more Electron based apps, this is somewhat moot since those apps draw their own thing. There are still performance improvements RDP brings over a simple "blit everything" protocol, but they're smaller and smaller.
My biggest complaint with RDP in general is how many Windows apps react poorly to changing resolutions and DPI. I'm practically an expert now in rescuing windows that have decided "I was over at here on the right side of your ultra-wide monitor, so I'm still over here in this no man's land on this tiny RDP resolution."
We are supporting up to 4 monitors, each up to 4k res, and the quality we are offering (60 Hz and low latency) allows you to do real-time editing, eg. when using Premiere Pro or Avid MC running on our Cloud Computers. We are also used a lot for 3D design and rendering with Blender.
Independent review: https://www.youtube.com/watch?v=LxM4mC5hwpo
Tutorial showing how to use Renderro: https://www.youtube.com/watch?v=Uw44El8kMxM
If you have any questions, don't hesitate to contact us at info@renderro.com
Please note, that with us you don't need any additional tools to connect to our Cloud Computers (no need for Teradici or Parsec) - we've got VDI solutions built in Renderro. We are focusing on delivering Renderro as super easy to use, out-of-the-box solution with clear pricing and superb quality, so I hope that you will enjoy it ;)
Cheers!
Not sure where you got that idea. RDP has been able to use the GPU for both encoding and the apps since Windows 7 era and both certainly still work on the current version of Windows 10. The main limitation I've run into is it's still limited to 30 FPS though.
Perhaps you're thinking of the deprecation of the RemoteFX guest vGPU for the Hyper-V role?
I suspect they will make VSCode RDP plugin and call it next generation remote desktop.
You can get very close today, but you need both very low network latency (<8ms or half a 16ms frame at a minimum) and a hardware + software stack that can quickly and efficiently dump it’s display framebuffer to the network.
Getting all that to work at the same time is a real trick. Getting it to all work consistently has so far not proven worth the effort vs some other solution like just using another device.
a better alternative would instead be something akin to Dockerfiles where you assert what packages you want to be installed on top of a base system, and if you want anything changed you update the dockerfile and rebuild the whole shebang.
i'm imagining a system where the local filesystem is more or less immutable and work is saved to separate filesystem. but having worked from computer labs in schools, i know how less than ideal that is when i need to use a program that isn't preinstalled to do my work, so maybe this all doesn't work and saltstack is a necessary evil
Something like Packer?
You are right that there are hidden pitfalls. But they shouldn't be a great deal if servers are cattle. A lot of corner cases are related to trying to manage servers that live forever. This shouldn't really be done in 2021 except in rare cases.
> i'm imagining a system where the local filesystem is more or less immutable and work is saved to separate filesystem
CoreOS worked like that. Immutable filesystem for the system itself. It also had a secondary "mirror" of this filesystem that was used for upgrades (and as a fallback in case boot failed). New stuff would either come when the instance first booted up (cloud-init), or via system upgrades.
They were acquired and so my experience with them ended there.
There's also https://nixos.org/
If I was doing a lot of 3d stuff I might want more horsepower but I'm pretty happy to be able to go sit out in the park and draw with all the Internet off to conserve power.
YMMV, obviously.
Not to mention some 3D/video workflows are just unfeasible with the lack of power available in laptops.
in filmmaking, personal equipment rental companies like sendgrid have raised the level of everything now I can buy my own Red and lenses and learn my tools intimately and pay for them (when I'm not paid to bring them on set) by rental customers who are liberated by the freedom from having to deposit the replacement value with traditional rental houses which risk is now insured. nvidia should sell a licence to use the 3090 professionally and rent it out like you can with the cards formerly known as Quadro now RTX A* beginning RTX A6000 (48GB VRAM at under $3K list is incredible if you can get one 96GB at 1TB/s+ linked..) if they really want to take the visual graphics industry for good. (equally AMD can grab this business forever just as easily with a turn on a penny worth of engineering)
Architecture and geographical systems are strongly and culturally averse to parallel programming. the complexity involved has only recently been significantly helped by multiple cores - the workstation on my desk in 2005 was dual dual core 3.9GHz xeon. 32GB RAMBUS RAM. early solid state swap. (and four FC drops from a SAN shared array with enough RAM cache to hold geometry models)
and laptops can be made to run single threaded codes very quickly single core, satisfying the (self imposed limits of) potential for most "graphics heavy" programs. But it isn't at all a good situation.
Ed removed confusingly placed sentence
Sendgrid rents camera equipment now? The email company?
its trivial to provide a machine on demand, We used to do it all the time. Because the version of software are known in advance[1], its super simple to make sure they are there and ready for the user. Our machine images were like 120 gigs, with 3/4 of the software on NFS shares. (still much faster than running in docker containers)
What they dont cover is the actual important bits:
o keeping the session colour correct
o making it responsive.
Remote workstations have been about for many years. Mainly because the big workstations were too noisy to use with clients. so they used teradici cards to allow dual screen 2k screens.
The third thing they don't really cover, but I suspect thats out of scope is how they sync assets. VFX is still a file based system, which translates poorly to object storage, even worse to object storage that doesn't support seek()
[1] vfx pipelines used fixed versions of software, because otherwise everything turns to shit.
For the vast majority of this industry, the two larger players are Teradici with their PCoIP protocol (also licensed to VMware) and HP Remote Graphics Software (now known as ZCentral Remote Boost). Teradici is definitely the most widely used, but I know a couple of shops using RGS. NICE is an interesting project coming up in AWS, particularly since it's free for systems within the AWS cloud, and available outside of the cloud for a price.
I don't have any HP RGS experience myself, but with Teradici it currently supports Windows and Linux hosts, with software clients for Windows, Linux, and macOS. They also have hardware solutions (zero-clients) available so end-user operating systems aren't needed. Those clients are provided by third-party OEMs (e.g. 10Zig, Dell, etc) using the Teradici supplied chips[0]. macOS host support is coming later this year[1]. The host support is split up into two different categories: hardware with the Remote Workstation Cards which have a hardware encoder on them (uses a PCIe slot) that you plug display output into, and their software agents (standard and graphics) that don't require hardware and encode on the CPU and/or GPU depending on the agent you're using.
Teradici has also been fairly clear that software is the future of the product line, so the hostcards are not a good buy for long-term support. However they will be continuing the hardware clients with a "next-gen" zero-client coming out this summer with a boatload of new features over their older clients, but it's not really a zero-client in the truest definition of the word.
[0] https://www.teradici.com/resource-center/product-service-fin...
Azure has been taking high end graphics workstation seats in the GFX and compositing and film production industry at a clip for close to a decade now.
the hourly cost of running £30,000 worth of hardware is below £5ph, which pays for the hardware running 24/7/365 over 3 years, but is clearly a saving serving 5*8/7 active employees. [ed] and that's before factoring in floating licence economies.
The low-latency high-quality requirements for game streaming is very similar to the requirements outlined by Netflix in this post.
Disclaimer: I am the CEO/co-founder
Would really like a service that works well for casual Blender usage. Vagon didn’t crash, but was too slow, even though the ping was in the low tens of ms.
https://www.reddit.com/r/cloudygamer/comments/m0kqpa/does_an...
I've not used Salt very much and went all in on Ansible years ago. I assume they're running it in master-minion mode but the article doesn't say... Maybe not and that's why they need a custom agent as well?
Am I missing something?
If you allow less latency you can't do as good a job at video encoding, and can't allow for any buffering (client side or encoder side) to smooth out bandwidth fluctuations or allow for more dynamic bitrates. That drastically lowers the achievable quality.
There are also some interesting application streaming services out there like fra.me, cameyo, aws appstream, etc..
Fundamentally remote workstations come down to interactivity. If you assume a 60Hz monitor (60 cycles a second), whenever I move my mouse or interact with an application I need all the computation and draw calls to happen on the computer and return/display the modified pixels in 16ms. So that's all sorts of latency but when you are talking about a remote computing environment the largest component will be your network latency. Now humans are incredibly adaptable and so you can go beyond that but the further you get from 16ms the more your brain will have to compensate for the delay. Data can travel around 122 miles per millisecond (2/3rds C). But with all the network switches and things in between it is not a "google maps" type calculation.
Remote workstations have a few advantages and disadvantages.
Advantages include easy provisioning, secure access to data and assets, centralized backups, access to high speed networking for large data sets, easy system config updates (add storage, add gpus, etc..), opex not capex, global systems deployment without going through customs, immutable desktops, etc..
Cons include expensive ($1000/month for a windows workstation. Linux is much cheaper but Adobe and Avid make Linux hard for some creative workflows), limited to hardware provided by cloud provider (quadro's not geforce, slower clock speeds affect single threaded applications, inability for other addon cards like BlackMagic / AJA), limitations for working with HDR, high resolution (beyond 4k), multichannel audio, multiple monitors, VR headsets, require a stable and consistent connection, etc..
For certain creative applications the flexibility of remote workstations is really great. For example if you need to access 100TB of raw video footage and 3d assets. For others (for example where you are painting on a canvas, need to hook up a peripheral like a wacom tablet, or need to hook up a VR headset), they can be non optimal depending on the user, video drivers, application, etc..
There is also an emerging hybrid coming out which leverages browser native applications so you get the local interactivity but offload the processing and assets to a backend in the cloud. Things like Storyboarder, Photopea, or the Blackbird Editor.
Teradici got there start over a decade ago primarily shipping ASICs that would connect to the video out of your GPU via physical cable, encode, and then stream over IP to a zero client (Wyse, eVGA, 10zig) that had another ASIC that would decode. Popular in the military and other areas where you want to provide a terminal. They excelled over other stuff like NX based tools or RDP for a couple reasons: 1) Full screen video playback 2) Optimized protocol for crisp text gives it an edge over the others that are x264/x265 streaming. Teradici will actually dynamically modify their stream based on what you are doing. So working in a DCC you get their crisp text goodness, but if you switch to full screen video they will switch over to x264/265 3) USB bridging for peripheral support 4) Local termination of the mouse cursor. So while draw calls may be delayed (think a trailing line when drawing), the mouse cursor is snappy which helps with latency adaptation.
The most recent version has gone all software and is actually what powers AWS Workspaces product. They also now support 4k and 10 bit color with their Ultra product. Audio is still limited to stereo.
NiceDCV was acquired by Amazon a while back and while it is "free" for ec2 instances, and does support 7.1 audio on Windows, there are some things it doesn't do as great some of the others.
Parsec was originally focused on the remote gaming space and I remember reading about it on reddit years ago where people were using it with Paperspace to run games remotely. Their newish Teams product is pretty cool moves them from a single user type use case to definitely worth a look for enterprise deployments
AWS WorkSpaces purports to have machines with meaty GPU's, are they not powerful enough?
Their Graphics Pro instance was okay spec: 16 vCPU, 122 GB memory, 8 GB VRAM. But they charge $1000/month for it.
Our $3000 workstations beat it in every benchmark, sometimes by 3-5x and have a useful lifetime of at least three years.
I'm sure for some people, that is a valid trade off, but for us, it was just too much.
The $100,000 GPU can be beaten by a $5,000 consumer space CPU for raw compute but is basically 3 to 4 times the amount of RAM.