Ah, 10-bit 4K. That completely explains 12Gbps.
I'm not entirely sure what you mean by "1920x1080x20x30", particularly the "20". I take the "30" to mean 30fps.
SDI rings an "I've heard of that!" bell. Huh, 37MB/s of overheard - nice. :(
I get the impression analog routers/multiplexers were basically every input port physically wired to every output port, with a tiny switch to disconnect the lines that weren't being used. Yeah, those would be exponentially expensive, because even though you can't physically have more than one input going to a given output (outside of practical special effects, maybe?), you can't get away from physically wiring all inputs to all outputs. And then I'm guessing the sea of internal wiring was all super high quality and so forth to minimize line losses...
40x40 for $1.3k is _very_ impressive. But yeah, it's just a specialized ethernet switch (albeit with really high maximum internal throughput), so that makes sense. The switch from analog to packet switched routing does make sense financially.
Hmm. What did everyone use after everything went digital, but before the switch to packet switching? What video formats got used back then? What time period am I even describing? I just realized I've always been mildly curious.
TIL about ARQ, and what to call what I've been using in my own messaging system design ideas. I do wonder though... in the context of video distribution, if you just repeat the last good frame you had if you don't get the next frame in time - why does there need to be realtime acknowledgement at all? Instead of acknowledging every single frame (I assume this is how current systems work), either send periodic beacons containing the last eg 10 or 100 or 1000 last message IDs, eg [ 1, 3, 8, 9, 10, 13, 17, 20, ... ] so the sender compute the number of frames dropped, or even better, if frame IDs are guaranteed to be sequential, the receiver can compute the number of dropped frames and simply just ping back the packet loss percentage value. That would take care of the sender blindly going too fast a la TCP rate control. Then, on the receiving end, if the receiver doesn't get the next frame in time for playback system to show it, it just shows the last received frame until it has a new one.
I get the impression ARQ works a bit like Skype on a bad day, where the feed locks up periodically then the image jumps forward in time (if you will) when the next properly decodeable frame comes in - except I suspect ARQ only has very tiny such jitters and jumps. What I just described would probably be equivalent, but (unless I'm horribly mistaken, or misunderstand) with much lower latency.
Note - I realize the way things are already done were carefully engineered and there are good reasons behind the techniques currently being used. I presume that the above idea has very possibly been thought of and disregarded - and I'm curious what the reason was.
400ms encode/decode is impressive. 40-80ms even more so. But that sounds perfectly reasonable given that this uncompressed video; I expect the faster encoder simply has faster silicon in it :)
TIL about the info about the traffic peaking and retransmission. That was interesting to carefully read through a couple of times :)
Talking about encoders and distances, that reminds me - I'm curious about how good those cellular backpack things are, the ones that take umpteen SIMs and seemingly set up a load-balanced aggregate connection over all of them. I'm guessing the latency is somewhere north of "higher than planes fly"? :P
What did you mean by "temporal shifting" in the context you used it? I also wonder what the traceroute on the private wire would have looked like. I can't imagine the link was tunneled (haha!) so if nothing showed up that would be perplexing.
I remember when satellite links used to have 3 second delays to TV live news broadcasts, only around 2006-2008 or so. Hah. Wow, expectations have (perfectly understandably) moved forward.
Agh. I'm 100% sure that it's possible to losslessly compress video and get good speedups... but the realtime requirements really make it so incredibly hard :( I realize HEVC has a lossless profile, but yeah, definitely not realtime. The problem is that anything that compresses at realtime on current silicon (and not anything custom) will probably only shave off a few tens or hundreds of Mbps. :/
TIL about the network upgrades. By "network" here, I presume you mean at least the in-building LAN, but I'm not sure where/what else classifies as "production". Surely the lines to the TV towers aren't uncompressed? I see no need for uncompressed video there (but I may be sorely misinformed). And wow, multi-Tbps backplanes... I was reading about how Facebook had deployed that sort of thing a few months ago, TIL it really isn't that unique.
As for getting 10Gbps kernel throughput, I understand a lot of groups are just doing networking in userspace and skipping the kernel IP stack entirely. But 10Gbps at nanosecond timing... haha, when I hear that, I say kick out Linux entirely, because it doesn't offer hard realtime AFAIK. Well, my knowledge may be hugely out of date, but last I heard Linux didn't really like doing realtime "in production". At any rate I know of no projects using it routinely for non-specialist tasks (ie, some kind of everyday desktop-usage use case, not embedded/appliance use-case).
I vaguely reading about the SNABB Switch folks doing really really fast packet switching, and they were getting all their speedups by using assembly language and doing instruction/cycle counting. Problem with using Linux is that you have a massive bunch of overhead (the mechanics of C, basically) you can't really shoo out of the way - sure, you can optimize your own stuff in assembly, but what's the point when you have tons of C kernel code darting in and out of the L1/L2/etc caches in between schedules of your process(es)...
Hmm. I just realized... it would be so nice if you could tell Linux to give you an entire CPU core. Just don't schedule anything onto this or that CPU, and don't even run kernel code on it. Then you could use the excluded core like a discrete application processor, and run your own kernel on it. THAT would be COOL. But I don't know if you can actually use x86 like that.
Somewhat less drastically, I wonder if you could build FPGAs - or, if the chipsets in NICs are amenable and fast enough, just bespoke firmware - and teach the NIC to receive and buffer SDI video frames really, really fast, doing some complex processing in hardware before the kernel gets the data over PCI-e.
That reminds of something that may or may not be interesting: https://news.ycombinator.com/item?id=13741975 - it's a totally new CPU architecture (it seems that the CPU arch wars/competitions are starting to warm up - nice) that basically, uh, moves the MMU into LLVM, instead of putting it in silicon. It's an interesting idea, but the reason I mention it is because I understand they're already at the point where they're taking orders (and have taped-out silicon to actually deliver). FWIW I was i336_ in that thread (accidentally locked the account, heh) but apart from the interesting interaction I have no affiliation.