Also consider fan-out. GC pauses add to the long-tail of latency. If your service depends on many backend services, your overall latency may well depend on the maximum of all your backends.
1. Allocate all your structures up front.
2. Your main video processing loop (or compression, encryption, signal processing, etc) does not do any memory allocation.
This is also like writing games in Java.
Even if it still isn't quite here today.
Memory safe system programming languages[0] with a GC by default, value types which can be allocated on global data/stack/registers, generic programming, ability to have more control over resource management than just finalizers, allocate memory outside GC control when performance requires it.
[0] .NET Native is not yet there, but is incrementally getting a few of the System C# features