RTB is an amazing challenge and way too much fun for those inclined to bang their heads against all sorts of problems you never run across in the "regular" world.
I've done some work on these systems, one of my favorite challenges is writing tools that segment impressions in real time, you have only 10ms to respond because you're augmenting the bid before others can bid. A RTB bidder typically has about 100ms to respond. When you're shooting for 10ms at 2.5m qps peaks you have to think about everything from kernel versions, network settings, pinning to processors, avoiding GC, logging to disk is too slow etc.