Thanks for asking. If you're interested in the nitty-gritty, Yehuda and I gave a talk on the architecture behind Skylight at RailsConf: http://www.confreaks.com/videos/3394-railsconf-how-to-build-...

The short version is that the agent runs on your servers and collects information from your Rails app using the ActiveSupport::Notifications instrumentation built in to the framework. We serialize that into a protobuf that's transmitted via IPC to a background daemon (written in Rust).

That daemon batches multiple reports into a single payload that is sent to the Skylight servers, where we use Storm and Cassandra to process the requests, and periodically do aggregate roll-ups.

Unlike New Relic, Skylight gives you access to the entire distribution of response times, not just the average response time. (According to DHH, averages are "useless.")[1] This ends up being a lot of data, and a lot of CPU-intensive processing, which is why we sell it as a hosted service.

[1]: https://signalvnoise.com/posts/1836-the-problem-with-average...