You can get 100x speedups, but the downside is that you wind up with big nasty SQL queries that duplicate Ruby logic and are hard to maintain. There was a nice gem that would automatically produce JSON-generating SQL [0], but it is abandoned now. It only supports Rails 4 and ActiveModelSerializers 0.8, which are both quite old. I just published a similar gem [1] that works for Rails 5 and AMS 0.10. Unlike the old gem, mine outputs JSON:API (common in Ember projects). I hope to add the traditional JSON format too. AMS 0.10 makes this easier, since it introduced the concept of "adapters". My gem is super new and doesn't yet cover a lot of serializer functionality, but I'm hopeful about supporting more & more. Feedback is welcome!
On the writing side, I've found activerecord-import [2] to be very useful. It batches up INSERTs and UPDATEs and for Postgres it even knows how to use INSERT ON CONFLICT DO UPDATE.
I also have a handful of Postgres extensions that use arrays to do fast math/stats calculations. [3, 4, 5] If you are thinking about moving something into C, it's natural to consider a gem with native extensions, but perhaps a Postgres C extension will be even better.
[0] https://github.com/DavyJonesLocker/postgres_ext-serializers
[1] https://github.com/pjungwir/active_model_serializers_pg
[2] https://github.com/zdennis/activerecord-import
[3] https://github.com/pjungwir/aggs_for_arrays