But session affinity is a scalability anti-pattern, which should generally only be used to support legacy applications. Ideally, you'd design your application to use stateless backends, which tend to share state at another layer (the database, memcache, Redis, etc).
If you don't want unused memory on your app servers, don't provision it. :-)
I just finished working on a real-time collaborative document editing service and chose to use document affinity so that the app server could handle collaboration directly. It is extremely performant and I'd choose this approach again any day versus farming out to Redis/RethinkDB/etc. It persists to a database as writes come in, and performs initial load from the database. Local memory accesses are orders of magnitude faster than going over the network, and it reaps the benefits.