Bunch of optimisations for large data, including: Smart Batching, Prompt Caching, Custom Routing Algorithms and hardware planning for batch.