Ember: A Compositional Framework for Compound AI Systems

Aspirationally, Ember aims to be for Networks of Networks (NONs) and Compound AI Systems what PyTorch and XLA are for Neural Networks (NNs). It is a compositional framework that offers both eager execution and graph-based optimization. Ember empowers users to build complex NONs while supporting automatic parallelization and system-level optimizations.

The long-term vision for Ember is to enable the development of compound AI systems composed of millions—or even billions—of inference calls. Surprisingly simple patterns, such as best-of-N graphs, verifier-prover architectures, and ensemble models with voting-based aggregation, perform remarkably well across a wide range of scenarios.

Design Philosophy

Ember is built on these foundational principles:

    Composability First: The ability to combine, chain, and nest components (e.g. Operator components) is central to Ember's design
    Type Safety: Comprehensive type annotations ensure robustness and IDE support
    Testability: Components are designed with SOLID principles in mind, for easy isolation and testing
    Scalability: support for Parallel execution is built-in at the framework's core. This is more Tensorflow/JAX, than classic Torch spiritually
    Extensibility: Registry-based design makes it simple to add new components
    Skeurmophism: APIs follow familiar patterns from PyTorch/JAX, to somewhat control the learning curve
    Simple-over-easy: Minimal "magic" and a focus on explicitness