I dont understand the comment. Is this a question?
17 karma · joined December 10, 2015
What I am saying is orchestration code can be generated deterministically and then we trust it. Agreed AI can generate the code that is invoked and we need to check the workings of a function. But the order which functions are executed can be moved to a mechanical process and trusted.
If AI generates the orchestration code you would need to validate every execution path in the system. At that point a developer would need to keep a global dispatch table in their head and check the AI generated orchestration code against that mental model.
What you raise is an interesting point should a developer understand everything in a program or accept contracts and work within those trusted boundaries? We work with a 7 layer OSI model and plug into without validating it. Do you need to understand every execution path in the program? In a microservices architecture we code a service to a specification and then deal with the orchestration as a separate issue. If that orchestration of a microservice based system is left to a trusted separate team then our focus of correctness is local and not global. If that orchestration team is sometimes unpredictable we lose faith but if it is predictable we can move forward with confidence.
Potentially that leaves us with three options for orchestration: 1. The developer is all knowing and validates all execution paths 2. Orchestration is a separate problem handled probalisiticaly by AI 3. Orchestration is a separate problem handled mechanically as a compiler style operation
Similar to moving from bespoke car manufacturing to a production line. The tolerance checking had to evolve in step with the rate of car production.
I come from an electronic-trading background. When an AI model is deployed inside an operational trading system, the model may be probabilistic, but its guardrails, permissions and surrounding behaviour must remain predictable.
We found that wiring these systems is expensive, error-prone and difficult for any one developer to hold entirely in their head. LLMs can make this harder: locally reasonable changes may silently alter the global execution order and introduce subtle bugs.
My thesis is that, for a closed object graph with declared local event semantics, much of the global orchestration can be derived. Components declare local intent through event handlers, triggers and lifecycle methods; a compiler can then calculate the coordination plan and generate a fixed, deterministic orchestrator for that event processor.
I included a browser-based playground in the article so you can inspect the Java components, inferred graph and generated orchestrator side by side, without installing anything or signing up.
I’m interested in where people think the boundary should lie between orchestration that must remain dynamic at runtime and coordination that can be derived and compiled.
Fluxtion is built to handle infinite streams, I cant tell are kotlin flows for processing batched data in a co-routine
Fluxtion is concerned with graph processing and handling multiple event types, is kotlin flow more focused on single pipeline?. A single Fluxtion processor has multiple paths that diverge and converge. The path is determined by the event type. Elements on the path are Fluxtion libraries or user functions. If you see https://github.com/v12technology/fluxtion-quickstart, input events are bytes(fluxtion generates a CSV parser) or SensorReading. The flow merges and applies grouping, windowing and filtering. The flow diverges and integrates a user function to perform an action.
There is no single output in a Fluxtion processor, any node can publish data or invoke user code, it operates like a network of conditional statements
Fluxtion actually serialises the graph as code, potentially ahead of time. Other event processing systems infer the behaviour at rumtime. If you are in a serverless environment where quick startup and short runs are cost effective this is very important. For the example above see - https://github.com/v12technology/fluxtion-quickstart/tree/ma.... The entry point is https://github.com/v12technology/fluxtion-quickstart/blob/ma...
When dealing with graphs it becomes very confusing to debug and fix problems, having code really does reduce the maintenance cost.
We are concerned with zero-cost abstraction, that is trying to use as few resources as possible when processing. We have an example here - http://fluxtion.com/solutions/high-performance-flight-analys.... This processes 30 million records a second with zero-gc, including soup-nuts processing
We support annotations for injecting dependencies. Each injected instance can inject dependencies etc. recursively. I think a kotlin flow requires the user to explicitly wire together all nodes. Fluxtion encourages re-use of graphs/sub-graphs. The Fluxtion compiler will then construct a processor that merges all the nodes and serialise a processor solution
Basically Fluxtion removes the need for the developer to solve all the complexities if some can be inferred. I have an article in progress about managing an FX portfolio where hedges are executed if the risk is too high in a currency. Its interesting as events handled are config, trades, control signals, Fx rates, there are stateful calculations and orders are pushed out. The execution graph is here https://raw.githubusercontent.com/gregv12/articles/master/20...
Fluxtion is not concerned with handling backpressure it handles events as fast as it can. The pipeline sending events to an event processor is expected to manage back pressure.
I hope this helps