Had a quick look and couldn't find anything obvious - is it easy to point to a llama.cpp server on a different machine pretending to be openai?
@action(reads=["prompt"], writes=["response"])
def my_action(state: State) -> Tuple[dict, State]:
prompt = state["prompt"]
# you stick in whatever library you want for the call
llm_response = whatever_library(prompt)
state = state.update(response=llm_response)
return {"response" : llm_response}, state.update(response=llm_response)
Does this help?