A framing from that podcast that helped me better understand the role of Bonsai is that it's really a framework for building incremental distributed state machines.
79 karma · joined June 7, 2023
Location: NYC (preferred) or Orlando, FL
Remote: Yes
Willing to relocate: NYC or Orlando only
Technologies: Python, TypeScript, Go, Rust, OCaml, Haskell, Lean, AWS, Kubernetes, Terraform, Envoy, Docker
Resume: available upon DM
GitHub: https://github.com/rdavison
LinkedIn: https://linkedin.com/in/richardndavison
Email: richardneildavison@gmail.com
As to your other point about anything you include in the prompt can and will be ignored. Yes, I agree. You could draw an analogy to how a teacher assigns an in-class reading assignment and follows it up with a reading comprehension quiz. If your mind wanders during the reading you may come to find that you will fail the quiz because “anything you include in the prompt can and will be ignored”. Therefore, the quiz result serves the purpose of an evaluation.
So for instance you could start by trying to classify the prompt in some way. If you use an LLM for this, you might need to get it to return a machine parsable data format. Then your harness can pattern match on the classification and use it to enrich the prompt with additional context. The challenge would be in determining how exactly you want to go about this, balancing tradeoffs such as accuracy, cost, time, etc..
For the classification step you might begin with something like "Determine whether the following prompt is a QUESTION or a STATEMENT. Respond using only one of the two words. Prompt: $PROMPT"
You could have multiple back-and-forths like this and at each round you gain more information about the prompt, and you can use that information to determine further classifications and/or context to include.
The tooling that invokes the model should really define some kind of guardrails. I feel like there's an analogy to be had here with the difference between an untyped program and a typed program. The typed program has external guardrails that get checked by an external system (the compiler's type checker).
https://github.com/rdavison/DXX-Raytracer-ar/releases/tag/ar...
(*
Quick note on notation: I will use "double quotes" when referring to _values_ and `backticks` when referring to _types_.
*)
(*
Think of this as an interface. It defines the shape of a module. Notice that the interface describes a module that defines a type called `t`, and two values: "of_string", and "to_string", and they are functions with types: `string -> t`, and `t -> string`.
*)
module type ID = sig
type t
val of_string : string -> t
val to_string : t -> string
end
(*
Below this comment is a module named "Id" that _is of type_ (in other words: it _implements the interface called_) `ID`. Due to the explicit type annotation (Id : ID), now from the perspective of anywhere else in the code, the exported interface of the module "Id" is `ID`.
Modules only contain two things: `type declarations`, and "values". Values are your primitives such as 1, '<', "hello", but also composite such as (fun x -> x + 1), (Some x), f x, { foo = "bar"; baz = 42 }, and even (module Id) (yes! modules can be values too!). Type declarations tell the compiler . Anything which is a value _always_ has a type that can _usually_ be inferred.
No type annotation is necessary when the compiler correctly deduces the type of your value through static analysis. For instance, in the module below, "of_string" is deduced to be of type ('a -> 'a). The ' on the symbol 'a signifies a "type variable", and it means that it can be filled in with any type. For instance (t -> t) and (string -> string), but not (t -> string) or (string -> t). For those it would have to be of type ('a -> 'b). We cannot deduce this type, however, because our implementations do nothing with their inputs besides return them. Since nothing is changed, it's always the same type.
Now, can you spot the pink elephant? Notice how the "ID" interface from above defines "of_string" to be of type (string -> t). How can this be possible? It's because we gave the compiler a hint when we said `type t = string`. This says that a "value" of type `t` is backed by a value of type `string`. If something type checks as `t`, it also type checks as `string`.
So, we could reason through and say ('a -> 'a) can be instantiated to (t -> t), but `t` is also equal to `string`, so we can mentally imagine a hypothetical intermediate type... something like ({t,string} -> {t,string}). This type and type equality is visible _inside_ the module. But when the `ID` interface was applied over the `Id` module as in (Id : ID), this has the effect of hiding the type equality (the fact that `type t = string`) because in the `ID` interface we define `t` without an equals sign: `type t`. This forces us to _choose_ a concrete type to expose externally, even though the type is less general than what the implementation sees.
NOTE: OCaml doesn't use parens for function definition or application. Compare this OCaml code against its Python equivalent.
> let hello_world h w = (h, w)
> let h, w = hello_world 1 2
vs.
> def hello_world(h, w):
> return (h, w)
> h, w = hello_world(1, 2)
*)
module Id : ID = struct
type t = string
let of_string s = s
let to_string s = s
end
let main () =
let s = "abc123" in
let id = Id.of_string s in
(* NOTE(type error): because the built-in "print_endline" function is of type (string -> unit) and not (Id.t -> unit) *)
(* NOTE: if an expression returns unit, you don't need to create a let binding for it. You can simply tack a semicolon to the end of it if you need sequence another expression to follow it. *)
print_endline id;
(* okay *)
(* STDOUT: abc123 *)
print_endline (Id.to_string id)
;;
main ()
Now, if you have a bunch of "values" of type `t`, you know that they _only_ could have been produced by the `of_string` function, because `of_string` might be the _only_ function that ends with `-> t`. Therefore, all the values maintain the same properties enforced by the `of_string` function (similar to class constructors in OOP). With this, you can create types such as `Nonnegative.t`, `Percent.t`, `Currency.t`, `Image.t`, `ProfilePicture.t`, and parsers from another type to the newly minted type.
The compiler can help you enforce these properties by providing guardrails in the form of static compiler checks (these checks are run _before_ your code can even be compiled). If I have a value of type `Nonnegative.t`, then not only do I not need to validate that it's not negative, I also don't have to validate that it's not negative everywhere else that values of that type are used -- the validation logic is baked into the constructor. Parse, don't validate.*
[0]: https://github.com/semilin/genkey/commit/fafed6744555c5a81fd...
EDIT: The fact that this was a bug at all makes me fear for the rest of the code base. If this one slipped through the cracks, how can I know that the rest of the code base is correct?
Or perhaps I'm misunderstanding your comment. When you do `as OrgId` or `as UserId`, where do you envision those casts in ways that would require handling failures?
That’s all, just wanted to give them a shoutout.
———
I’m also an avid language learner myself, and I feel like what has worked for me has been to study a wide range of words with Anki, but then also try to use them in conversation. Often times I wouldn’t remember the words I studied, but it would be “on the tip of my tongue”. Then the speaker would fill in the word, and from then on I would never forget the word.
Finally, I think you guys are also headed in the right direction with the idea of “expression”, because I feel it is often difficult to map words 1 to 1 with a language you already know. Oftentimes the expression will be said differently “my name is” vs “to me the name is”. The idea is to capture a unit of meaning in a set expression and to be able to combine the expressions to communicate.
This site is one of the pillars grounding me to the reality that once was the internet.