bytebach

Sadly the amazon kindle version is not readable by their kindle web reader - tells me to download android or ios version of kindle :(

Related HN discussion - https://news.ycombinator.com/item?id=41831735 using Prolog as an intermediate target language for LLM output improves their 'reasoning' abilities.

An application I am developing for a customer needed to read constraints around clinical trials and essentially build a query from them. Constraints involve prior treatments, biomarkers, type of disease (cancers) etc.

Using just an LLM did not produce reliable queries, despite trying many many prompts, so being an old Prolog hacker I wondered if using it might impose more 'logic' on the LLM. So we precede the textual description of the constraints with the following prompt:

-------------

Now consider the following Prolog predicates:

biomarker(Name, Status) where Status will be one of the following integers -

Wildtype = 0 Mutated = 1 Methylated = 2 Unmethylated = 3 Amplified = 4 Deleted = 5 Positive = 6 Negative = 7

tumor(Name, Status) where Status will be one of the following integers if know else left unbound -

Newly diagnosed = 1 Recurrence = 2 Metastasized = 3 Progression = 4

chemo(Name)

surgery(Name) Where Name may be an unbound variable

other_treatment(Name)

radiation(Name) Where Name may be an unbound variable

Assume you are given predicate atMost(T, N) where T is a compound term and N is an integer. It will return true if the number of 'occurences' of T is less than or equal N else it will fail.

Assume you are given a predicate atLeastOneOf(L) where L is a list of compound terms. It will succeed if at least one of the compound terms, when executed as a predicate returns true.

Assume you are given a predicate age(Min, Max) which will return true if the patient's age is in between Min and Max.

Assume you have a predicate not(T) which returns true if predicate T evaluates false and vice versa. i.e. rather than '\\+ A' use not(A).

Do not implement the above helper functions.

VERY IMPORTANT: Use 'atLeastOneOf()' whenever you would otherwise use ';' to represent 'OR'. i.e. rather than 'A ; B' use atLeastOneOf([A, B]).

EXAMPLE INPUT: Patient must have recurrent GBM, methylated MGMT and wildtype EGFR. Patient must not have mutated KRAS.

EXAMPLE OUTPUT: tumor('gbm', 2), biomarker('MGMT', 2), biomarker('EGFR', 0), not(biomarker('KRAS', 1))

------------------

The Prolog predicates, when evaluated generate the required underlying query (of course the Prolog is itself a form of query).

Anyway - the upshot was a vast improvement in the accuracy of the generated query (I've yet to see a bad one). Somewhere in its bowels, being told to generate Prolog 'focused' the LLM. Perhaps LLMs are happier with declarative languages rather than imperative ones (I know I am :) ).

A lightweight but very scalable, robust (like Erlang's 'let it crash') and distributeable complete Actor system for Java 21 and above. Collection of implemented Actor idioms supporting the 'Dust way'. Also see dust-ville ... a toy 'digitally twinned' town built as a tiny Dust example - complete with a murmuration of Starlings circling the power station :)

Sure, the app selects matching patients based on demographics, disease, prior treatments and biomarkers. It also has to be able to express numeric constraints as well ('no more than two surgeries ..', 'at least one of the following biomarkers'). The following prompt sets up the Prolog toolkit the LLM is allowed to use. The generated Prolog conjunction is then run through a Prolog meta-interpreter that generates the matching code. Though it sounds long-winded it is more than fast enough to generate responses to user queries in an acceptable time:

-----------------

  Now consider the following Prolog predicates:
  
    biomarker(Name, Status) where Status will be one of the following integers -
   
     Wildtype = 0
     Mutated = 1
     Methylated = 2
     Unmethylated = 3
     Amplified = 4
     Deleted = 5
     Positive = 6
     Negative = 7
   
    tumor(Name, Status) where Status will be one of the following integers if know else left unbound -
    
     Newly diagnosed = 1
     Recurrence = 2
     Metastasized = 3
     Progression = 4
    
    chemo(Name)
   
    surgery(Name)  Where Name may be an unbound variable

    other_treatment(Name)

    radiation(Name) Where Name may be an unbound variable
     
    Assume you are given predicate  atMost(T, N) where T is a compound term and N is an integer. 
    It will return true if the number of 'occurences' of T is less than or equal N else it will fail. 
   
    Assume you are given a predicate atLeastOneOf(L) where L is a list of compound terms. 
    It will succeed if at least one of the compound terms, when executed as a predicate returns true.
   
    Assume you are given a predicate age(Min, Max) which will return true if the patient's age is in between Min and Max.
     
    Assume you have a predicate not(T) which returns true if predicate T evaluates false and vice versa. 
    i.e. rather than '\\+ A' use not(A).
   
   Do not implement the above helper functions.
  
   VERY IMPORTANT: Use 'atLeastOneOf()' whenever you would otherwise use ';' to represent 'OR'.  
   i.e. rather than 'A ; B' use atLeastOneOf([A, B]).
    
  EXAMPLE INPUT: 
   Patient must have recurrent GBM, methylated MGMT and wildtype EGFR. Patient must not have mutated KRAS.  
  
  EXAMPLE OUTPUT:
    tumor('gbm', 2),
    biomarker('MGMT', 2),
    biomarker('EGFR', 0),
    not(biomarker('KRAS', 1))
  
  Express the following constraints as a Prolog conjunction. 
        Do not enclose the code in a code block. Return only the Prolog code - no commentary.
        Be careful to use only the supplied constraints, do not add any: 
        
        $constraint

I recently has a consulting gig (medical informatics) that required English Declarative -> Imperative code. Direct code generation by the LLM turned out to be buggy so I added an intermediate DSL implemented in Prolog! The prompt described the Prolog predicates it had to work with and their semantics and the declarative goal. The resulting (highly accurate and bug free) Prolog code was then executed to generate the conventional (Groovy) imperative code that was then executed dynamically. In some hand-wavy way the logical constraints of using Prolog as an intermediate DSL seemed to keep the LLM on the straight and narrow.

Yes - first shot was correct. 2nd shot said his second movie was the same as his first (?). 3rd shot was just wrong (said he was not in Little Shop). 4th shot was interesting (listed Little shop as his 3rd movie) then it all went downhill from there.

Using the free sign-up - probably 3.5. I guess the point is that even if 4 is better the amount of trust being put into this thing (as amazing as it is) is somewhat worrying. I still get the 'In vino veritas' vide, FWIT.