Hacker News Reader

Comment by behnamoh | Hacker News Reader

I've come to realize you better develop your own little library for LLMs instead of depending on complex libraries with hidden prompts here and there which you can't control.

People think LLMs are about connecting things (chaining) but you'll get that for free if you focus instead on the "content" and "behavior" of your LLMs.

View on HN

The goal of instructor is that i dont write any prompts for you, i just make the openai.ChatCompletion.create call easier to manage and let you return a pass in a pydantic object and return an instance of it,

Whole codebase is 300 lines of code you can copy paste into your own repo into function_calls.py :)

Oh yeah I really like instructor and have been using it! I was more talking about complicated libraries (as opposed to 300 lines of — well structured code like instructor).

This was exactly my experience trying to build a RAG pipeline that answers complex questions by generating sub-questions [1]. The hidden prompts in the complex libraries are quite difficult to get to and then start tweaking, whereas writing my own pipeline boiled down the code to 4 LLM calls. +1 for using the instructor library for quick schema definitions!

[1] https://github.com/pchunduri6/rag-demystified

Yeah, obligatory reminder that "chaining" is just function composition. As in:

  SecondQuery(FirstQuery(userInput));

"Advanced" chaining can be as simple as:

  do {
     partial = FirstQuery(input);
     choice = DeciderQuery(partial);
     input = choice ? Query2A(partial) : Query2B(partial);
  } while(!ValidationQuery(input));
  return input; // or print it or whatever

It only looks complex when you need to chain things at runtime through some interface, but that's the usual problem of having to reinvent basic blocks of your language as objects or data structures because your language doesn't let you treat code and data interchangeably. But if you're just experimenting with chains, using the most basic programming building blocks directly - functions, conditionals, loops - is by far the easiest.

Or put even simpler: calling a function with the result of another function.

I’d question how much value FP abstractions like partials and composition are really adding in your example.

Oh, I didn't mean to invoke FP abstractions - "composition" is used outside of FP, and "partial"... that's just bad naming choice on my end. This example was meant to be dumb, imperative C-like code.

Ah sorry, I saw ‘partial’ and thought you were passing around functions. Please disregard my comment :)

Reply on news.ycombinator.com