144 karma · joined April 25, 2018
The recent Kimi-K2 supposedly works great.
This is my workflow for 90% of tasks:
1) /ask: I want to do X. / How would you do X? / What options do I have for X?
2) (optional) /ask>: make these adaptations / consider this / follow up question.
3) "go ahead" (this defaults to /code mode)
For a model I like the style of Sonnet3.7 best.
One caveat: I add my files manually. Knowing what you want and where you want it is 90% of the game and this is where LLMs usually get it wrong.
It is less autonomous but I consider it a feature, it's more of a precision tool.
EDIT: follow up question, how long does the structure-building take for 100 pages and how big are the chunks you are feeding in?
This is the typical "Ah yes you are right, I made a mistake. Let's correct the thing...."-type hallucination or whatever you want to call it. Calling it power-seeking or deceptive behaviour I find overblown.
We will see, maybe models do get good enough but I think we are underestimating these last few percent of improvement.
My question: "Which waiter got second most total money from tips? create a jpeg with a bar chart of the total tips for each waiter."
Step 0 included a correct sql query. Step 1 included python code with matplotlib, which failed because matplotlib was not in the allowed imports. The system then pivoted to another python script which printed a bar plot with ## characters.
Final answer: The waiter with the second most total tips is Corey Johnson
Bar Chart of Total Tips for Each Waiter:
Michael Watts ################################################## (5.67)
Corey Johnson ########## (1.2)
Margaret James ######## (1.0)
[0] - https://github.com/huggingface/smolagents/blob/main/docs/sou...