ollama run phind-codellama "write c code to reverse a linked list"
To run this on an m1 Mac or similar machine, you'll need around 32GB of memory for the 4-bit quantized version since it's a 34B parameter model and is quite big (20GB).Then comes the real question, which is "let's write fizz buzz so it generates at above 55Gbytes/second".
Previously discussed here on HN: https://news.ycombinator.com/item?id=29031488
I tried with ollama, installed from Homebrew, on my M1 Max with 64GB RAM.
I downloaded the phind-codellama model using
ollama pull phind-codellama
But when I give it a prompt, for example ollama run phind-codellama "write a production grade implementation of Sieve of Eratosthenes in Rust"
It prints the following error message Error: Post "http://localhost:11434/api/generate": EOF
and exits.Even though it worked to run ollama with some other models.
Is the version in homebrew not able to run phind-codellama?
The download (both as a Mac app and standalone binary) is available here: https://github.com/jmorganca/ollama/releases/tag/v0.0.16. And I will work on getting that brew formula updated as well! Sorry to see you hit an error!
I'm on a MacBook Pro M1 Max 64GB.
% ollama --version
ollama version 0.0.16
% ollama run phind-codellama "$(cat prompt.txt)"
⠧ Error: Post "http://localhost:11434/api/generate": EOF
% wc prompt.txt
335 1481 11457 prompt.txtDo you mean triple double quotes?
I created an issue, if you have an example prompt to add that would be helpful! https://github.com/jmorganca/ollama/issues/422
But I'd like to also be able to run these models on my Linux desktop with two GPU's (a 2080Ti and a 3080Ti) and a Threadripper. How difficult would it be to set some of these up on there?
I personally use llama.cpp as the driver since I run CPU-only but another may be better suited for GPU usage. But then it's as simple as downloading the model and placing it in the directory.
I'm admittedly running Sonoma beta, that's probably why
I will try it on my Linux machine later tonight (currently AFK with laptop)
> write c code to reverse a linked list
The number of times I had to do it in real production code amounts to zero.
The number of times I had to piece code from poorly documented external services, conflicting product requirements, and running on complex and weird environments has been ... well ... multiple times a day for the past 20+ years.
"instruct-like prompts" is what you give a very junior engineer out of college, and then you have to carefully review their code.
To say anything as general as ‘the new file format is broken’ just means you either don’t understand the project basics or do not follow closely the commits.
Llamacpp underwent some major changes last few weeks. And following the commits it took few days to stabilise. Try now , works as bliss. And compared to other inference engines such as tinygrad - is much more versatile in options how to be run.
Pretend to be a commenter on hackernews. Respond to the comment below:
[parent comment inlined]
what is your response?
"Wow, that's great to hear! It sounds like you had a really positive experience with the 34B last night. I'm also excited to see what's in store for Phind and its potential applications. Have you tried using the 34B for any specific tasks or projects yet? And do you think the text completion feature would be useful for your use case if it were improved further?"It answers every prompt with “well actually…” and if it doesn’t know the answer it hallucinates one.
And since that question has been answered in the negative, I'm inclined to agree.
But you should be able to get it to do stuff just by telling it what you want. Note that it's completion tuned (not chat-tuned), so it should perform better on single-turn conversations.
Here an example output (I can get different results every regenerate since the question is pretty vague):
Prompt:
Give me an example db query saving to file in json.
Result:
Sure! Here's an example of how you can save a database query result to a JSON file in Python using the sqlite3 module and the json module:
import sqlite3
import json
# Connect to the database
conn = sqlite3.connect("example.db")
cursor = conn.cursor()
# Execute a query that returns a single row
cursor.execute("SELECT \* FROM mytable WHERE id=123")
result = cursor.fetchone()
# Convert the query result to a dictionary
data = {key: value for key, value in result}
# Save the data as a JSON object
with open("output.json", "w") as f:
json.dump(data, f)
This code connects to a SQLite database named example.db, executes a query that retrieves a single row from a table called mytable, converts the resulting tuple into a dictionary, and then saves the dictionary as a JSON object to a file named output.json.