npsomaratna

Whoops, sorry. 34b :)

Not necessarily true anymore. As a counterexample, consider CodeLlama 33b, which is quite good (and which has replaced GPT-4 for my coding assistant needs).

OpenAI's models are likely to remain the best, but I see open models catching up and becoming "good enough." Why pay for GPT-4 when I can run a model locally for free? (Barring the initial capital cost of a GPU; and not even this if you're, say, using a Macbook)

Made me think of touch typing. I can touch type at a decent speed (~100 wpm). However, if I let myself think about where the various letters are on the keyboard, I stumble and start having trouble typing. I need to clear my mind to start touch typing again.

Understood, thank you. That makes sense.

Interesting, thank you. I wonder if that that area had stagnant water, allowing mosquito growth.

This is fantastic. One thought that came to my mind is: would mosquito borne diseases have been a problem in city surrounded by so much water?

I reevaluated this in my 20s and 30s - and I still learned the true meaning only just now.

Came here to say this. My understanding is that locked Macbooks are unusable, only have salvage value, and are therefore sold at pennies on the dollar. (Many such laptops are quite likely stolen.)

When the OP mentioned a different login, etc, I started to wonder if he'd been sent a different laptop. This fits in with the other guy asking OP to cover the courier costs, etc.

Understood, thank you. I appreciate the explanation.

Can you explain more about the voltage/clock band please? Do you mean that they are very energy inefficient?

Yes, this was interesting to learn.

On a tangent: it's awe-inspiring just how much older the Egyptian pyramids are, vis-a-vis the other pyramidal structures listed there.

Came here to say that. I believe that AI assistance will make a big difference to many professions going forward—certainly any profession that has to deal with large volumes of information, whether it's education, knowledge, or red tape. But I wouldn't conflate that with copilot. I hope that open-source AIs eventually occupy that role.

I was in an apartment building in Santiago, Chile—in fact, pretty much all of the earthquakes I experienced were in Santiago. Never heard any (out of the normal) sounds. Is that because the buildings were built to handle earthquakes?

I remember the first earthquake I experienced. Same feeling: everything around me started shaking; for several seconds, I couldn't figure out what was happening; and then it suddenly hit me: it's an earthquake.

(I'm from a country where earthquakes do not occur. For some reason, I always thought that earthquakes were accompanied by, well, movie-style earthquake sounds. It was kinda eye-opening to realize that earthquakes are silent...)

NTSB investigates/provides advice on civil aviation and commercial space accidents as well—not just land stuff.

Possibly a lot. See: https://espadrine.github.io/blog/posts/chinchilla-s-death.ht...

I’m wondering about economics though. The cost of solar panels vis-a-vis the increased range of only 5000km per year.

Even if you amortize this cost over several years, and even in climes with more sun, would the economic math work out?

Sure. So, my startup[1] does data processing for clients in aviation and finance. Due to security, legal, and regulatory requirements, sending our code on to a third party (e.g., OpenAI) is a no-go. Thanks to CodeLlama (the WizardCoder fine-tune), I've finally been able to get AI assistance with my code.

My fundamental programming skills are solid (30+ years programming!), but I only started using Python extensively relatively recently. I can get the language to do what I want it to do—but often not in the most "Pythonic" way. With CodeLlama, I'm able to check if I've done things in the "right" way, and I'm also able to ask questions when I'm trying to figure out how to implement something.

As mentioned above, I cannot use GPT-4 for my startup, but I have used it for personal stuff. My feel so far is that wizardcoder-python-34b is surprisingly good. I certainly haven't thought "oh, I wish I had GPT-4 instead"—at least, not yet! (I haven't used Copilot either, for the same reasons listed above.)

--------

[1] Shameless plug: https://www.amw.ai

WizardCoder is fantastic. I use it as my local "code assistant" LLM[1].

I'd really like to hear from the Phind team about this. Using someone else's work without attribution isn't cool (assuming this thread is true).

------

[1] In fact, I put out a set of instructions on how to get it to work with VS Code (with the help of Continue + llama.cpp) a couple of days ago: https://news.ycombinator.com/item?id=37296728

Thank you. I had to reduce the context length to get this to work without crashing (from 16k to 8k)—and I'm seeing the ~100% speed up you mentioned.

However, when I run the LLM, OSX becomes sluggish. I assume this is because the GPU's utilized to the point where hardware-based rendering slows down due to insufficient resources.

I wonder if there's a way to avoid that slowdown?

Do a search on the HuggingFace models page, e.g.:

https://huggingface.co/models?sort=trending&search=wizardcod...

Yes. In the llama.cpp server command, specify the number of layers you'd like offloaded to your GPU via the -ngl parameter, e.g.:

  ./server -t 8 -m models/wizardcoder-python-34b-v1.0.Q4_K_S.gguf -c 16384 --mlock -ngl 60

(You might need to play around with the number of layers.)

[Edit: make sure to compile llama.cpp with GPU support first, e.g., "make clean && LLAMA_CUBLAS=1 make -j"]

M1 with 32 GB RAM. I can just about fit the 4-bit quantized 33 GB Code Llama model (and it's finetunes, e.g. WizardCoder, etc.) into memory. It's somewhat slow, but good enough for my purposes.

Edit: when I bought my Macbook in 2021, I was like "Ok, I'll just take the base model and add another 16 GB of RAM. That should future proof it for at least another half-decade." Famous last words.

You don't need so many layers of stuff (or API keys, signups, or other nonsense).

Llama.cpp (to serve the model) + the Continue VS Code extension are enough.

The rough list of steps to do so are:

  Part A: Install llama.cpp and get it to serve the model:
  --------------------------------------------------------
  1. Install the llama.cpp repo and run make.
  2. Download the relevant model (e.g. wizardcoder-python-34b-v1.0.Q4_K_S.gguf).
  3. Run the llama.cpp server (e.g., ./server -t 8 -m models/wizardcoder-python-34b-v1.0.Q4_K_S.gguf -c 16384 --mlock).
  4. Run the OpenAI like API server [also included in llama.cpp] (e.g., python ./examples/server/api_like_OAI.py).

  Part B: Install Continue and connect it to llama.cpp's OpenAI like API:
  -----------------------------------------------------------------------
  5. Install the Continue extension in VS Code.
  6. In the Continue extension's sidebar, click through the tutorial and then type /config to access the configuration.
  7. In the Continue configuration, add "from continuedev.src.continuedev.libs.llm.ggml import GGML" at the top of the file.
  8. In the Continue configuration, replace lines 57 to 62 (or around) with:

    models=Models(
        default=GGML(
            max_context_length=16384,
            server_url="http://localhost:8081"
        )
    ),

  9. Restart VS Code, and enjoy!

You can access your local coding LLM through the Continue sidebar now.

Exactly. LLMs are useful out of the box—and that enables tons of use-cases that just weren't economical (or even possible) with older models.

Disagree. As a lawyer I use LLMs with RAG to help me surface information all the time. Often, this allows me to find niche case law that I just wouldn't have had the time to find on my own. However, I double-check everything, and read all the original sources.

LLMs are best treated as the AI equivalent of a human assistant who is knowledgeable and fast, but also inexperienced, and thus, prone to making mistakes. You won't throw out the work of such an assistant-it'll still save you hours of effort. However, you won't take the work at face value either.

Isn't the Chinchilla estimate considered to be wrong now?

https://espadrine.github.io/blog/posts/chinchilla-s-death.ht...

Non-US doctor here. Completely agree. Licensing exams mainly confirm that the aspiring doctor won't harm/kill patients by mistake. Specialist-level exams assess standard of care.

Not a U.S. lawyer, but check the concept of "puffery."

Realms of the Haunting is unique. I’d love a modern remake.

Folks who haven’t played this-the playthrough by Kikoskia is quite interesting: https://www.youtube.com/watch?v=e_mkpxHcdew