Getting Started with Orca-2-13B

secondstate.io

68 points·3Sophons··17 comments

I see that they recently changed the intro of WasmEdge on Github [1] to " WasmEdge is the easiest and fastest way to run LLMs on your own devices. "

Since it's a wasm runtime capable of many things I find bizarre that they now start describing it with a ultra-specific use case

- [1] https://github.com/WasmEdge/WasmEdge

I'm assuming they're both VC companies in the same portfolio and are cross-marketing. If not, the the descriptions on both sides seem quite excessive indeed...

for those who want a less command line focused approach, I'd really recommend https://lmstudio.ai/

Amazing enough, there are a couple of orca models in LM Studio/huggingface.

Not sure what the differences are between them.

Are these like AMIs, where you can stuff all kinds of odd shit into them?

I am not at my computer and won’t be for several days, so I can’t check this myself: is Orca-2-13B available in their model library? How about Ollama?

I for one cannot stand projects that depend on Python due to the multitenant and dependency management problems, so this is welcome

Yeah. If I understand this correctly[0], this would work with any model in GGUF format.

[0] https://www.secondstate.io/articles/convert-pytorch-to-gguf/

Which means any model derived from llama2

Indeed. There are the others like LlamaCode, Yi and so many more

Reply on news.ycombinator.com