Hacker News Reader

If you add an issue you can do so on https://github.com/trholding/llama2.c

I have a planned a web api. Are you looking for python binding? I'm interested to know. The issues would keep me organized.

The ability to hook it into frameworks like streamlit would be huge. Sorta like llama-cpp-python

Agreed and yes would be awesome... But currently this does not infer large meta 7b model efficiently - like 1 token or lower per seconds. But the small toy story (not so useful) models are fast.

If the above mentioned API / python binding is ready, I'll make a streamlit interface demo. The streamlit demo should be simple. But I have to figure out python binding.

not yet