Hacker News Reader

Foreign manufacturers shipping devices to this country should make frequent firmware updates available. If they use open source / GPL'ed software then should release that too.

There should be a period of X (maybe 5) years after which a device is considered obsolete. At that time all sources inclusive of the opaque binary blobs should be released.

There should be a site / infra by FCC where manufacturers could dump those files.

The FCC itself should get a copy of the source of opaque binary blobs as soon as the device is introduced in the local market.

If that is made a mandate, there are couple of benefits: No device that suspiciously sends data to foreign servers. Less e-trash as community can build / maintain firmware as there is access to source code.

AMICABoardon Llama2.c L2E LLM – Multi OS Binary and Unikernel Release

I have updated the description and added the requested clarification.

To avoid trolls and invisible downvotes, you can always use github issues. You are welcome there.

Thanks for the initial question. Ultimately it seems that there must always be a human in the loop somewhere.

AMICABoardon Llama2.c L2E LLM – Multi OS Binary and Unikernel Release

Pressing the up arrow beside the comments is upvote right? I hope this doesn't have some inverted logic.

AMICABoardon Llama2.c L2E LLM – Multi OS Binary and Unikernel Release

No idea. I have just upvoted your comment.

AMICABoardon Llama2.c L2E LLM – Multi OS Binary and Unikernel Release

That's something we would have to figure out on the way - especially due to hallucinations. See the post under this for ideas on how I would try to solve that.

AMICABoardon Llama2.c L2E LLM – Multi OS Binary and Unikernel Release

I grew up with the Amiga 500, it was my first love, how can I forget those times!

AMICABoardon Llama2.c L2E LLM – Multi OS Binary and Unikernel Release

Agreed and yes would be awesome... But currently this does not infer large meta 7b model efficiently - like 1 token or lower per seconds. But the small toy story (not so useful) models are fast.

If the above mentioned API / python binding is ready, I'll make a streamlit interface demo. The streamlit demo should be simple. But I have to figure out python binding.

AMICABoardon Llama2.c L2E LLM – Multi OS Binary and Unikernel Release

If you add an issue you can do so on https://github.com/trholding/llama2.c

I have a planned a web api. Are you looking for python binding? I'm interested to know. The issues would keep me organized.

AMICABoardon Llama2.c L2E LLM – Multi OS Binary and Unikernel Release

Will be updated with the next commit.

AMICABoardon Llama2.c L2E LLM – Multi OS Binary and Unikernel Release

This is a fork of https://github.com/karpathy/llama2.c

karpathy's llama2.c is like llama.cpp but it is written in C and the python training code is available in that same repo. llama2.c's goal is to be a elegant single file C implementation of the inference and an elegant python implementation for training.

His goal is for people to understand how llama 2 and LLM's work, so he keeps it simple and sweet. As the project progresses, so will features and performance improvements added.

Currently it can infer baby (small) Story models trained by Karpathy at a fast pace. It can also infer Meta LLAMA 2 7b models, but at a very slow rate such as 1 token per second.

So currently this can be used for learning or as a tech preview.

Our friendly fork tries to make it portable, performant and more usable (bells and whistles) over time. Since we mirror upstream closely, the inference capabilities of our fork is similar but slightly faster if compiled with acceleration. What we try to do different is that we try to make this bootable (not there yet) and portable. Right now you can get binary portablity - use the same run.com on any x86_64 machine running on any OS, it will work (possible due to cosmopolitan toolchain). The other part that works is unikernels - boot this as unikernel in VM's (possible due unikraft unikernel & toolchain).

See our fork currently as a release early and release often toy tech demo. We plan to build it out into a useful product.

AMICABoardon Llama2.c L2E LLM – Multi OS Binary and Unikernel Release

It is not inspired by, it's friendly fork, so its more than inspiration. Code has diverged a bit, yes I do try to share. Karpathy's llama2.c upstream project has clearly stated goals of elegance and simplicity, so code that has lot of pre-processor directives like ours won't help upstream. Apart from this we are very thankful to jart & contributors from the cosmopolitan project and also the unikraft folks without which binary portability or unikernels wouldn't have been possible.

TL;DR Simple elegant stuff that we added will be shared to upstream. Complex non elegant code won't be shared as it won't be accepted upstream.

AMICABoardon Llama2.c L2E LLM – Multi OS Binary and Unikernel Release

not yet

AMICABoardon Llama2.c L2E LLM – Multi OS Binary and Unikernel Release

What you said is true, its something we would have to figure out on the way.

What I have in mind is -

1. Topic specialized models which are frequently updated maybe every month or two.

2. Fact Checking & Moderation specialized, models which moderate or do fact checking on other model's output.

Kind of a chicken and egg problem. But I believe on the way we will be able to minimize the effects of hallucinations through output validation (both neural and rule based).

AMICABoardon Llama2.c L2E LLM – Multi OS Binary and Unikernel Release

If you add it as an issue, I'll address it in future. I like the idea, but with the current toy size small models it won't be too fun (I did a manual try). Once larger models run at good pace, it would be absolutely cool.

AMICABoardon Llama2.c L2E LLM – Multi OS Binary and Unikernel Release

It's just the beginning, still optimization and some figuring out to do.

This fork is based on karpathy's llama.c and we try to mirror its progress and add our patches to it to add performance, be binary portable or run as unikernel. However there is a catch, this doesn't currently infer the 7b or bigger meta llama 2 models yet. It's too slow and memory consuming.

My plan is to get to a stage where we can actually infer larger models at a comfortable speed like llama.cpp / ggml does, add GPU acceleration along the way.

Also this doesn't have a web api, I'll be adding that in the next update, then it would actually make sense to deploy it on a server to test it out.

Right now you would have to manually spawn vm instances with qemu like this:

qemu-system-x86_64 -m 256m -accel kvm -kernel L2E_qemu-x86_64

qemu-system-x86_64 -m 256m -accel kvm -kernel L2E_qemu-x86_64 -nographic

and that's not very practical especially as there is no web api yet. So see this more as a tech preview - release early release often thing.

Yeah as I'll get time, I'll be adding a build for firecracker and also write instructions to spawn 100's of baby llama 2 kvm qemu / firecracker builds on a powerful server.

Thank you for your interest. As per your suggestion, a comprehensive howto is planned. Feel free to add any issue / wants / suggestions to https://github.com/trholding/llama2.c , I'll address those as I get time.

I'm stuck with bigger IRL projects, but if there is deep interest from the community I'll be sure to spend more time on this.

AMICABoardon Llama2.c L2E LLM – Multi OS Binary and Unikernel Release

Have you ever wanted to boot and inference a herd of 1000's of Virtual baby Llama 2 models on big ass enterprise servers? No? Well, now you can! (Almost baremetal)

Also drop the binary portable run.com cosmocc build on any OS and run! Truly portable. (Soon baremetal)

Special Thanks & Credits:

llama2.c - @karpathy

cosmopolitan - @jart

unikraft - @unikraft

Would love to hear your feed back here!

AMICABoardon Riffusion – Stable Diffusion fine-tuned to generate music

Awesome, there is another project out there that does it with CPU https://github.com/marcoppasini/musika maybe mix the both, ie take initial output of musika, convert to spectrogram and feed it to riffusion to get more variation...

AMICABoardon Show HN: Port of OpenAI's Whisper model in C/C++

OpenCL would be appreciated much... Opens the door to use this on many more low powered devices but, it could be very difficult as you have already mentioned.

AMICABoardon Show HN: Port of OpenAI's Whisper model in C/C++

I vouch for this. Pretty solid and keeps improving. The OP is in the class of Magic Wizards of programming like Fabrice Bellard!

There are frequent updates and performance improvements. There is also a small community of active users around this.

All most all feedbacks get implemented and the OP is very responsive.

The OP made it possible to do state of the art voice recognition without the PyTorch baggage and in C/C++, pretty incredible! Its one of those rare high value projects.

Very grateful for this project and respect to the OP!

Some day if a ChatGPT open version becomes available, this could mean voice assistants that speak sense and understand the human - as long you have a beefy machine.

The current efficiency is pretty surprising, even on a low spec device it performs faster than real time.

I don't know what to say. But I'm blown away.

I expect to see more magic from the OP in future.

He has even a project for a cool sound modem that works over ultrasonic! Not new stuff, but the implementation is the most robust I have seen.

I recommend hackers here to check out his other project too and maybe contribute with testing and patches and stuff!

AMICABoardon Show HN: I'm trying to guess your personality by your comments with an NLP model

Do you have a git?

AMICABoardon Show HN: I'm trying to guess your personality by your comments with an NLP model

Primary Theme Purgatorial

Your personality theme is purgatorial. It feels like we're in the in-between times. You wonder: What's next? What's the meaning of it all? You may feel like we've lost our way. You're tired. You have no energy. You're may be struggling to keep up with the demands of your work, your family, or maybe kids. You have no time to relax or to explore yourself. You may be burned out. You have a nagging feeling that something's wrong. You feel like you're not doing what you're meant to be doing. Secondary Theme Energy

You are a pioneer. You like to work on projects. You want to do things that you know how to do, so you like to learn new things. You don't like to waste time and money. You like to have fun. You like to explore new things. You like to learn. You like to do things you can be proud of. You have lots of patience and can be patient with others. You have a lot of energy. Secondary Theme The Great Adventure

You have probably debated smashing your phone or at least deleting your social media profile. It’s not easy to shut out the world when you’re on a career trajectory, though. Maybe you'd rather read books, learn a new language, take a yoga class, or take up knitting. Maybe you need a little push in the right direction. A little help to get going, a little encouragement to grow. Needs/Desires Your Needs, Desires, and Values (beta)

Acceptance Health Good Mental Health This feature isn't really tested yet. Let me know how it works in the subreddit. Recommendation You might own

You might already own or plan to buy something like the sodastream because you'd get to create a bunch of random drinks before you went back to just buying name brand soda Possible Theme Stargazing

Your personality theme is stargazing. Your head is far above the clouds. It's in the stars. You see the world in the broadest of terms. You are not a person of narrow interests. You enjoy wide and varied conversation. You have a knack for bringing out the best in others, working with them and motivating them to achieve their highest potential. You can usually be found doing something unusual and creative. Possible Theme Expression

You put a lot of thought into what you say and how you say it. You use your words to make an impact on people. You give life to the words you speak. You are passionate about your personality and what it is you do. You care about what you put out into the world. You are intuitive, and you use it to find your way around this world or the one you create. Possible Theme Mover and a Shaker

You say what you mean and you don't mince words. You have a quick wit and are often funny. You can be sarcastic and blunt, maybe with a quick temper. You are quick to make decisions and you are often decisive. You are a risk taker and are constantly looking for new challenges. You prefer to do things your way, and you are usually successful. You aren't easily controlled or manipulated. Possible Theme Craving

There's something you want right now, but for whatever reason you can't or shouldn't have it. It may be a response to something in your environment that's pulling on your subconscious, or just an old habit you're trying to kick. Whatever it is, there's a reason you can't have what you want. Possible Theme The Reluctant Leader

You are reserved, intellectual, and logical. You may appear shy and awkward at first, but they're actually very witty and have a great sense of humor. You are practical, practical, practical. You are detail-oriented and driven. You're a bit of a perfectionist, but you know when to give someone a break. You're very serious about your work. You can shine in the background, but you know when to stand up and be counted. You can be a little stubborn, especially if you believe you know what's best for everyone.

AMICABoardon Show HN: I'm trying to guess your personality by your comments with an NLP model

Here is my comment. For science! Good Luck! Awesome. Maybe. Why? Why Not? What did you learn?

All your base are belong to us!

AMICABoardon Redbean web server debugging with ZeroBrane Studio

Redbean and Cosmopolitan is like magical alien technology that unites all computing on earth! Now we have a visual debugging method, good work kind ladies & gentleman!

The team is like nuts aliens and I admire their brains.