dsemakin

You got me interested, a rough table of model size to instance type to spot $/hr would help a lot. The 0.5B example is CPU only, so it doesn't say much about what a 7B or 70B actually costs.

What triggers a memory getting written in practice? is it on me to remember engrim add?

Nice concept, the offline provisioning is the part that stands out to me vs Tailscale. How does revocation work in that model? Didn't see it in the docs, if a laptop with a .peer profile gets stolen, what then?

Just at the right time, will be trying this out. I see it as an addition to local LLM, where on a job cycle it can learn from the LLMs contexts and free it up and help it in the more complex work.

Would be really cool to see two things.

First some kind of before and after benchmark, so you can actually see if defragging made a difference. Second is a recommendation for defragging based on things like the fragmentation, free space, and whether it’s an HDD or SSD.

Especially the second one feels useful since defragging doesn’t necessarily mean improvement to the system as you said yourself.

Does it work also on multi modal LLMs?

Interesting concept, I really like the idea of monitoring the domain. I tried 1 domain I am looking at: bein.com, resulted in a thousand domains and not this one. Will give it a better look though.