Hacker News Reader

https://github.com/exo-explore/exo

If you set CPUSchedulingPolicy=idle Nice=19 IOSchedulingClass=idle in the ollama server configuration it should run in the background with lowest priority.

Seems like so much more work than "just" paying for https://huggingface.co or whichever other neocloud who already did all the setup for you and just waits for your credit card per minute/seconds/token.

It is much more work because for many workloads you have geographic ringfencing and cannot send it out to the cloud

Doubt this kind of workloads would agree to send data then to a cloud of randos devices, precisely when cloud providers to certify they aren't looking at clients data (Customer-managed encryption keys, CMEK).

>> Doubt this kind of workloads would agree to send data then to a cloud of randos devices,

Totally agree, which is why i said "I'd love a way to do this locally -- pool all the PCs in our own office for in-office pools of compute."