Hacker News Reader

Cool idea, but kinda sad that it has to go through a cloud-provider. I feel like there's a possibility with an accelerator-board (Coral TPU or something), to make this into a totally local thing maybe? The longer-waiting time is surely not an issue when considering how many people still use Polaroids.

View on HN

We were looking to add on-device styles with the Raspberry Pi in order to keep the device cost low, though a Coral TPU would make this easier. The OnyxStream library appears to be able to do SD1.5 generation in 10 minutes on a Pi Zero, so with some optimization and reducing image resolution img2img may be possible on the Pi in ~1 minute. We were also looking at style transfer models, which are much more lightweight and could run fast on a Pi (https://github.com/tyui592/AdaIN_Pytorch/tree/master). Eventually our goal is to make this both on-device and relatively cheap.

We were looking into OnnxStream (https://github.com/vitoplantamura/OnnxStream) and modifying it to support img2img. We got pretty close but yeah capability of running diffusion models on a Raspi are quite limited lol.

Alternatively we could use compute from your iPhone, but it adds additional dependencies to external hardware that I don't quite like. We could use a Jetson, but then power draw is quite high. I agree with you that on-device inference is the holy grail, but figuring out the best approach is something we are still trying to figure out.

Reply on news.ycombinator.com