DarthNebo

HiddenBar

This is old news

Oh I see, my bad

They did use the Canny ControlNet Pipeline

I'm hitting 3.9tok/s with CTX of 300 tokens on Android/778G via Userland & this is with an older unoptimized build of llama.cpp

Feels like there should be two branches of system design for unit profitable architecture for paying users vs VC backed architectures to support non-paying users.

For long running stuff https://developer.apple.com/tutorials/app-dev-training/trans... should be straightforward to translate as well using ported on-device BERT models

JM2C

lood_in_4bit=True will let you run Llama2-7B variants at 6.3GB VRAM.

I'm building a tool for transcriptions where your brand, product or any other technical jargon or heck even your own name does not need a fine-tuned model all the time. Both as a free native Mac app & a SaaS tool for those who need to process in bulk. Hit me up at nebo@minusgreed.com to check it out, will launch as FortuneSpeech.com once out of beta.

Thanks to this post, I kind of have another idea for a 'corrections to transcript' feature that Llama2-7B even on CPU can help with.

It's like a repeat of c++ or python. If you can find folks & tooling for what you want to accomplish then go for it. I don't think founders who actually have paying customers concern themselves with what works best per cent of compute spend rather what allows them to improve the product, while as an outsider it may seem that they can reduce costs by choosing X over Y.

I would suggest getting your feet wet with HuggingFace Spaces free/Pro plan to get started & then their APIs once you get the hang of setting up things there. After that you can start with setting up LangChain pipelines or direct vector DB queries for which sort of columns or SQL queries to formulate(for the latter).

As for the former classifier you can try doing zero-shot classification between n number of categories + others. Models like Flan-T5/T5/Flan-UL2/DistillBART(also ~7B-40B param LLMs can also do this but would be overkill).

You can always pull from the recovery/cloud of Apple. It's just an excuse to push you to newer hardware

Another person actually tested out the bot & tried to search for a product called mCaffiene & instead whatever LLM they're using just changed it to McAfee anti-virus to try & answer the query. Pretty untested & thrown into the user's hands.

Bring the calculator widget back cowards!

Another person actually tested out the bot & tried to search for a product called mCaffiene & instead whatever LLM they're using just changed it to McAfee anti-virus to try & answer the query. Pretty untested & thrown into the user's hands

You can alway setup a kafka or message queue consumer on these sort of hardware. I'm testing my RTX card for some offloading of free-tier volume as well

I did my share of OCR funny business by running the Firebase OCR SDK inside an android app acting as a webserver, through nested virtualization back in 2018. Nothing at that time beat it's accuracy & throughput for something which ran offline once the weights were fetched into the container.

Devs who do not have these in their workstations will not be experimenting on rented VMs, unless it is company infra. Dunno why Radeon cards aren't the focus instead of these MI variant

I moved everything to Zoho with custom domain to get rid of google's passive Ads from shopping, banking & whatnot

I used to just leave my work laptop back in the office locker, coz apparently we can't work from home anymore so no 'emergency/important' work after 5PM lol.

Given that there's plenty of options for every point in the README.md, one thing missing is how to guarantee that your stack does not miss requests from paying customers, metering usage & avoid ballooning server costs. I see a lot of YC startups trying to solve this Lago, Paigo etc.

I'm trying to evaluate best serverless solutions for inference without compromising on client usage & reducing idle time on GPU boxes. So far its down to base10, HF, Banana, I'll end up pooling them all & then sending requests between them. For dedicated training boxes Lambda, Modal, Oblivus, Runpod are the contenders.

I yearn for a frosted UI which has chilly condensation which shifts around the edges. Something from TRON but mixed with John Carpenter's 'Thing' vibe which white, blue & other nordic colors.

Yeah no, R2 is a way better choice than being vendor locked like this when burning millions

Absolutely love this on my Android TV, I just spin up a container on my laptop with mounted folders & I am able to play large files without plugging & copying anything to a USB drive. Android TVs for some reason cannot fathom that video files >4GB do exist along with other basic filesystems like ExFAT & NTFS......smh

They should undercut Nvidia at both pricing for datacenter cards & increase CUDA like framework adoption with developer accessible cards & boatloads of VRAM. Dev's wont buy cards unless they demonstrate perf/$ over nvidia for standard models

That's the problem with humans as well, since we don't exactly memorise the spec, we just check & update basis errors & looking up references. But since this is an LLM we are talking about, it should be able to infer the same once given the spec of a language without needing a compiler or atleast being able to check with one via plugin or api call.

I setup Screenshot app to save directly to the Google Drive folder, so it's always available to me from any device. Used Dropbox up until last month before getting rid of the horrendous native client.

My downloads folder is effectively categorized as

AW - artwork, could be pics, videos that I create or download, further into monthly phone backup, jellyfin etc

SW - software, contains anything I've downloaded, specific folders for each caterory like 3D printing, setup files, repo code zips/model weights etc

PDFs - as the name implies PDFs only, mostly arxiv papers grouped by month-year folders & backed off monthly to phone & NAS

-----

Has worked like a charm for me for a while & very easy to take backups of my machine since everything essential is present within the Download sub-folders

what a spam article for HN

From the creator Luke Muscat himself

macOS has built in head tracking & control with expressions or additional hardware switches