2,440 karma · joined December 30, 2018
The AI Edge Gallery app on Android (which is the officially recommended way to try out Gemma on phones) uses the GPU (lacks NPU support) even on first party Pixel phones. So it's less of "they didn't want to interface with Apple's proprietary tensor blocks" and more of that they just didn't give a f in general. A truly baffling decision.
It's a 100% replacement for free ChatGPT/Gemini.
Compared to the paid pro/thinking models... Gemma does have reasoning, and I have used the reasoning mode for some tax & legal/accounting advice recently as well as other misc problems. It's worked well for that, but I haven't tried any real difficult tasks. From what I've heard re. agentic coding, the open weight models are ~18-24 months behind Anthropic & Google's SOTA.
Qwen 3.5 122B-A10B should just fit into 128 GB with a Q4/5 and may be a bit smarter. There's apparently also a similar sized Gemma 4 model but they haven't released it yet, the 26B was the largest released.
The only downside is that I suspect the Framework would be a decent bit quieter under load (not that this thing is abnormally loud). As well as you're limited to a single M.2 2230 internal SSD slot in this (I believe Micron recently launched a 4 TB model, but generally you'll max out at 2 TB without using an external enclosure).
I don't have anything against the Framework, I'm sure it's a great machine, but the Z13 is an incredible portable all-in-one device that can handle everything from general PC use to gaming to tablet/entertainment to LLMs & high perf.
On a societal scale, no. Occasionally this works in some individual cases. Like the online outrage over SOPA/PIPA 15 years ago.
But when entity X can gain $$$$$$ (or power) from doing an action, and that action costs everyone only $ (or a minor bit of inconvenience or ideological righteousness), then the average person has very little incentive to take time out of their day-to-day life to fight it.
Meanwhile the entity will do whatever it takes to get the $$$$$$/power because they have a huge incentive. This is the same mechanism that allows democracies to be eroded, as we're seeing right now in the US.
There's also the Nvidia DGX Spark.
https://web.archive.org/web/20080626154537/http://blog.seatt...
98% of people literally do not care and/or are too dumb to understand. You could force them at gunpoint to sit in the education class, and give them a simple basic quiz afterwards, and they'd get half the answers wrong. They will continue to not even read what's on their screen, and just click the big highlighted button every time they see one.
Is Kagi still better than Google? Probably, I don't really know because I don't use Google anymore. But at this point I feel like I'm with them out of inertia more than being an avid supporter. One of these days I'll re-evaluate Google and decide whether to switch back or not.
It does occasionally surface interesting results from small sites that you wouldn't get on Google. I do find that to be useful.
Kagi definitely isn't a bad search engine by any means. Honestly if you haven't used it, try the 100 search free trial on one device. Maybe you'll like it. This feels more like a general decline of the open web.
I configured/disabled RGB lighting in Windows before wiping and the settings carried over to Linux. On Arch, install & enable power-profiles-daemon and you can switch between quiet/balanced/performance fan & TDP profiles. It uses the same profiles & fan curves as the options in Asus's Windows software. KDE has native integration for this in the GUI in the battery menu. You don't need to install asus-linux or rog-control-center.
For local AI: set VRAM size to 512 MB in the BIOS, add these kernel params:
ttm.pages_limit=31457280 ttm.page_pool_size=31457280 amd_iommu=off
Pages are 4 KiB each, so 120 GiB = 120 x 1024^3 / 4096 = 31457280
To check that it worked: sudo dmesg | grep "amdgpu.*memory" will report two values. VRAM is what's set in BIOS (minimum static allocation). GTT is the maximum dynamic quota. The default is 48 GB of GTT. So if you're running small models you actually don't even need to do anything, it'll just work out of the box.
LM Studio worked out of the box with no setup, just download the appimage and run it. For Ollama you just `pacman -S ollama-rocm` and `systemctl enable --now ollama`, then it works. I recently got ComfyUI set up to run image gen & 3d gen models and that was also very easy, took <10 minutes.
I can't believe this machine is still going for $2,800 with 128 GB. It's an incredible value.
Setting the kernel params is a one-time initial setup thing. You have 128 GB of RAM, set it to 120 or whatever as the max VRAM. The LLM will use as much as it needs and the rest of the system will use as much it needs. Fully dynamic with real-time allocation of resources. Honestly I literally haven't even thought of it after setting those kernel args a while ago.
So: "options ttm.pages_limit=31457280 ttm.page_pool_size=31457280", reboot, and that's literally all you have to do.
Oh and even that is only needed because the AMD driver defaults it to something like 35-48 GB max VRAM allocation. It is fully dynamic out of the box, you're only configuring the max VRAM quota with those params. I'm not sure why they choice that number for the default.
Well akshually.... the bootloader is initially not unlockable. You must connect the phone to the internet. Within a few minutes a background process will reach out to Google servers to check whether it was purchased outright or with a payment plan. It will only enable the bootloader unlocking toggle after this step. Phones bought with a carrier contract won't be unlockable until paid off.
In those initial few minutes (/ before you connect it to the interwebs), the bootloader unlock option in the developer settings & fastboot will be disabled.
I'm not sure what exactly you're referring to with "Only Apple has the unique dynamic allocation though." On Strix Halo you set the fixed VRAM size to 512 MB in the BIOS, and you set a few Linux kernel params that enable dynamic allocation to whatever limit you want (I'm using 110 GB max at the moment). LLMs can use up to that much when loaded, but it's shared fully dynamically with regular RAM and is instantly available for regular system use when you unload the LLM.
I have a different heart rate monitor (Amazfit smartwatch, mine has their latest sensor that matches the higher end Garmin watches for accuracy, it can be used as a Bluetooth device or you can develop software to run on it directly). What topics/keywords should I look into if I want to develop the equivalent application for my hardware?
AdAway (in F-Droid) can block with /etc/hosts (no VPN involved) if you have root. The hosts blocking still works even when connected to a VPN. Aside from loading ad domain lists into /etc/hosts, it also allows you to specify custom domains to block - I personally have Reddit and HN in there :)