Apple is reportedly spending ‘millions of dollars a day’ training AI

Not yet, but the desire is there, especially because the generic GPT-4 API is quite slow in responding to more complex prompts, not to mention the privacy concerns. I think the next version of my home NAS is likely to have a GPU in it and will run things like an appropriately tuned llama2 or similar to be the brain backbone of my smart home. Feels like an obvious direction for commercial NAS to go in as well.

Can't wait for a future where we can buy a Rasberry Pi 7 with a little analog compute ASIC running local inference with ease ... intelligent controllers everywhere!

The boards are raised off the wood surfaces by PCB spacers (I embedded M2.5 threaded sockets into the wood) and I bought speaker grilles that bulge out a little at edge of the cylinder, so that the 4-mic array would remain fully exposed also laterally. I covered the side speakers and the mic array speaker grille with acoustic textile.

The onboard code is written in very pedestrian Python and uses porcupine and the OpenAI APIs.

It roughly works like this:

1. Capture audio frames and run overlapping frames by porcupine to perform hot word detection (overlapping avoids the problem of the hotword falling inbetween frames, at a cost to latency)

2. Once the hot word has been detected, buffer all audio frames into a command buffer until silence is detected as a stop (detecting "silence" is a bit involved, taking noise levels into account, and a few other tricks, more below)

3. The command buffer is sent to Whisper for transcription

4. GPT-4 is prompted with a system message steering it's behavior, the user command transcription and a JSON print out of the state of all devices (e.g. lights and Sonos speakers) in the home, grouped by rooms

5. Following the system message, GPT-4 replies with a JSON structure of changes it would like to make to the device state, omitting unchanged bits from the original

6. Add the sensor event and memory system described above

There's a few other tricks. To improve the audio capture, I take note of spatially where the hot word is detected (i.e. which mic in the array gets the best signal) and then capture the rest & perform the silence detection with a corresponding bias.

This is actually done in a distributed fashion over the network, so if two of the AI speakers hear the same command, only one of them will end up processing it.

They end up making mainly HTTP calls to APIs that already exist around my house. I have a second RasPi in my LED shelf (another old project, https://github.com/eikehein/hyelicht/) that doubles as a Philips Hue bridge with a zigbee dongle. That's what the DIY AI speakers interact with when making changes to the lighting.

I will say: Depending on the user command and the weather in the cloud, it's pretty slow. I've tried my best to optimize the client side for perceived user latency, but there's no way around the GPT-4 API just being pretty slow, even if it's amazingly low-friction and reliable otherwise. And 3.5-turbo just doesn't cut it for what I'm trying to do.

I'd like to get all of this out of the cloud entirely. I predict the next generation of my home NAS will have a GPU in it and try to run things like fine-tuned llama2 for the home.

At least Google kills something when they lose interest.

Apple sometimes does incremental improvements, and sometimes does an entirely new release.

Five years to improve? Sure, probably could've done a lot. Five years to start completely new? Not enough time to make it land with a bang compared to the existing Siri.

It'd be more accurate to say that biographers would not have heard anything (from Steve or any other in-the-know Apple employees) about any projects that hadn't already been unveiled by Apple.

Additionally, it seems likely that biographers would've been required to not ask about possible future projects hinted at by public resources like patents: https://www.patentlyapple.com/2023/08/apple-won-a-patent-tod...