russ

Yeah, that would be really handy too.

This is very cool! I’ve wanted something like CodeMic for a long time.

Back when I was at Twitter, we used Review Board for code reviews (this was in 2009, before GH was a thing for most companies). It was tough to thoughtfully review large branches, especially for parts of the codebase that I wasn’t familiar with. I remember thinking, if I could somehow record the development process for a PR I was reviewing, it would be easier to understand what the submitter was trying to accomplish and how they went about doing so. I found myself more so reviewing code style instead of functionality, architecture, or design.

I watched most of the intro video, but didn’t go deeper on the site. Does CM integrate easily into the code review/PR process? I suppose I could just attach a link in any PR description?

Great work!

You got it! Hope you have some fun with it. :)

Haven’t played Codenames in a long while, but made this 8 years ago to play with family and friends on TVs. Right in time for the holidays!

demo: https://dsa.github.io

code: https://github.com/dsa/dsa.github.io

Haha yup, I’m that Russ. Really appreciate your kind words. <3

I’ll shoot you an email and let’s chat!

This is super cool. One neat idea: when I'm in offline mode, I can clone my voice, provide some context data/sources, and have my AI clone answer calls for me. It can give me a summary of conversations it had each day and allow me to follow up.

Heh, actually I'm pretty sure I've come across your X profile before. :) You're definitely in a small minority of folks with a deep(er) understanding of WebRTC.

You really don’t need to know about WebRTC at all when you use LiveKit. That’s largely thanks to the SDKs abstracting away all the complexity. Having good SDKs that work across every platform with consistent APIs is more valuable than the SFU imo. There are other options for SFUs and folks like Signal have rolled their own. Try to get WebRTC running on Apple Vision Pro or tvOS and let me know if that’s no big deal.

Doesn’t sound right. I’d love to dig into this some more. Would you mind shooting me a DM on X? @dsa

Field CTO — hi @Sean-Der :wave:

Fractional CTO sounds like a disaster lol

Which components feel ad hoc?

In most real applications, the agent has additional logic (function calling, RAG, etc) than simply relaying a stream to the model server. In those cases, you want it to be a separate service/component that can be independently scaled.

There’s Ultravox as well (from one of the creators of WebRTC): https://github.com/fixie-ai/ultravox

Their model builds a speech-to-speech layer into Llama. Last I checked they have the audio-in part working and they’re working on the audio-out piece.

There’s a ton of complexity under the “relatively simple use case” when you get to a global, 200M+ user scale.

50% human speaking at $0.06/minute of tokens

50% AI speaking at $0.24/minute of tokens

we (LiveKit Cloud) charge ~$0.0005/minute for each participant (in this case there would be 2)

So blended is $0.151/minute

We had our playground (https://playground.livekit.io) up for a few days using our key. Def racked up a $$$$ bill!

I had no idea! <3 Thank you for sharing this, made my weekend.

It's using the same model/engine. I don't have knowledge of the internals, but a different subsystem/set of dedicated resources though for API traffic versus first-party apps.

One thing to note is there is no separate TTS-phase here, it's happening internally within GPT-4o, in the Realtime API and Advanced Voice.

It’s insanely fast.

Here’s an AI voice assistant I built that uses it:

https://cerebras.vercel.app

Here’s an AI voice assistant we built this weekend that uses it:

https://x.com/dsa/status/1828481132108873979?s=46&t=uB6padbn...

LiveKit Agents, which OpenAI uses in voice mode is also open source:

https://github.com/livekit/agents

But we don't handle interruptions yet, that's some cool stuff @yanyan_evie!

We recently made it a lot easier to build your own KITT too: https://github.com/livekit/agents

100%. My dad was an electrical engineer in the 70s and 80s and ended up transitioning to sales, given that it paid much more.

Zoom does use a custom protocol. This is why it doesn’t work nearly as well when you take a call in the browser client. Not because WebRTC isn’t up to the task, but because Zoom hasn’t invested in it.

Ignoring costs, while having someone host infra for you will always be easier than managing it yourself, I think we’ve really improved the DX of hosting your own WebRTC infra with LiveKit: https://github.com/livekit/livekit

You can do that with LiveKit Egress: https://github.com/livekit/egress

HLS is a client-driven protocol, so it has high-scale but variable latency. You could build a mostly one-way webinar experience using it, but definitely not a conference call experience.

The primary issue with traditional WebRTC media servers and services is they didn’t horizontally scale. That’s changed recently. You can get pretty high numbers of users in a single WebRTC session now.

It really depends on the use case. In vanilla WebRTC, all media is transmitted directly between peers. In practice, this doesn’t scale beyond 5-10 users in a session. Most home internet connections can’t sustain that amount of upstream bandwidth.

Twitter has been and still is, its own worst enemy.

I agree — I’d love that.

Yeah! Wanted a really fast/frictionless way for people to jump into a context based on signals in their environment. This was also around the time of Color (the photo app which raised 40m or something off just a pitch deck) — “ambient computing” was becoming part of the zeitgeist.

Thanks for the kind words! <3