Hacker News Reader

Comment by sp332 | Hacker News Reader

According to https://research.fb.com/wp-content/uploads/2020/01/Scaling-u... the benchmarks were run on "Intel Skylake CPUs with 18 physical cores and 64GB of RAM."

View on HN

Yes, in the paper we discuss our benchmarks on Intel CPUs. But, as we mention in the final section, we also made the system work efficiently on IOS and Android, but we haven't open-sourced them in this release. This will be in our future work.

For reference, our system runs at 0.1 RTF on iPhone 10 using Accelerate framework under FP16 precision. INT8 should be better but haven't benchmarked yet!

The real-time factor for a single audio stream looks like 0.1 (based on eyeballing the graph), so it should be possible to achieve acceptable speeds even with a slower CPU (maybe not a Pi). The memory requirements for the intermediate results are likely to be substantial, though. They say they have "carefully optimized memory use", but don't give any figures.

They benchmarked several configurations, but I couldn't match up which configuration of models produced which results. I was trying to figure out if you drop the CPU power so throughput just handles one person talking at a normal pace, whether latency would necessarily get crazy high.

So no, lol

They say they're coming out with Android and iOS versions soon, so maybe take a look after that point to see how they've tweaked the models and if the error rates are a lot higher.

FWIW, we have already working versions in Android, iOS but didn't have time to open-source it with the current release. This is certainly in our future work.

Reply on news.ycombinator.com