similar energy to my https://github.com/lunixbochs/crossldso
1,848 karma · joined April 25, 2012
I think my small Talon models are next, around 3% lsc clean at ~28M (greedy CTC decoding, no external encoder, no LM, not trained in a domain specific way). I reached around 6.5% at 10M.
I've been working on some new baselines I want to release soon as public artifacts. This article is inspiring me to try pushing the param size down a bit. I suspect we can do large vocabulary end to end in the <5M range.
> You're not interested in addressing customers' needs
I would love to support Wayland, but it is my position that it is impossible to "support Wayland" for Talon. I can only support a subset of the features and only on specific compositors, and it would be a lot of work.
> or giving them ways to address their needs themselves
As I said at the top of the message you are replying to, I believe today users already have the tools to address their needs themselves with about the same level of jank I'd be able to provide on Wayland. If this is a veiled hard line on open source being the only way for users to address their needs themselves, we have a philosophical difference that won't be sorted out in this thread.
> If slack interactions are so unpleasant, why do you direct all support through it?
That's a whole new sentence. I was specifically referring to the support requests for Wayland, which in the long tail have been more hostile toward me than is likely warranted.
> "yes this is a hack, but we'll live with it for now until an actually good solution is available"
The hack is switching to X11, which is fully supported, or working around it in your user scripts, which has already been done by some users for their specific environment.
It's possible to do the simple compositor specific hacks from Talon's scripting system to give yourself partial Wayland support at roughly the quality I'd be able to provide myself, and I know of a couple efforts to do this.
The tentative plan for "dropping support for X11" is just to do one more public Linux X11 release, stop there, leave it available to download, and make it very clear what to expect when you download for Linux or run on Wayland. I plan to continue supporting X11 on the paid version indefinitely.
Most requests relating to Wayland on the Slack have not been offers to help, and way too many have ended up being unpleasant conversations.
I have a standing offer to reconsider my stance if someone can show the vast majority of the necessary APIs are available and well supported without compositor specific hacks.
For some of the other points above, consider me disappointed but not surprised.
e.g. dual channel zen 1 showing 25GB/s on a single core https://stackoverflow.com/a/44948720
I wrote some microbenchmarks for single-threaded memcpy
zen 2 (8-channel DDR4)
naive c:
17GB/s
non-temporal avx:
35GB/s
Xeon-D 1541 (2-channel DDR4, my weakest system, ten years old)
naive c:
9GB/s
non-temporal avx:
13.5GB/s
apple silicon tests
(warm = generate new source buffer, memset(0) output buffer, add memory fence, then run the same copy again)
m3
naive c:
17GB/s cold, 41GB/s warm
non-temporal neon:
78GB/s cold+warm
m3 max
naive c:
25GB/s cold, 65GB/s warm
non-temporal neon:
49GB/s cold, 125GB/s warm
m4 pro
naive c:
13.8GB/s cold, 65GB/s warm
non-temporal neon:
49GB/s cold, 125GB/s warm
(I'm not actually sure offhand why asi warm is so much faster than cold - the source buffer is filled with new random data each iteration, I'm using memory fences, and I still see the speedup with 16GB src/dst buffers much larger than cache. x86/linux didn't have any kind of cold/warm test difference. my guess would be that it's something about kernel page accounting and not related to the cpu)
This was a very basic experiment. I expect you could perform the DCT more intelligently on the vector dimensions instead of trying to pack the embeddings into pixels, and get higher quality semantic compression.
Neverball was working in the original glshim project before ptitseb forked it to gl4es. (Not to discount the significant work he's put in since, including the ES2 backend)
If you encounter what feels like poor recognition in Talon, I recommend enabling Save Recordings and zipping+sharing some examples on the Slack and asking for advice.
The current command set is definitely harder to learn than a system designed for chat/email where "what you say is what you get", but it's much more powerful for tasks like programming once you learn it.
I'm dubious about what kind of general command accuracy Numen is able to get with the Vosk models, as Vosk to my understanding is more designed for natural language than commands.
Assuming you mean speaking in natural language, that's slower to say, and likely less precise and predictable if you want to be able to just say "anything" any have a result.
You need a command system either way. If you want to express some precise intention, you need to understand what the command system will do.
There is a combined "mixed mode" system I've been testing in the talon beta where you can use both phrases and commands without switching modes.
It's also worth saying that if you only tried things out briefly, there are a handful of reasons recognition may have seemed worse. Talon uses a strict command system by default, because that improves precision and speed for trained users, but the tradeoff there is it's more confusing for people who haven't learned it yet.
For example, Talon isn't in "dictation mode" by default, so you need to switch to that if you're trying to write email-like text and don't want to prefix your phrases with a command like "say".
The timeout system may also be confusing at first. When you pause, Talon assumes you were done speaking and tries to run whatever you said. You can mitigate this by speaking faster or increasing the timeout.
The default commands (like the alphabet) may also just not be very good for some accents, and that will be the case for any speech engine - you will likely need to change some commands if they're hard to enunciate in your accent.
I recommend joining the slack [1] and asking there if you want more specific feedback. I definitely want to support many accents and even have some users testing Talon with other spoken languages.
Due to whisper's weakly supervised training on a large amount of automatically scraped data and reliance on a bigger language model, it's far more likely whisper had seen some of the test data before.
https://ai.googleblog.com/2021/01/improving-mobile-app-acces...
[1] https://twitter.com/lunixbochs/status/1574848899897884672
This approach even works with x86 user-mode emulation, by serializing render commands into a shared memory ring buffer and running the driver in native code out-of-process (which as a bonus was also generally faster than rendering in-process).
I built glshim after hand porting a bunch of open-source Linux games from OpenGL to OpenGL ES. The last game I hand-ported like this was Introversion's Uplink Hacker Elite. The diff was thousands of lines, and the resulting port had buggy rendering and blurry fonts, so I got fed up with hand ports and started working on a universal solution.
After a few years, I had ironed out most of the bugs in glshim, and ports to the Pandora were much easier. Uplink, however, was still in a sorry state. It made heavy use of glBitmap for crisp font rendering, and glBitmap was very slow to emulate on top of OpenGL ES 1.x. glBitmap basically manipulates framebuffer pixels directly, and the easiest way I found to do that was to upload a texture and render it with GL_NEAREST. Texture uploads seemed to be extremely slow and CPU-bound on the Pandora's Cortex A8 / PowerVR SGX 530. I ended up batching subsequent glBitmap calls, trying to coalesce them into a single texture upload per frame, and with that I still only managed single-digit fps.
With glshim, I had a moderately complete OpenGL implementation that only relied on a small subset of the OpenGL ES API. Then I stumbled on TinyGL. I realized that despite TinyGL not implementing much of the full OpenGL API, it was very fast and implemented basically everything I needed from OpenGL ES for glshim to work. So I forked it as "TinyGLES", deleted the extra GL parts, and optimized the rendering paths for ARM NEON. It ended up making a really good software rendering backend for glshim. I used this to finally release a good version of Uplink for the Pandora, which ran at around 80fps! With TinyGLES, glBitmap could blit directly to the software framebuffer, avoiding the slow/buggy texture emulation path.
It had been a few years and I couldn't find the modified source for my Uplink port, so I just swapped out the libGL.so file when I updated Uplink to use TinyGLES. But some users found an annoying bug in the port. Anytime you restarted the game, it put you directly in the "new game" menu, instead of allowing you to pick between new game or loading a save. I ended up fixing this with a binary patch to the game executable (which forced the game to the main menu screen on every launch), and that's the version currently available for the Pandora: http://repo.openpandora.org/?page=detail&app=uplink
My aotool project uses a trick to extract the AOT binary without root or disabling SIP: https://github.com/lunixbochs/meta/tree/master/utils/aotool
Talon's speech engine backend is modular, with Dragon, Vosk, the WebSpeech API, and Talon's own engine all used in different ways by users.
You could use really small chunk sizes and process them in a streaming fashion, but that would impact accuracy, as you're significantly limiting available context.
Sure, the Whisper Tiny model is probably going to be fast enough, but from my preliminary results I'm not sure it will be any better than other models that are much much faster at this power class.
Whisper Large looks pretty cool, but it seems much harder to run in any meaningful realtime fashion. It's likely pretty useful for batch transcription though.
Even if you hit a realtime factor of 1x, the model can leverage up to 30 seconds of future audio context. So at 1x, if you speak for 10 seconds, you'll potentially need to wait another 10 seconds to use the result. This kind of latency is generally unsatisfying.
tiny.en: ~18 sec/sec
base.en: ~14 sec/sec
small.en: ~6 sec sec/sec
medium.en: ~2.2 sec/sec
large: ~1.0 sec/sec (fairly wide variance when ramping up as this is slow to process individual clips)