same thing, done in a different way. the only notable difference (except of writing less code) is that zig does not type-check the code which you don't use, so the genericity goes a bit further than what you can do in rust.
601 karma · joined May 29, 2017
TBH perf was not my priority yet, I only wanted predictable memory usage and Zig was perfect choice for that (when compared to node.js).
- zig has powerful meta-programming, it's a bit like functional programming with structs. I would consider that high-level concept, at least comparable to traits and constraints
- zig has anytype duck-typing. in any function, you can say that some arg is anytype, and then you can pass anything to it. type-checking still works, because it's done when you actually instantiate the function with known type. this is great for flat and simple API surface.
- server-side code usually have short life-time (request), so you can just use arena for all allocations, and free everything when you finish, this is super-simple to do in zig, because every api accepts allocator if it needs to allocate. rust arenas have lifetimes, which makes them problematic to use/embed deep down in the hiearchy. zig is not aiming for 100% safety so this is easy.
Instead of doing quadratic attention (computing how much each token attends to every other token) you just "somehow" compute a long (same length as input) convolution kernel, and then you apply the conv1d.
Again, from my limited understanding, it's bit related to applying FFT, doing some matmul and then IFFT back. We know that this works but it's slow. But there are many ways to compute FFT and one of them is with something called butterfly matrices. I think it's just approximation but it's good enough and it's very fast/efficient on current hardware.
To put this in context, quadratic sounds bad, but in practice, subquadratic algos are often slower because of hw limitations. So while there was a lot of excitement about SSM it's not so easy to say that llama is over now. Also, we don't know if mamba will scale up, and the only way to know that is to actually pay few millions for training. But I am optimistic.
Another interesting model from subquadratic family is RWKV. Worth checking, but I think you had a podcast about it :)
BTW: I am self-thought and I've only skimmed the paper some time ago so I might be very wrong.
BTW2: Another thing with attention is that there's usually KV-cache, which helps a lot with performance, and I think you cannot do that with mamba.
The Workflows screen is feature-flagged for now and only visible in the DEV build.
Yes, that's the whole idea behind QuickTools - to make repetitive tasks available quickly. It's also supposed to be in the main menu (drag-drop, reorderable, etc.) but I didn't have time to work on that yet.
Very cool idea but it never worked out (to my knowledge).
yes, I know it definitely works, so it has to be some stupid mistake but it's impossible to reproduce it without actual intel mac :-/
I had enough trouble with supporting monterey, which also does not work 100% AFAIK
The big news here is that it's now open-source: https://github.com/cztomsik/ava
If a function is called in comptime, it can also return types. So for example:
// this is a function which accepts a type
// if you accept type, you also have to restrict the arg to be comptime
// if the arg is comptime it still does not mean that the function cannot be called in runtime,
// but in this case, it returns type, so it is comptime-only function
// there are also cases where this is not true, like std.mem.eql(u8, a, b) which accepts type,
// but can be called in runtime because it does not return type
fn Wrapper(comptime T: type) type {
return struct { value: T };
}
const F32Wrapper = Wrapper(f32);
// @TypeOf(x.value) == f32
var x: F32Wrapper = .{ value: 1.0 }And you just keep refactoring/improving/desperately trying different ideas, because it never feels right.
It's ok if you don't agree but pls don't try to make my responses look like I like Rust, I don't and I'd happily get those years back if it was possible.
So, if there is some information at the bottom which is dependent on something which is at the top, your embedding could be entirely wrong.
Zig doesn't have any of that. It's very interesting combination of low-level, predictable code, with meta-programming, where you get some of that abstraction back.
i.e. Zig does not have generics, but your function can return type, so generic list is just a function which returns a newly created struct.
You can always improve it, but there's no need to. With Rust, I was never happy, even after 5 years, I was still thinking about better abstractions and implementing more traits, making it more generic, etc.
I'm not sure if you are familiar with how virtual memory works but if you are, then I don't understand what's your point.
And on the OS side, I feel like Apple is creating features that I instantly disable because they do the exact opposite of what I want.