Comment by rvrb | Hacker News Reader

Comment by rvrb | Hacker News Reader

Parent Full thread

I haven't worked with GTK, but what you are describing here sounds reminiscent of what we have been dealing with trying to build Godot bindings in Zig with a nice API. the project is in mid-flight, but Godot:

  - has tons of OOP concepts: classes, virtual methods, properties, signals, etc
  - a C API to work with all of those concepts, define your own objects, properties, and so on
  - manages the lifetimes of any engine objects (you can attach userdata to any of them)
  - a whole tree of reference counted objects

it's a huge headache trying to figure out how to tie it into Zig idioms in a way that is an optimal API (specifically, dealing with lifetimes). we've come pretty far, but I am wondering if you have any additional insights or code snippets I should look at.

working on this problem produced this library, which I am not proud of: https://github.com/gdzig/oopz

here's a snippet that kind of demonstrates the state of the API at the moment: https://github.com/gdzig/gdzig/blob/master/example/src/Signa...

also.. now I want to attempt to write a Ghostty frontend as a Godot extension

Hopefully it's improved, but the last time I wrote a GTK binding for a language, it was miserable. 98% of it was sane, but the remaining 2% had things like "whether or not this function takes a reference to this object depends on the other parameters passed" which made liveness analysis "interesting."

This is what people mean when they say "make invalid states unrepresentable", languages with union types can easily avoid this problem.

It doesn't sound like they are talking about invalid states, more like they are taking about the kind of thing that in Rust would be represented by `Option<Box<dyn SomeTrait>>` or suchlike. Maybe your point is that in Rust much less ceremony is necessary to avoid hitting a null pointer when doing this. But still, in either language it's easy to end up with hard to follow logic when doing this.

not super familiar with Rust but isn't Option<T> just an union type of null and T? I get the language has special semantics for this compared to a union type but it is conceptionally just an union.

For example this is something you can do with typescript.

  function(args: Arguments) { ... }

  type Arguments = { a: number, b: number } | { a: number, b: string, c: number }

the Arguments { a: 1, b: 1, c: 1 } is not representable.

> not super familiar with Rust but isn't Option<T> just an union type of null and T?

Only if there is a niche optimization happens if T is never null, otherwise it's a tagged union.

That's not what you're replying to is about.

I'm pretty sure they're talking about reference counting that depends on the arguments, not about optional arguments or invalid argument combinations.

Thx for the work you're doing! Just out of curiosity, I sometimes struggle to write performant C# Godot code because it's hard to interface with the engine without doing a lot of back and forth conversions to engine types. You end up doing a lot of allocations. Did you run into that kind of stuff while creating your bindings?

My understanding may be out of date, but the C# support was created before GDExtension existed. The team has been working hard on porting it over to GDExtension. Once they are done, it should be much more performant, and they will finally be able to ship only one version of the editor. I believe the original C# bindings do a lot of unnecessary marshaling at the ABI.

With GDExtension, the core builtin types like `Vector3` are passed by value. Avoid unnecessarily wrapping them in Variant, a specialized tagged union, where you can. You can see the documentation here; you have direct access to the float fields: https://gdzig.github.io/gdzig/#gdzig.builtin.vector3.Vector3

Engine objects are passed around as opaque pointers into engine managed memory. The memory for your custom types is managed by you. You allocate the engine object and essentially attach userdata to it, tagged with a unique token provided to your extension. You can see the header functions that manage this: https://github.com/godotengine/godot/blob/e67074d0abe9b69f3d...

But, this is how the lifetimes for the objects gets slightly hairy (for us, the people creating the language bindings). Our goal with the Zig bindings is to make the allocations and lifetimes extremely obvious, a la Zig's philosophy of "No hidden memory allocations". It is proving somewhat challenging, but I think we can get there.

There's still a lot of surprising or unintuitive allocations that can happen when calling into Godot, but we hope to expose those. My current idea is to accept a `GodotAllocator` on those functions (and do nothing with it; just use it to signal the allocation). You can read the code for the `GodotAllocator` implementation: https://github.com/gdzig/gdzig/blob/master/gdzig/heap.zig#L8...

If we succeed, I think Zig can become the best language to write highly performant Godot extensions in.

Another thing it does badly and I am not sure if they are sorting out with the new design, is the magic methods crap.

They had to copy that bad idea from Unity, where methods are named in a specific way and then extracted via reflection.

Either provide specific interfaces that components have to implement, use attributes, or make use of generics with type constraints.

Maybe for Unity that made sense as they started with scripting languages, and then bolted on Mono on the side, but it never made sense to me.

> extracted via reflection

I think you are talking about dispatch of virtual methods, which is still a thing, but the performance cost can be somewhat mitigated.

the names of the methods are interned strings (called `StringName`). a naive implementation will allocate the `StringName`, but you can avoid the allocation with a static lifetime string. we expose a helper for comptime strings in Zig[0].

then, extension classes need to provide callback(s)[1] on registration that lookup and call the virtual methods. as far as I know, the lookup happens once, and the engine stores the function pointer, but I haven't confirmed this yet. it would be unfortunate if not.

at least right now in GDZig, though this might change, we use Zig comptime to generate a unique function pointer for every virtual function on a class[2]. this implementation was by the original `godot-zig` author (we are a fork). in theory we could inline the underlying function here with `@call(.always_inline)` and avoid an extra layer indirection, among other optimizations. it is an area I am still figuring out the best approach for

virtual methods are only ever necessary when interfacing with the editor, GDScript, or other extensions. don't pay the cost of a virtual method when you can just use a plain method call without the overhead.

[0]: https://gdzig.github.io/gdzig/#gdzig.builtin.string_name.Str...

[1]: https://github.com/godotengine/godot/blob/e67074d0abe9b69f3d...

[2]: https://github.com/gdzig/gdzig/blob/5abe02aa046162d31ed5c52f...

That and the exposition of _Something methods in C#, which don't follow C# conventions, private/protected methods have keywords for that.

Thanks for the overview.

I did not know this was a project that was in progress, and its quite exciting. I love Godot, and am quite fond a Zig as well. I'll be keeping my eye on this.

Reply on news.ycombinator.com