Doesn't lazy.nvim have some support for that?

I use the LazyVim distribution but I remember something in the docs about per-repo plugin settings