On the neovim subreddit one of the core maintainers said that relying on users to lazy load plugins is an extremely poor practice and something that should be done by the author's of said plugins. It is just a matter of how you initialize your plugin/name-of-plugin.lua file.

Don't know if it helps but I recently migrated to vim.pack. With another neovimmer he helped me create a defer function and pack update. The only plugin I initially load is a dashboard while deferring everything else. Brought my startup time to sub 100ms.

https://github.com/azemetre/dotfiles/pull/61/files

Depending on the number of plugins you have, you may not notice the difference in practice. I certainly don’t but I’m very much on the minimal side of things. This all said I’m definitely of the school of thought that plugin developers should be responsible for ensuring their plugins are lazy loaded, rather than leaving it up to the user, who is not as well placed to make the decision.