I don’t think most will outright not visit a site for not having dark mode, but it’s certainly good enough reason to avoid it and find alternatives that do have dark mode.
Agreed. It seems like a lot of people don't understand that dark mode is an accessibility issue for many people. If I couldn't use dark mode for the majority of my apps, I probably couldn't be a developer. Even with a monitor with a blue light filter and carefully adjusting the brightness and contrast to be as gentle on my eyes as possible, some days, my eyes are sensitive enough that even 15 minutes on light mode apps causes blurry vision and headaches. Dark mode I can use all day, no problem.