Did you add a captcha to your login? You could add one after 2 failed attempts from an ip.
I understand not wanting 2FA for lower conversion rates, but I'd recommend implementing it in the future. It's one of the best ways to mitigate an attack against knowing your password. The session cookie is another story.
As for the passwords, I'd enforce standards around what a user is allowed to use. You want to have it be at least 10+ chars with special characters. If they reuse the password on another site and that gets compromised there really isn't much you can do. However, if we assume the reused password from the other site is hashed with a salt and you enforce more chars, they are much harder to crack and thus prevent the plaintext password from being recovered and accessing your site.
A great way to demo what more chars does to a password here
https://www.betterbuys.com/estimating-password-cracking-time...
You should also maintain a record of the common IP address an account uses and require an email/text to allow access that would further mitigate your problem. I know your a startup, so this is more of a luxury feature.