Built in rechargeable batteries are a plus, not a minus.
I have three recent electronic devices that I would like to keep using but cannot, as their battery has reached end of life, replacements are hard to find, and changing the battery involves performing surgery on the device that I'm not confident I can do safely.
The battery door pops on and off with a fingernail.
They really do look like dead-ringers for AA batteries. I bet you could run the mouse off of a regular AA as long as you didn't try to plug it in!
If 3.7-4.2V is too much to handle, jump with a 1n4148 diode or two in series.