I just tried to turn off one of my droplets and had to click "okay" on a message that said "You will still be billed for Droplets that are turned off because your disk space, CPU, RAM, and IP address will still be reserved."

If you clicked okay without actually reading the message, that is entirely your own fault.