The only point at which an address can be used to prevent fraud, is between the credit card charging process and your bank. If it's paypal charging your credit card, they're the only ones who need your address.
For example, an annoying system which will decrease chargebacks is "If your IP address goes to a country other than your billing address, shoot first and ask questions later." It bites me all the time, but there are sensible business reasons for it.