It is important to remember that malicious websites and malware in general may negatively impact your security and privacy in extremely harmful ways (malware compromises PII, website credentials, financial information, uses webcam and microphone to photograph/film/record you from blackmail/revenge porn purposes, ...)
For context, please see these relevant Mozilla bugs about SafeBrowsing privacy concerns: [0], [1]. tl;dr Firefox must set a cookie for SafeBrowsing, but it uses a separate cookie jar for SafeBrowsing so Google cannot tie the Safebrowsing activity to anything else you do related to Google or their services (which is the biggest concern here). They can learn a limited profile of your browsing activity, along the lines of "Random user x often uses their browser between 9am and 5pm on M-F".
The Safebrowsing implementation is specifically designed to be privacy-preserving. [2] It uses a Bloom filter to implement fast lookups in a minimally sized hash table of known malicious URL's. The only time a full URL (actually various hashes of multiple prefixes of the full URL, including the full URL) that you browse it sent to Google is when a prefix of it collides with a known malicious URL, in which case the URL must be sent to Google to resolve the question of whether the URL you are trying to visit is actually malicious or just a false positive from the Bloom Filter. Yes, the hashes are unsalted so it would be possible for Google to check if you were trying visit some pre-determined URL ("were they trying to visit www.thoughtcrime.org?") but only if it collided with a known malicious URL.
It would be helpful to know what the average rates of collisions and false positives are to get a sense of how much of an average user's browsing history is leaked to Google through Safe Browsing - can anybody from Google comment?
[0]: https://bugzilla.mozilla.org/show_bug.cgi?id=368255 [1]: https://bugzilla.mozilla.org/show_bug.cgi?id=897516 [2]: https://code.google.com/p/google-safe-browsing/wiki/SafeBrow...