"Random scenario: a burglary gang gains access to the database and can thereby identify patients aged over 80 living alone in up-market neighbourhoods who have recently been admitted to hospital with conditions suggesting that they will be vulnerable but not supported by full-time carers. A religious organization targets men of a certain age who are HIV positive. Or women below a certain age who are single and pregnant. Or an insurance company notes that a patient made a mistake in their declaration of a pre-existing condition, and thereby invalidates their claim. An identity thief uses the postcode and date of birth, in conjunction with a copy of the public electoral register, to pick victims. The possibilities are endless"
I can add a few more. A wife beater looks up his wife's records to see if she has gone into hospital. A rape victim doesn't want her work colleagues to idly google her medical records and find out. A hopeful job candidate doesn't want her interviewer to know about her problems with self-harm 15 years ago.
Perhaps none of those situations are relevant to you. But they apply to some people and that's why we keep all records private.