I was in the same boat with genetics and cholesterol. It is all the diet.
At some point I realized I had to stop eating cheese completely. No pizza, no nachos, no tacos with cheese. I can stay in good shape with a once a week cheat night but I don't have the genetics to deal with the cholesterol like that.
It sucked at first but the things I do enjoy now, I enjoy them just as much.