Its not inherently wrong, but flying cross-controlled feels wrong to me. Probably because I don't do it very often, and I'm aware of the fact that it increases my stall-to-spin risk if I don't properly manage airspeed.
Here are some suggestions. As one of the child comment stated, go fly a glider for a few hours. It will immensely help with flying "not straight". I started it for gaining better understanding of flight but ended up finishing a commercial ticket on glider as well, because I enjoyed it so much and also just for the bragging rights.
Secondly, cross control is perfectly safe till the time you pull back on the stick. That is how you get into cross control stalls, the starting configuration for a spin. When you are slipping, you should actively trim nose down, (that you should have done already, remember you are approaching to land), and maintain just a little forward pressure as you fly sideways. I would recommend, grab a competent CFI and get cross control stalls nailed. I know it is not a part of PPL, something I am very pissed about. Hell, PPL in US does not even need spin training. Bollocks, if you ask me....
Private Pilot ACS Section IV Task M requires both demonstration and knowledge of "Forward Slip to a Landing"
https://www.faa.gov/sites/faa.gov/files/training_testing/tes...
It's actually a really nice way to land, I wish powered aircraft had this.
If you have the chance (and didn't already do so), make a few flights in a glider with an FI.
I thinks it's a different kind of flying and allows you to be more confident in certain situations such as slips, slow flights and engine out landings.