As for what makes practice "self defence" as opposed to "martial arts" -- there really shouldn't be any contradiction - "martial arts" really is about fighting, maiming hurting and killing people. Or it was. But of course there are lot of other benefits to what is good "martial arts" training - soundness of mind and body etc.
But if what one wants to learn is primarily "self defence", then one has to, unsurprisingly, practice self defence. Practice the type of situations you want to defend yourself from. I could probably link to videos all day, but eg: this one with Phil Thompson explains some of the basics (despite the cheezy intro):
"Protect Self Defence on TV3's Pacific Beat Street"
https://www.youtube.com/watch?v=LLkKE-gSuj0
Some more snippets with Thompson:
"Self-Defence Seminar with Phil Thompson - The Passive Stance"
https://www.youtube.com/watch?v=S9cU6yGSgB8
"Self-Defence Seminar with Phil Thompson - Wrist Grabs"
https://www.youtube.com/watch?v=KlI3rzsml_g
There's also something to be learned from military martial arts, like (this is more typical Martial Arts wanking, so to speak, but I really love this guy):
"Systema Demonstartion on Korean Special Forces"
https://www.youtube.com/watch?v=SfHfua9ku4s
Note, don't put too much emphasis on the "Systema"-bit, DK Yoo probably holds high level dan grades in more martial arts styles than I even know about. That said, Systema is a very interesting style - developed by and for the Russian/Soviet special forces with inspiration from traditional Russian wrestling and judo -- it's a surprisingly soft style for a modern military martial art. Like man other styles, if you can find a good instructor/dojo I'm sure it can be a perfectly fine approach to self defence. But as Thompson says above, it's not the physical techniques that are the hard part.
The reason I call out Krav Maga (developed from practical self defence in the Warsaw ghettoes, and evolved by the Isreali military and special forces) is that it is modern (as in new), it has a pretty reasonable and well-functioning system of grading instructors - so if something "really is" Krav Maga, it's a bit easier to know a little about the quality of the training - unlike, say "karate" or "jujutsu" - or "taekwondo" (which encompass everything from the funny stuff that is in the olympics to more traditional stuff that is closer to the founder's original vision [ed: note that the second two videos with Thompson is from a taekwondo seminar, so there are obviously still places where you can practice taewkondo and actually learn something useful]) - and the general civilian training (to the best of my knowledge) has a strong focus on self-defence - practising actually useful stuff like verbal assault, multiple attackers, situation awareness and so on.
It very much is a system for self defence, while most other things one could practice ends up being "martial arts with self defence added on". Personally I prefer that, I don't really enjoy practising just self defence - not to mention the fact that I'm not entirely convinced there are enough situations that are dangerous enough that I really need to defend myself violently, and yet not so dangerous that I won't simply be stabbed to death or shot -- that's its really worth the effort to practice all that much with the aim of self defence.
But practising any kind of physical confrontation for fun will boost confidence, and give one a better idea of when a situation is dangerous, and how it might be dangerous.