First, you should define your goal. Do you want to be generally fit, or do you just want larger biceps? Can’t force you to be well-rounded and ask you to change your perception of beauty, so if you just wanted bigger biceps then go ahead and just do curls. Be warned though: other fit people are going to notice the imbalance in your body such as skinny legs, underdeveloped forearms, bad upper body posture, etcetera; and even your perception of beauty will change as you progress, so I strongly suggest that you do not train for just one muscle.
If you go for general fitness, start with the most basic compound lifts: barbell bench presses, either squat or deadlift, and then barbell rows (funny how no one mentioned this exercise, lol). These are the most essential movements involving the largest and most essential muscle groups that are used in any other exercise. And note how I speak of muscle groups — in order to perform a muscle contraction, you’re actually not just using a single muscle. You have prime mover muscles, and others are stabilizers, and you want to train the whole group by doing exercises in full-range of motion so that they are all developed, because that will keep you safe from injury when doing more challenging exercises with them.
If you want to know more about the whys, the subject that you want to study is called exercise physiology.
There’s also the fact that should rock climbers, boulderers, and mountaineers have lives and real jobs outside what they do for recreation, then the reality is that they have a variety of bodily movements that just need to be generally done with, again, good posture, which you simply can’t have with an imbalance in the essentials.
As a beginner, you would not. You want to work the whole system, and to do that, you only need a handful of compound movements: squat, deadlift, and overhead press.
Once you slow down in progress on these bigger compounds, that's when you would want to start doing more targeted or more specific assistance exercises, in addition to the compounds.
Starting Strength is a good enough book to learn this stuff. Read it and beware advice on the subject of fitness, for every nerd, bro, and their brother gets a kick out of speaking with authority on this stuff, even if they don't know what they're talking about - just go straight to a reliable source: Starting Strength. It's become a meme, but it's good enough.
Disclaimer: I'm not a fitness expert either, just train regularly. So, verify anything I (or anyone else) tells you.
They work different muscles.
OHP/MP mainly works the delt, while the BP targets vrious parts of the pecs (depending on incline). Also grip width can change the balance between triceps/pec activation during BP
OHP gains certainly do carry over to bench press though. You can treat bench press as a "heavy" day lift and then use overhead press as a lighter pressing variation (but still go heavy) because there's enough overlap that it'll help your bench but different enough to where going heavy on overhead doesn't impact your bench press recovery.
Example:
Monday: 5 sets 5 reps of bench press.
Wednesday: 3 sets 5 reps of overhead press.
Friday: 1 set 5 reps bench press (trying to get as many reps as you can with good form, and increase the weight for next week depending on how many reps you went past 5)
Another way I've seen it done is something like this A B style workout:
Week A:
Monday - bench Wednesday - OHP Friday - bench
Week B:
Monday - OHP Wednesday - bench Friday - OHP
One really good source of info for powerlfiting / strength programming on youtube is Alex Bromely
My program is structured around the compound movements forming the Cartesian product of {horizontal, vertical, leg} × {push, pull}; bench press is a horizontal push in this arena. Overhead press would be a vertical push, and squat would be a leg push.
I'm rambling a bit, but the point is that compound movements are so effective and they're really the most efficient way to drive strength and hypertrophy gains, especially as a noob, that it wouldn't make sense to do a ton of isolation movements. And even if you did do a bunch of isolation (as opposed to compound lifts), it wouldn't make sense to just pick one muscle while minimizing others while you're still building up a base of strength and physique.
For many people, it's aesthetically driven: bodybuilding is a form of beauty contest where a certain kind of muscle development and leanness is considered ideal, depending on the specific contest you're entering. And so you train and diet accordingly: gaining mass in areas that need to be brought up, trying to even out two sides where their size differs, dieting down to show more of the anatomy. Even if you aren't aiming to enter shows, many people will use these parameters as an aesthetic target.
If you're in it for general health, the distinctions are interesting trivia, but you're basically served by doing exercises that are known to train all parts of the body. Combat sports offer a good model for this kind of training since they demand an overall balance of athleticism, and to eliminate exploitable weaknesses.
1. Aesthetics - you want to change some part of your physique by targeting certain muscles. Think doing curls for bigger biceps.
2. Performance - you want to get stronger/improve mobility by targeting muscles or groups of muscles associated with a movement. Think doing hamstring curls to improve one's vertical jump or deadlift.
3. Therapeutic - you want to gain/regain/prevent the loss of capability. Something like deadlifts for back pain and posture or band pull aparts for shoulder pain.
More specifically:
Most strength and bodybuilding programs follow a template where each session has a main compound movement followed by accessory movements that target a handful of different muscles, usually split by upper/lower body or by type of movement (pushing/pulling) with the intent of working most of your muscles. Many non-compound movements for any given muscle are more or less interchangeable if you want variety in your program, have something that prevents you from doing a particular movement or just don't have access to the required equipment.
For instance, overhead dumbbell tricep extensions give me elbow pain, so I do cable tricep extensions or skullcrushers if I don't have access to a cable machine.
Edit, this is my website.
In regards to your "why," it's a pretty broad question to answer. Why should you do weight training? It's good for your health, usually, ask your doctor for if that's true for you specifically (usually the answer is yes). As in, studies show health benefits unrelated to the actual benefits of bigger muscles and greater strength:
1. Improves heart health 1, 2, 3, 4 (sorry too lazy to properly format cites)
2. Lowers risk of diabetes, good for managing blood sugar 5 6 7 8
3. Positive psychological effects (outside of just feeling better about your appearance, it literally affects your brain chemistry) 9 10 11 12
Consider also some of the more "common sense" or "obvious" side effects of simply being stronger: reduces your risk of injury, increases your bone strength, boosts your self esteem (even if your culture doesn't value macho muscly people, simply being stronger can be a self esteem boost), and, being stronger is just kinda fun and nice and convenient. Consider: one trip from car to the house with grocery bags, or, picking up your dog / kid / friends more easily. Even when I reach up for a big bag of flour and feel my muscles at work, I find some amount of satisfaction.
If you mean "why should I work a certain muscle group," the answers are all over the place, and really situational. Someone might be working a specific muscle group because of an injury and are doing restrengthening or creating greater strength in a way to prevent future injury, or, because they need a rebalancing for compound exercises (i.e. their abdominal strength is a blocker for greater deadlift weight), or, aesthetics (bicep curls, all day, every day, or, the more modern variation: waaaay too many squats).
If you're "someone who knows almost nothing about muscle building," and are also interested in it, I recommend a workout plan that involves compound exercises. Start with your why so you have a goal in mind to drive you, i.e., I wanna be able to carry my kid for longer, I want to increase my quality of life, I want to appear more aesthetic, I want to feel more confident when hiking with weight, whatever, pick something. Then, consider your equipment situation: do you have access to a gym and the time to go there? If so, barbell compound exercises are worth investigating. If not, bodyweight exercises (these are great to do with barbell exercises if you have the time, I really think everyone should be doing at least a little bit of bodyweight exercises every few days).
From there hit the forums:
https://thefitness.wiki/getting-started-with-fitness/
https://thefitness.wiki/muscle-building-101/
https://thefitness.wiki/routines/
https://www.reddit.com/r/Fitness/
If you're looking for something to do literally right now, and you haven't had a doctor advise against physical exercises for whatever reason, I guess get on the ground next to your chair and see how many pushups you can do? Here, the muscle wiki is useful for this: https://musclewiki.com/bodyweight/male/chest/push-up though it might help to watch a youtube video as well, I really love the "Hybrid Calisthenics" channel https://www.youtube.com/watch?v=0GsVJsS6474 if you can't do a "full plank pushup" they have a good pushup "ramp up" method.
1 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6563593/
2 https://pubmed.ncbi.nlm.nih.gov/27680663/
3 https://pubmed.ncbi.nlm.nih.gov/28769100/
4 https://pubmed.ncbi.nlm.nih.gov/31023184/
5 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5446383/
6 https://pubmed.ncbi.nlm.nih.gov/26562712/
7 https://pubmed.ncbi.nlm.nih.gov/27926890/
8 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6339182/
9 https://pubmed.ncbi.nlm.nih.gov/28819746/
10 https://pubmed.ncbi.nlm.nih.gov/29800984/