I'd highly recommend the YouTube channel "Journey to the Microcosmos" for videos that do a great job of improving that intuition.
All you have to do is go to a pond or creek scoop out some weeds, dirt and water and stick it in a jar and there is hours of exploration in front of you. Once you start looking its like seeing an evolutionary world in front of you, everythings eating each other and trying to not be eaten. If you're thinking of getting a telescope (I had one), think of a microscope they open up a new world, its incredibly easy to see things - I thought it would be a lot more work.
Probably there's so many on line resources now to - this guy is a great start - microbe hunter https://www.youtube.com/channel/UCDpKY8-oz5PWlBPNzNsbLug
In short, energy gradients (ie, on hydrothermal vents) can cause matter to organize itself into structures that more efficiently dissipate this energy. You can see how the creation of this structure would in turn enable the creation of a new kind of structure that lives on top of that.
and your last statement isn't getting to why. it's just what happens.
every living organism has some way of perpetuating itself, either by growing indefinitely or by making copies of itself. there's no reason "why", other than that organisms that can't do this get filtered out very early in the game. this implies having some sort of "strategy" for traveling up the energy gradient.
short of somehow evolving the biological capability to fuse/split atoms, all the usable energy on earth comes from the sun. so the first link in the chain has to be something like photosynthesis, where you collect and store energy directly from the sun (ie, "building a house"). but in doing this, you change the gradient, creating an opportunity for other organisms take your stored energy for themselves. in some sense, it's much "easier" for a rabbit to eat a plant in one minute than it was for the plant to collect all that energy over its entire life. there's only so much solar radiation per square meter, which limits a plants energy budget. a rabbit can eat many plants in a day, effectively multiplying the area of the solar radiation it can capture (and spend on acquiring even more). but then the rabbit creates an opportunity for a predator to harvest its stored energy and the cycle continues...
it seems like you're maybe asking "why can't all life just cooperate"? this would probably look like a planet full of plants (or a similar lifeform at a different scale). the problem is that this leaves a huge opening for the first "defector"; it's an unstable equilibrium. life does not evolve to leave opportunities on the table.
Organisms that consume plants are capable of more complex behaviour and growth because the amount of energy and nutrients etc. is greater that they receive by conauming things that photosynthesize.
Anyway... The 'why' is trivial-seeming to me. Once one organism can consume ready-made nutrients from another organism for less energy cost than producing, it will, or it will be out competed by others who do. So carnivores exist everywhere. Even deer happily eat baby birds that they come across, and rabbits eat their own.
Almost Tragedy of the Commons?
^ here is the link. Pretty great animation. Thanks for the recommendation.