Think of sports tickets. There is massive inefficiency in the primary market, meaning tickets you buy directly from the arena or on TicketMaster. One inefficiency comes from all the season ticket holders who skip the majority of the games and only go to the ones they actually want to see.
We can fix this inefficiency on a secondary market (the scalpers, StubHub, etc). A scalper might make a decent take by finding season ticket holders and buying the unused tickets. Then, the scalper sells the tickets to fans who waited too late to buy (or won't pay full price).
Now we think the scalper is evil, because he just made money without "adding value." Except he did add value... the season ticket holder and the last-minute-fan wouldn't have found each other without great effort. Both ended up happier than if the scalper didn't exist. Also, the scalper takes on some risk... he guaranteed the price he paid to the season ticket holder, had to hold the ticket, and then determine the correct time to sell. If he held the ticket too late (or got too greedy), the game would begin and he'd have a worthless ticket!
We also need to somewhat regulate scalpers because if they have too much of an impact, they can seriously distort and manipulate prices (a very wealthy scalper can buy all the tickets to a popular game and sell them all at 10x the price).
This analogy isn't perfect for high-frequency trading or all market making, but I hope it shows why arbitragers both add value and reduce risk yet seem evil to all the participants.