While it may or may not justify feeling bad, there are major moral theories (in particular virtue ethics and deontology, as well as metaethical-agnostic views) which easily accommodate discussion of virtual actions from moral points of view, and explain why they matter (in contrast to "it's just a game"). Fun alone (or the good feelings of playing such games with friends) does not excuse wrongful behaviour, in the same sense that ganging up with friends and bullying someone for fun does not excuse the moral weight of the action.

There is also the need to explain why many people do feel moral sentiment or even guilt after playing some games, and the moral intuition that some virtual actions are widely regarded as morally wrongful, such as virtual child abuse (and in contrast, why virtual murder would be okay, or at least better). Gary Young in particular investigates an empirically-minded approach to the problem.

It's often inconvenient for people who spend a lot of time playing games to even begin to question their actions in game, and unfortunately a lot of such thought is dismissed as moral pearl-clutching, but from what I can tell of the philosophical literature, there are good arguments from many points of view. It's very similar to the way in which even the discussion of the morality of killing animals was (and often still is) dismissed by people ignorant of the philosophical research because meat is just so tasty.

Here are some starters in the growing literature on the significance of actions in video games and virtual worlds generally:

Moral License in Video Games: When Being Right Can Mean Doing Wrong https://www.liebertpub.com/doi/abs/10.1089/cyber.2014.0599

Video Games, Violence, and the Ethics of Fantasy: Killing Time by Christopher Bartel (2020)

Defending the morality of violent video games by Marcus Schulzke (2010)

Violent video games and morality: A meta-ethical approach by Gary Young (2015)

The amoralist challenge to gaming and the gamer’s moral obligation by Sebastian Ostritsch (2017)

Getting away with murder: why virtual murder in MMORPGs can be wrong on Kantian grounds by Helen Ryland (2019)