Excerpts from the wiki page below; seems like the monster did plenty harm to cause Frankenstein pain.
> He [the creature] traveled to Victor's [Frankenstein ] family estate using details from Victor's journal, murdered William, and framed Justine.
>Should Victor [Frankenstein] refuse his request, The Creature also threatens to kill Victor's remaining friends and loved ones and not stop until he completely ruins him.
> The Creature demands that Victor [Frankenstein] create a female companion....Fearing for his family, Victor reluctantly agrees.
> the Creature had strangled Clerval to death and left the corpse to be found where his creator had arrived
> The night following their [Victor [Frankenstein], Elizabeth] wedding, ... the Creature strangles Elizabeth to death