It's not an issue of "alignment." The reward function was poorly written. That's it. It's like claiming the Ariane 5 explosion shows that FORTRAN needs "alignment."