You make a good point about colleagues in the same lab noticing unethical behavior. A shared research culture that values integrity and transparency is a good defense against malpractice and fraud. Building up and maintaining that culture—and the methodological skills to back it up—is a continual challenge.
For example Yoshihiro Sato was recently (2016-2017 time frame) caught fabricating results, turns out he had been doing it for 30 years. He had more than 20 papers retracted. Also note he wasn't caught by a fellow scientist in the field he was caught by a statistician bulk processing papers looking for anomalies. So yeah big penalties if you are caught, but the odds of getting caught are so low you have to be stunningly stupid to get caught.