Except most US voters disagree with you. Someone married to a US citizen does have residency rights, notwithstanding the paperwork quirk that you're supposed to exit and re-enter, which typically involves flying somewhere going to the US embassy to get a stamp and flying back. So just as most people don't support the death penalty for speeding, most people don't support criminal deportation for someone who has the right to be in the US but for whatever reason (perhaps lack of money or perhaps fear of strip searching and disappearing to the gulag) didn't follow the proper process. Because most voters don't see this situation as a crime and certainly not one requiring deportation, the law doesn't treat this situation as a serious crime, or actually a crime at all.
If you want to aggressively going after folks who have skirted immigration rules perhaps the place to begin is in the east wing (if it still existed).