Well international law prevents you from making someone stateless so they get thrown in jail instead. But yes, non-citizens who commit crimes are generally deported.
Also, the US classes crimes as either felonies or misdemeanours (possibly more categories?). In Canada we have indictable crimes, summary offences and things like jaywalking aren't even crimes, just ticketable offences. It's disingenuous of you to compare something like jaywalking to actual crimes.