The sound of the engine is usually the giveaway to me. Clatterclatterclatter... I sometimes have to roll up my window when I am at a stoplight next to a heavy-duty diesel.
But a big diesel will always sound a bit beefy I think. The smaller 4-cylinders shouldn't be discernable from gas at all.