Several years ago, I almost did the same thing with my infant daughter. Put her car seat in the back seat. Got in. Started driving. On auto-pilot started heading towards the office.
It was only when she happened to make a sound (she often slept on the way to daycare) that I remembered she was still in the truck. A combination of factors led to this:
1. I wasn't getting anywhere near enough sleep at night with feedings every few hours.
2. Carseats can't go in the front seat anymore. Good for safety, but bad for remembering a kid is in the car since they're out of sight.
3. Did I mention I wasn't getting enough sleep?