For me my fingers and face are the part that suffers most. Beause you are working out you do warm up.
Its only about 35-40 minutes for me so I can be a little lighter on the clothing. A balaclava for my face (Its a a ski mask with an big circular opening for my nose and eyes). Thats usually enough, but I have a thin hat that fits under the helmet. my work shirts are golf shirts and wick sweat away, but wool or polypro long underwear is helpful when its really really cold. You don't want to get wet from sweat, because it will suck the heat out of you. I usually just use normal pants. I have a flease jacket and a windproof shell on top.
I use wool glove liners and good windproof gloves. My hands can still get cold. Especially when its windy along the river. I'm experimenting with usb li-on glove warmers, which seem to help on the two times I've used them. They're kind bulky in my gloves and I don't love that.
(I've seen people with fixed hand warmers on there bikes but I haven't gone that far). Ski googles too (I wear glasses so that helps the wind in my eye problem).
Good socks too. I lower my seat in the winter to make it easier to touch ground if I hit ice.