Perhaps the most difficult body parts to keep warm on cold days are the hands and feet. For finger protection, a lot depends on how warm your hands get while riding. Cyclists with excellent circulation and hot hands can get by with basic long-finger gloves (anything much thicker may cause overheating and sweating).
If your fingers are more susceptible to the cold, consider a thicker or different glove type such as a mitten design that keeps the fingers together to add warmth. It's also possible to purchase glove liners and benefit from the same layering approach you use on your torso and legs to stay warm.
Frozen toes are no fun, either. Like keeping your hands comfortable, what you wear has mostly to do with how your feet handle the cold. Shoes often determine comfort. For example, if you normally ride in thin, lightweight road shoes, consider riding in a heavier off-road pair if you have them. Often the off-road shoes are thicker and much warmer. Even if you have to swap your pedals from your off-road to street bike, it's worth it to keep your feet from freezing.
Other suggestions for keeping the tootsies toasty include thicker socks, wind-proof, breathable socks, and sliding plastic bags over your feet before putting on your shoes. Wearing booties, which are heavy-duty insulating shoe covers, is another option. And, booties do more than keep your feet warm. They also keep water out of your shoes and protect your cycling dogs from the ravages of winter weather. Neoprene toe covers are fairly new items that work great on all but the coldest of days. They provide your toes with a complete wind shield, but preserve your ankle flexibility.
Now that your hands and feet are comfy, attend to some other important parts of your anatomy, namely your ears and face. Something that cold-weather diehards find useful is a balaclava, a thin, breathable hood that fits under the helmet to protect the ears, neck and face. For less frigid conditions, a simple ear band can work, too. It's best to be prepared by carrying a selection of these easily packed items in your bike bag or pack.