How is a coworker dragging you down? You just do your own work? Leave it to the manager to deal with it. Now if you're the manager, then I'd say you should question yourself if you really should be one if you can't deal with such a common situation.