That's exactly what a threading library abstraction aims to do. For example the Intel TBB (Threading Building Blocks).