I was thinking along the lines of a simple extension to the Mach-O fat binary concept to allow each member of the fat binary archive to be associated with one or more ABIs rather than just a single architecture.
Then all the time_t size-independent code would go into a member associated with both 32-bit and 64-bit time_t ABIs, and all the time_t size-dependent code would be separately compiled into single-ABI members, one for each ABI.
For both executables and libraries, name conflicts between members associated with the same ABI would be prohibited at compile/link time.
The effective symbol table for both compile time and runtime linking would then be the union of symbol tables defined by all members associated with the active ABI.
Language-level mechanisms that allow compilers to recognize and separately compile time_t-dependent symbols would be required, ideally implemented in ways that allow such dependence to be inferred in most if not all cases.
While compilers would be free to size-optimize code by splitting functions into dependent and independent subparts, I see no immediate reason why this needs to be explicitly supported at link level.
Finally, as I imagine it, mixing 32- and 64-bit time_t ABIs would be prohibited — implicitly, as 64-bit time_t symbols aren't ever visible to 32-bit time_t code and vice versa — with code that needs to support non-native time_t values (for I/O, IPC, etc.) left its own devices, just like code dealing with, e.g., non-native integer and floating-point formats today.
Admittedly this sounds like a lot of unnecessary work vs. similar alternatives built on existing multi-arch mechanisms, but it's still an interesting idea to contemplate.