They could have used a more classic NT environment subsystem architecture for WSL1, it would have mostly worked. The primary reason they didn't was performance, and to a lesser extent compatibility.
Unix systems create lots of short-lived processes, because processes on Unix are relatively cheap. By contrast, NT processes are much heavier weight. This is part of why the POSIX subsystem / Interix / SFU / SUA and Cygwin too always have had performance problems. Introducing a new lightweight process concept, the picoprocess, helped get over the performance issue. But these new lightweight processes can't be allowed to make arbitrary NT syscalls, which rules out the classic NT environment subsystem implementation pattern - have DLLs which make NT syscalls instead of e.g. OS/2 syscalls.
If they'd followed the classical Windows NT environment model, they would have enhanced NT to be able to load ELF executables and then shipped their own libc.so which made NT syscalls. That would have worked fine for most apps which don't make Linux syscalls directly, but would have had worse performance than WSL1 had. It also would have meant more work for them, since they'd be emulating not just the syscall layer but the libc layer too. (They could have sped things up by reusing the GNU libc code; to do that, they would have had to have open sourced most of the code of their subsystem, which would have probably been a positive overall, but I'm not sure how feasible that would have been given internal Microsoft politics around open source.)
To support the minority of apps which make Linux syscalls directly, bypassing libc, they needed something in the kernel to intercept the SYSENTER/SYSCALL/INT 0x80. They could have redirected it back to user mode for handling in their libc.so. That would have worked, it just would have been slower than WSL1 is. (And WSL1 already has enough performance issues, that performance was one of the main motivators for switching to the new architecture in WSL2.)