IBM's z/OS UNIX System Services is very nearly this. Many UNIX systems have tons of utilities that more resemble BusyBox than GNU coreutils, and AFAIK no C compiler is anywhere
near as compatible with GCC as clang (including, notably, slightly older versions of GCC itself).
But AFAIK z/OS is the only UNIX[1] variant that offers a "POSIX" environment
Where no subset of the native character set even resembles ASCII.
Where "everything is a file", but where many of these "files" — including, notably, both shared libraries and executable programs — are mostly inaccessible using anything resembling traditional byte-oriented UNIX utilities, library functions, and system calls.
Where process creation is not only "not cheap", but where it can in fact be so cripplingly expensive vis-à-vis conventional UNIX systems that the z/OS designers have (wisely) not only resurrected something akin to the traditional meaning of the "sticky bit" on executables, but have also made extensive provisions to allow both "subshell" scripts and executable subprocesses to be created within one's own address space, both via system calls and from within the POSIX shell itself[2].
On a related note, I was thinking the other day that an "evil", yet at least minimally standards-compliant filesystem in the spirit of your system might actually useful to have around for testing. Or, if not, would at least be fun to design.
Think a filesystem that imposes random per-directory length limitations on filenames and or file sizes, and occasionally allocates space for files in terms of random, per-file allocation unit sizes just because it can.
Or imagine a filesystem that flushes the overwhelming majority of data to disk promptly, but which also maintains a large "evil cache" containing a few blocks out out of the middle of every few million write calls that is flushed as infrequently as is permissible by the relevant standards.
And why let previously-allocated free space go to waste when, given careful planning, its contents could be used to present stale, yet technically "valid" data to applications that use I/O operations whose ordering with respect to one another is not formally defined for IPC (e.g., POSIX only explicitly defines ordering between its own read() and write() calls, so I/O by any means not passing through particular versions of these functions explicitly designated as POSIX-compliant can, in terms of standards-compliance, be "safely" ignored).
And so on.
[1] https://www.opengroup.org/openbrand/register/
[2] Google "_BPX_SHAREAS" for details.