It's from an era before ubiquitous MMUs, even in IBM mainframes. It therefore was structured as a library OS, with even less separation between it and the application code than even MSDOS and it's applications.

Transaction Processing Facility: A Guide for Application Programmers by R. Jason Martin is a decent book I'd recommend on the subject.

I don't know of much public information other than their job postings.