I noticed the Certificate Assistant UI in Keychain Access only lets you choose as high as 2048 bits if you're generating certs that way.

com.apple.security isn't a file, it's a preferences domain. Preference domains are stored in ~/Library/Preferences/<domain>.plist for per-user domains, and /Library/Preferences/<domain>.plist for system domains. In this case, it's stored in /Library/Preferences/com.apple.security.plist.

If a preferences domain doesn't exist (or if the key doesn't exist within the domain), the app or system default is used.