Not by blocking outbound ldap by port, because ldap://hurrdurr:443/Evil.class
If you want to go down the route, you should check if log4j allows a port to be specified in the LDAP URI. If it does, firewalling one port won't do anything.
Yes, you can specify a port.
The vulnerability affects any process with network-facing endpoints that log user-input data. It's not LDAP-specific.