Correct, though both layers remain active. The application-level firewall in the macOS GUI and the packet-based pf layer work on top of each other (I believe pf is on top of the application layer one but not 100% sure).

So if you have the application firewall on, opening ports in pf won't help.

I'm kinda surprised pf is still in there to be honest. I know some security solutions like McAfee Firewall use it under the hood. But they could do similar things with network extensions. I have expected them to drop it for years now.