You don't know much about it. When you connect to a Plex server, you have to log into their backend, or it doesn't even know how to connect. It also does all the accounts/permissions stuff. An internet outtage is, unfortunately, a Plex outtage.
In rare cases, you may decide that you wish to allow very specific access from the local network without authentication. You might do this if using a third-party Plex app, which doesn’t support authentication, for instance (though all modern official and third-party apps should already support authentication).
To make an exception, look in your Plex Media Server’s advanced network settings, under Settings > Server > Network > List of IP addresses and networks that are allowed without auth.
Here, you can specify LAN addresses as either a specific IP or a IP/netmask (to specify a range). Separate multiple values by a comma and be sure not to include any whitespace (e.g. spaces).
I believe that the regular "external" connection requires a STUN server, so it will fail without Internet.