I ended up blocking Edge in the firewall so it would at least close quickly without loading any useless stuff. Even then the firewall rule would reset periodically so I made a new one that points to the Edge binary.
1. Find out where the Edge executable is; copy its path.
2. Open 'Windows Defender Firewall with Advanced Security'
3. Select Inbound Rules (Left pane).
3.1 Click on 'New Rule' (Right pane)
3.2 Set the type of rule to Program, click Next and paste the exe path and delete the quotation marks. Click Next.
3.3 Set the Action to 'Block the connection' and then select where this rules applies (select all three options). Click Next.
3.4 Specify a name (and an optional description) for the rule.
Done!
I also made a blocking outbound rule, but I don't think it's necessary for this purpose.