For standard Wi-Fi the biggest factors for a fresh association are:
- Discovery. You have to wait to see which saved networks are broadcasting. Broadcasting more often = less efficient airspace for already attached clients. Broadcasting less often = longer delay for clients to "see" the network when they start listening.
- External authentication. E.g. if you're doing RADIUS auth or MAC auth with an external database instead of a PSK exchange there is extra time in setting up this exchange and then waiting for the external authenticator to validate it.
- DHCP / NDP. Wi-Fi assumes you more or less want to emulate a standard Ethernet + IP session but over this new fangled air connection. This is an additional delay for an additional exchange with the services responsible for this. Typical clients e.g. Windows will also perform extra address duplication checks slowing things further.
There are some extensions in more modern Wi-Fi standards for clients that want to "sleep" for long periods, immediately do some stuff, then sleep for long periods (like IoT). Particularly TWT (target wait time). These, and more, are already found in purpose built protocols like LoRa/LoRaWAN though.