At 3:39p, unplug your router and leave it unplugged for an hour, then plug it back in. If the outage then moves to 4:40p every day, it's a 24 hour IP address lease expiring.
Are you talking about IP leases from the router to the local network or IP leases from the ISP to the router?
likely router -> ISP - if it was the devices, they could show up as not being connected to the network. also weird that all devices' leases would expire at the same time