In Spain, where the system is similar to what you described (staff for the day is chosen at random from all citizens in that voting district, and similar to jury duty), some professionals (doctors, nurses, air controllers, etc.) can decline the request if they will be on watch, and there are assigned replacements for them.
A non-critical worker cannot refuse but:
a) Chosen citizens are paid, even if not much (60€ I think).
b) Chosen citizens have the following Monday morning free, should they need to travel back to their job site, etc.
c) Chosen citizens have strong legal protections should their boss try anything.
In practice, when an employee is chosen for election duty the boss just deals with it as best as they can, since it's a case where judges and police are super strict: messing with election duty will get you heavy fines and/or jail time. So no-one is going to fire you or retaliate if you are assigned election duty.