You don't need to link your vote to aadhaar. Vote can be anonymized, but polling booths can still check and identify voters using aadhaar and easily avoid duplicate votes & rigging.
Loading the polling unit with aadhar database can be one approach. Very few know (relative to the number of personnel involved in the polling process), as to which polling unit will go to which polling station. Trying to load constituency specific database into polling unit will increase the surface area of the people who have to know which unit goes where. Loading the aadhar database of full constituency into polling units might not be technically feasible.