Personally, I think it's a necessary balance to Prosecutorial discretion however.
And probably would've been beaten to death or hanged in public for the wrong form of dereliction of that duty. ;- )
We do have records of the arguments that were made for the elector system mandated by the Constitution, and they demonstrate that your "precise idea" is a fabrication.
Here is some reading for you: https://en.wikipedia.org/wiki/Federalist_No._68
It had almost nothing to do with the practicality (or lack thereof) of the popular vote. The debates focused upon balancing the right of the people with the right of the states, i.e., the ability of states with large populations to make the votes of smaller states irrelevant. Hamilton also focused on the need for trusted electors so that the people both had a say, and trusted individuals could avoid rash populism. I.e., the value of a representative form of election. The transient nature of the college was also important. The electors would have one task, and then be dismissed. They could not be people with a vested interest, such as members of congress, governors, etc. This also made them less likely to be influenced by foreign powers, as they would not be electors long enough to be so influenced.
Federalist 68 proposes a mechanism we have not used since before the Civil War. The ballot would have been write-in only, and the name you were writing in was a person who lived in your own state, not a presidential candidate.
but to men chosen by the people for the special purpose
The choice of SEVERAL, to form an intermediate body of electors, will be much less apt to convulse the community with any extraordinary or violent movements, than the choice of ONE
Unfortunately, the Founders appear to not have anticipated the wholesale corruption the party system would employ to sabotage both common sense and deliberate design, by making laws at state level to explicitly make Electors not from the general population but exclusively party loyalists: people who have sworn money and allegiance to their respective party. Who and what you are really voting for when you tick a box for a candidate, is not the candidate, but the candidate's party and that party system, and the state laws say the popular vote result is to cause that party's pre-selected loyalists to become Electors.
What we have isn't anything like Federalist 68.
It's a pattern that happens all over the government. The masses are the first filter, a second pass is done by a smaller (generally more recognized) few, and then reconcile until everyone is satisfied.
But you are not wrong in your basic argument. The corruption of the college by the party system is a travesty. We may as well have a parliament / prime minister at this point. It would be preferable to the current system, IMO, at least in being more honest.