During an election, you give your proxy to anybody who is willing to be a representative. Each representative votes in Congress with a vote weighted by the number of proxies they received. The top 500 or so representatives get to sit in Congress. Those who didn't reach the threshold must pass their proxy on to another.
Nobody is no longer "represented" by somebody they didn't vote for. You can be represented by somebody who much more closely matches your interest. "Districts" will still be represented because many people will choose a local candidate, but everybody is free to choose someone who will represent them better if no local candidate does.
This won't eliminate political parties, but it will significantly weaken them as strong representatives will no longer be as dependent on the party machinery.