This argument is called "Bertlmann's Socks"

https://en.wikipedia.org/wiki/Reinhold_Bertlmann#Bertlmann%E...

The other replies explain why it's wrong, but here's a link to Bell's refutation for good measure

http://cds.cern.ch/record/142461/files/198009299.pdf