Many webservers have the ability to do GeoIP lookups from a database. I did a setup before that used a GeoIP database ( country level in your case ) from MaxMind which I used in conjunction with Nginx geo module ( http://wiki.nginx.org/NginxHttpGeoModule ) to direct people from certain countries to different domains.

In the case of that setup, I needed a script that converted the MaxMind textual format over to a format that nginx could understand and then a bit of nginx code to do redirects based on the country that came up.

You need to be careful with something like that though. You want to make sure you aren't redirecting google's bot etc away from the site it is trying to get to.

In terms of your .com/.ca search question. They are going to get your .com. You could setup google campaigns that target users from specific countries so that your .ca appears above the organic listing...