"bit" (the english word) is pronounced exactly like "bite" (the french word), \bit\. The funny thing is that we often use "octet" to refer to bytes (which are pronounced /baɪt/ so no problems here) but we have no alternative for bit.
I don't really know what else to say other that I'm French, went through university in France and we had this confusion.