with open('some latin-1 file', 'rb) as f:
text = f.read().decode('latin-1')
with open('some utf8 file', 'wb') as f:
f.write(text.encode('utf-8'))
If you use bytes as a string you are Doing It Wrong.
If you use bytes as a string you are Doing It Wrong.
I do that operation on a file I get from an API. I know for a fact that the encoding I'm receiving is latin-1.
I run exactly that operation on the file that you wrote out in code.
When I try to read that file back in as UTF-8, I get encoding errors. That does not make for "super good." That makes me want to scream.
I do not have this problem when I use Python 2.7.x
To be perfectly clear: bytes (b'') is not a string. Again: bytes is NOT a string. It is an array of octets, aka bytes, aka unsigned 8 bit integers. NOT characters. NOT a string.
If you are dealing with bytes that are encoded representations of a string, then you have to know what encoding they use to decode them and treat them as strings.