TextEdit removes Byte-Order-Mark (BOM) from Unicode/UTF files. How to fix?

Solution 1:

TextEdit → Preferences → Open and Save

Set “Plain Text File Encoding” to “Unicode (UTF-8)” for both opening and saving files.

If that doesn’t fix it already, open Terminal.app and enter the following:

echo "0x08000100:0" > ~/.CFUserTextEncoding

Then logout and login again.

This fixes the ancient UTF-8 bug in OS X that also manifest in QuickLook.

To fix it for just a single file rather than for all files (I don’t see why you would, though), just write the extended attribute that TextEdit uses:

xattr -w com.apple.TextEncoding 'UTF-8;134217984' filename.txt

Hope this helps!

Solution 2:

This isn't the main solution you're looking for, but TextWrangler is free, handles UTF-8 (and many other encodings) very well, and doesn't have too steep a learning curve, particularly if you're just using functionality similar to TextEdit. I know you'd like to fix TextEdit, and I hope you get an answer explaining how to do that, but in the absence of a primary solution, this should be a good backup plan.

Solution 3:

I think using TextWrangler to always save as "UTF-8 with BOM" is the best way to make sure all other systems will see your files as UTF-8. Another option would be to use UTF-16 instead, but that is unusual and could confuse some users.