Force strings to UTF-8 from any encoding

Solution 1:

Ruby 1.9

"Forcing" an encoding is easy, however it won't convert the characters just change the encoding:

str = str.force_encoding('UTF-8')

str.encoding.name # => 'UTF-8'

If you want to perform a conversion, use encode:

begin
  str.encode("UTF-8")
rescue Encoding::UndefinedConversionError
  # ...
end

I would definitely read the following post for more information:
http://graysoftinc.com/character-encodings/ruby-19s-string

Solution 2:

This will ensure you have the correct encoding and won't error out because it replaces any invalid or undefined character with a blank string.

This will ensure no matter what, that you have a valid UTF-8 string

str.encode(Encoding.find('UTF-8'), {invalid: :replace, undef: :replace, replace: ''})

Solution 3:

Iconv

require 'iconv'
i = Iconv.new('UTF-8','LATIN1')
a_with_hat = i.iconv("\xc2")

Summary: the iconv gem does all the work of converting encodings. Make sure it's installed with:

gem install iconv

Now, you need to know what encoding your string is currently in as Ruby 1.8 treats Strings as an array of bytes (with no intrinsic encoding.) For example, say your string was in latin1 and you wanted to convert it to utf-8

require 'iconv'

string_in_utf8_encoding = Iconv.conv("UTF8", "LATIN1", string_in_latin1_encoding)

Solution 4:

Only this solution worked for me:

string.encode('UTF-8', 'binary', invalid: :replace, undef: :replace, replace: '')

Note the binary argument.