Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (11)
Destaque
Destaque (8)
Semelhante a Guide to String Encodings
Semelhante a Guide to String Encodings (20)
Último
Último (20)
Guide to String Encodings
- 9. BUT WHAT ABOUT THE 8TH BIT?
- 10. UH OH!
- 11. BUT MY LANGUAGE HAS 2000 CHARACTERS
- 12. WELCOME TO MULTIBYTE ENCODINGS
- 13. UNICODE
- 14. CHARACTER SETS vs ENCODINGS
- 15. CHARACTER SETS U+0061 = 97 = a
- 16. ENCODINGS " abc ".encode("UTF-32BE") "x00x00x00ax00x00x00bx00x00x00c"
- 17. UTF-8
- 18. RUBY 1.8 STRINGS ARE JUST BYTE ARRAYS!
- 19. $KCODE=’u’
- 20. RUBY 1.9
- 21. M17N
- 22. STRING EXAMPLES e = "é" e.encoding.name # => UTF-8 e.size # => 1 e.bytesize # => 2
- 23. FORCING ENCODINGS x.encoding.name #=> ISO-8859-1 x.bytesize #=> 6 x.valid_encoding? #=> true x.force_encoding("UTF-8") x.encoding.name #=> UTF-8 x.bytesize #=> 6 x.valid_encoding? #=> false x =~ /x/ #=> invalid byte sequence in UTF-8
- 24. TRANSCODING x.encoding.name #=> ISO-8859-1 x.bytesize #=> 6 x.valid_encoding? #=> true x.encode!("UTF-8") x.encoding.name #=> UTF-8 x.bytesize #=> 12 x.valid_encoding? #=> true
- 25. COMPATIBILITY if Encoding.compatible?(ascii_string, utf8_string) new_string = ascii_string + utf8_string new_string.encoding.name #=> UTF-8 end
- 26. ITERATION x.bytes x.each_byte {|b| puts b } x.codepoints x.each_codepoint {|c| puts c } x.chars x.each_char {|c| puts c } x.lines x.each_line {|l| puts l }
- 27. SOURCE ENCODING # encoding: UTF-8
- 28. INTERNAL ENCODING > ruby -E :UTF-8 Encoding.default_internal = 'UTF-8'
- 29. EXTERNAL ENCODING > ruby -E UTF-8: Encoding.default_external = 'UTF-8' File.open("file.txt", "w:UTF-8")
- 30. WHAT DO? USE MAGIC COMMENTS DECLARE IO ENCODINGS CONVERT BEFORE COMPARISONS UNICODE-UTILS GEM
- 31. C’EST FINI
- 32. We’re hiring!
- 33. Thanks! http://blog.grayproductions.net/ articles/understanding_m17n
Notas do Editor
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- 128 Chars\nEnglish upper, lower symbols + some control charaters\n
- 128 chars = 7 bits.\n1 byte = 8 bits.\nWe have a whole 128 other characters to play with.\n
- Everyone wants a different 128 chars.\nEncodings were born.\nLatin 1 is one of the most popular, it uses the other 128 chars mostly for accented alphabets.\n
- It would be simple if we all spoke english!\njapanese, chinese, etc.\n
- The only way we can deal with this is to comprise characters of more than 1 byte.\nThis is a big issue for programming as we now have to be very mindful not to split in the middle of a char.\n
- Ideally we would only ever have 1 encoding to deal with.\nThis isn’t the case but unicode is as close as it gets.\n
- There different\n\n
- Character sets are just internal mappings from numbers to strings\n\nhex - int - char\n
- here we are writing a lot of null chars\n
- Unicode encoding that is 100% compat with US-ASCII\nuses variable length characters to maintain compat\nBest current solution for compat, size, support\n
- as we’ve seen multibyte sets are common and useful.\nRuby doesn’t care about encodings, it just displays characters based on a mapping.\nreverse(), split(), size() all break multibyte strings very easily.\nBut ruby doesn’t check so you’ll never really know!\n
- Ruby 1.8 supports utf-8 right?\nkinda\nunicode support extends to understanding boundaries so magically, reverse(), split(), match() all work.\nUnfortunately it’s very basic support so transforms like upcase() will bite you in the arse.\nAlso kcode is global so there is no way to deal with more than 1 encoding at a time.\n
- Ruby 1.9 brings full string encoding support for over 80 encodings.\nString objects now reference both raw bytes and an Encoding object.\n\n
- \n
- a string is aware of it’s encoding, character length and byte length\n
- force_encoding will retag a string with a different encoding, however the actual bytes are not modified.\nIf the string is tagged with an invalid encoding then it can cause ArgumentException’s when trying to manipulate the strings characters.\n
- calling encode() actually modifies the underlying string.\n
- \n
- String.each is gone in 1.9\nreplaced with explicit each methods and Enumerator functions\n
- \n
- \n
- \n
- http://unicode-utils.rubyforge.org/ for locale management\n
- \n
- \n
- \n