Edit File: Encode::Locale.3pm
.\" Automatically generated by Pod::Man 2.22 (Pod::Simple 3.13) .\" .\" Standard preamble: .\" ======================================================================== .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" Set up some character translations and predefined strings. \*(-- will .\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left .\" double quote, and \*(R" will give a right double quote. \*(C+ will .\" give a nicer C++. Capital omega is used to do unbreakable dashes and .\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff, .\" nothing in troff, for use with C<>. .tr \(*W- .ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' .ie n \{\ . ds -- \(*W- . ds PI pi . if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch . if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch . ds L" "" . ds R" "" . ds C` "" . ds C' "" 'br\} .el\{\ . ds -- \|\(em\| . ds PI \(*p . ds L" `` . ds R" '' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is turned on, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .ie \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . nr % 0 . rr F .\} .el \{\ . de IX .. .\} .\" .\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2). .\" Fear. Run. Save yourself. No user-serviceable parts. . \" fudge factors for nroff and troff .if n \{\ . ds #H 0 . ds #V .8m . ds #F .3m . ds #[ \f1 . ds #] \fP .\} .if t \{\ . ds #H ((1u-(\\\\n(.fu%2u))*.13m) . ds #V .6m . ds #F 0 . ds #[ \& . ds #] \& .\} . \" simple accents for nroff and troff .if n \{\ . ds ' \& . ds ` \& . ds ^ \& . ds , \& . ds ~ ~ . ds / .\} .if t \{\ . ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u" . ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u' . ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u' . ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u' . ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u' . ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u' .\} . \" troff and (daisy-wheel) nroff accents .ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V' .ds 8 \h'\*(#H'\(*b\h'-\*(#H' .ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#] .ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H' .ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u' .ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#] .ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#] .ds ae a\h'-(\w'a'u*4/10)'e .ds Ae A\h'-(\w'A'u*4/10)'E . \" corrections for vroff .if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u' .if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u' . \" for low resolution devices (crt and lpr) .if \n(.H>23 .if \n(.V>19 \ \{\ . ds : e . ds 8 ss . ds o a . ds d- d\h'-1'\(ga . ds D- D\h'-1'\(hy . ds th \o'bp' . ds Th \o'LP' . ds ae ae . ds Ae AE .\} .rm #[ #] #H #V #F C .\" ======================================================================== .\" .IX Title "Encode::Locale 3" .TH Encode::Locale 3 "2012-02-11" "perl v5.10.1" "User Contributed Perl Documentation" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH "NAME" Encode::Locale \- Determine the locale encoding .SH "SYNOPSIS" .IX Header "SYNOPSIS" .Vb 2 \& use Encode::Locale; \& use Encode; \& \& $string = decode(locale => $bytes); \& $bytes = encode(locale => $string); \& \& if (\-t) { \& binmode(STDIN, ":encoding(console_in)"); \& binmode(STDOUT, ":encoding(console_out)"); \& binmode(STDERR, ":encoding(console_out)"); \& } \& \& # Processing file names passed in as arguments \& my $uni_filename = decode(locale => $ARGV[0]); \& open(my $fh, "<", encode(locale_fs => $uni_filename)) \& || die "Can\*(Aqt open \*(Aq$uni_filename\*(Aq: $!"; \& binmode($fh, ":encoding(locale)"); \& ... .Ve .SH "DESCRIPTION" .IX Header "DESCRIPTION" In many applications it's wise to let Perl use Unicode for the strings it processes. Most of the interfaces Perl has to the outside world are still byte based. Programs therefore need to decode byte strings that enter the program from the outside and encode them again on the way out. .PP The \s-1POSIX\s0 locale system is used to specify both the language conventions requested by the user and the preferred character set to consume and output. The \f(CW\*(C`Encode::Locale\*(C'\fR module looks up the charset and encoding (called a \s-1CODESET\s0 in the locale jargon) and arranges for the Encode module to know this encoding under the name \*(L"locale\*(R". It means bytes obtained from the environment can be converted to Unicode strings by calling \f(CW\*(C`Encode::encode(locale => $bytes)\*(C'\fR and converted back again with \f(CW\*(C`Encode::decode(locale => $string)\*(C'\fR. .PP Where file systems interfaces pass file names in and out of the program we also need care. The trend is for operating systems to use a fixed file encoding that don't actually depend on the locale; and this module determines the most appropriate encoding for file names. The Encode module will know this encoding under the name \*(L"locale_fs\*(R". For traditional Unix systems this will be an alias to the same encoding as \*(L"locale\*(R". .PP For programs running in a terminal window (called a \*(L"Console\*(R" on some systems) the \*(L"locale\*(R" encoding is usually a good choice for what to expect as input and output. Some systems allows us to query the encoding set for the terminal and \&\f(CW\*(C`Encode::Locale\*(C'\fR will do that if available and make these encodings known under the \f(CW\*(C`Encode\*(C'\fR aliases \*(L"console_in\*(R" and \*(L"console_out\*(R". For systems where we can't determine the terminal encoding these will be aliased as the same encoding as \*(L"locale\*(R". The advice is to use \*(L"console_in\*(R" for input known to come from the terminal and \*(L"console_out\*(R" for output known to go from the terminal. .PP In addition to arranging for various Encode aliases the following functions and variables are provided: .IP "decode_argv( )" 4 .IX Item "decode_argv( )" .PD 0 .IP "decode_argv( Encode::FB_CROAK )" 4 .IX Item "decode_argv( Encode::FB_CROAK )" .PD This will decode the command line arguments to perl (the \f(CW@ARGV\fR array) in-place. .Sp The function will by default replace characters that can't be decoded by \&\*(L"\ex{\s-1FFFD\s0}\*(R", the Unicode replacement character. .Sp Any argument provided is passed as \s-1CHECK\s0 to underlying \fIEncode::decode()\fR call. Pass the value \f(CW\*(C`Encode::FB_CROAK\*(C'\fR to have the decoding croak if not all the command line arguments can be decoded. See \*(L"Handling Malformed Data\*(R" in Encode for details on other options for \s-1CHECK\s0. .ie n .IP "env( $uni_key )" 4 .el .IP "env( \f(CW$uni_key\fR )" 4 .IX Item "env( $uni_key )" .PD 0 .ie n .IP "env( $uni_key => $uni_value )" 4 .el .IP "env( \f(CW$uni_key\fR => \f(CW$uni_value\fR )" 4 .IX Item "env( $uni_key => $uni_value )" .PD Interface to get/set environment variables. Returns the current value as a Unicode string. The \f(CW$uni_key\fR and \f(CW$uni_value\fR arguments are expected to be Unicode strings as well. Passing \f(CW\*(C`undef\*(C'\fR as \f(CW$uni_value\fR deletes the environment variable named \f(CW$uni_key\fR. .Sp The returned value will have the characters that can't be decoded replaced by \&\*(L"\ex{\s-1FFFD\s0}\*(R", the Unicode replacement character. .Sp There is no interface to request alternative \s-1CHECK\s0 behavior as for \&\fIdecode_argv()\fR. If you need that you need to call encode/decode yourself. For example: .Sp .Vb 2 \& my $key = Encode::encode(locale => $uni_key, Encode::FB_CROAK); \& my $uni_value = Encode::decode(locale => $ENV{$key}, Encode::FB_CROAK); .Ve .IP "reinit( )" 4 .IX Item "reinit( )" .PD 0 .ie n .IP "reinit( $encoding )" 4 .el .IP "reinit( \f(CW$encoding\fR )" 4 .IX Item "reinit( $encoding )" .PD Reinitialize the encodings from the locale. You want to call this function if you changed anything in the environment that might influence the locale. .Sp This function will croak if the determined encoding isn't recognized by the Encode module. .Sp With argument force \f(CW$ENCODING_\fR... variables to set to the given value. .ie n .IP "$ENCODING_LOCALE" 4 .el .IP "\f(CW$ENCODING_LOCALE\fR" 4 .IX Item "$ENCODING_LOCALE" The encoding name determined to be suitable for the current locale. Encode know this encoding as \*(L"locale\*(R". .ie n .IP "$ENCODING_LOCALE_FS" 4 .el .IP "\f(CW$ENCODING_LOCALE_FS\fR" 4 .IX Item "$ENCODING_LOCALE_FS" The encoding name determined to be suiteable for file system interfaces involving file names. Encode know this encoding as \*(L"locale_fs\*(R". .ie n .IP "$ENCODING_CONSOLE_IN" 4 .el .IP "\f(CW$ENCODING_CONSOLE_IN\fR" 4 .IX Item "$ENCODING_CONSOLE_IN" .PD 0 .ie n .IP "$ENCODING_CONSOLE_OUT" 4 .el .IP "\f(CW$ENCODING_CONSOLE_OUT\fR" 4 .IX Item "$ENCODING_CONSOLE_OUT" .PD The encodings to be used for reading and writing output to the a console. Encode know these encodings as \*(L"console_in\*(R" and \*(L"console_out\*(R". .SH "NOTES" .IX Header "NOTES" This table summarizes the mapping of the encodings set up by the \f(CW\*(C`Encode::Locale\*(C'\fR module: .PP .Vb 7 \& Encode | | | \& Alias | Windows | Mac OS X | POSIX \& \-\-\-\-\-\-\-\-\-\-\-\-+\-\-\-\-\-\-\-\-\-+\-\-\-\-\-\-\-\-\-\-\-\-\-\-+\-\-\-\-\-\-\-\-\-\-\-\- \& locale | ANSI | nl_langinfo | nl_langinfo \& locale_fs | ANSI | UTF\-8 | nl_langinfo \& console_in | OEM | nl_langinfo | nl_langinfo \& console_out | OEM | nl_langinfo | nl_langinfo .Ve .SS "Windows" .IX Subsection "Windows" Windows has basically 2 sets of APIs. A wide \s-1API\s0 (based on passing \s-1UTF\-16\s0 strings) and a byte based \s-1API\s0 based a character set called \s-1ANSI\s0. The regular Perl interfaces to the \s-1OS\s0 currently only uses the \s-1ANSI\s0 APIs. Unfortunately \s-1ANSI\s0 is not a single character set. .PP The encoding that corresponds to \s-1ANSI\s0 varies between different editions of Windows. For many western editions of Windows \s-1ANSI\s0 corresponds to \s-1CP\-1252\s0 which is a character set similar to \s-1ISO\-8859\-1\s0. Conceptually the \s-1ANSI\s0 character set is a similar concept to the \s-1POSIX\s0 locale \s-1CODESET\s0 so this module figures out what the \s-1ANSI\s0 code page is and make this available as \&\f(CW$ENCODING_LOCALE\fR and the \*(L"locale\*(R" Encoding alias. .PP Windows systems also operate with another byte based character set. It's called the \s-1OEM\s0 code page. This is the encoding that the Console takes as input and output. It's common for the \s-1OEM\s0 code page to differ from the \s-1ANSI\s0 code page. .SS "Mac \s-1OS\s0 X" .IX Subsection "Mac OS X" On Mac \s-1OS\s0 X the file system encoding is always \s-1UTF\-8\s0 while the locale can otherwise be set up as normal for \s-1POSIX\s0 systems. .PP File names on Mac \s-1OS\s0 X will at the OS-level be converted to NFD-form. A file created by passing a NFC-filename will come in NFD-form from \fIreaddir()\fR. See Unicode::Normalize for details of \s-1NFD/NFC\s0. .PP Actually, Apple does not follow the Unicode \s-1NFD\s0 standard since not all character ranges are decomposed. The claim is that this avoids problems with round trip conversions from old Mac text encodings. See Encode::UTF8Mac for details. .SS "\s-1POSIX\s0 (Linux and other Unixes)" .IX Subsection "POSIX (Linux and other Unixes)" File systems might vary in what encoding is to be used for filenames. Since this module has no way to actually figure out what the is correct it goes with the best guess which is to assume filenames are encoding according to the current locale. Users are advised to always specify \s-1UTF\-8\s0 as the locale charset. .SH "SEE ALSO" .IX Header "SEE ALSO" I18N::Langinfo, Encode .SH "AUTHOR" .IX Header "AUTHOR" Copyright 2010 Gisle Aas <gisle@aas.no>. .PP This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
Back to File Manager