Branch
Hash :
9f28054d
Author :
Date :
2024-12-15T12:52:18
Implement the //NON_IDENTICAL_DISCARD suffix from POSIX:2024. * include/iconv.h.in (ICONV_GET_DISCARD_INVALID, ICONV_SET_DISCARD_INVALID, ICONV_GET_DISCARD_NON_IDENTICAL, ICONV_SET_DISCARD_NON_IDENTICAL): New macros. * lib/converters.h (struct conv_struct): Change type of discard_ilseq to 'unsigned int'. (DISCARD_INVALID, DISCARD_UNCONVERTIBLE): New macros. * lib/iconv.c (iconv_open, iconv_open_into): Change type of discard_ilseq to 'unsigned int'. (iconvctl): Implement ICONV_GET_DISCARD_INVALID, ICONV_SET_DISCARD_INVALID, ICONV_GET_DISCARD_NON_IDENTICAL, ICONV_SET_DISCARD_NON_IDENTICAL. Change the implementation of ICONV_GET_DISCARD_ILSEQ, ICONV_SET_DISCARD_ILSEQ to test/set both bits. * lib/iconv_open1.h: Update comment. Recognize //NON_IDENTICAL_DISCARD. * lib/iconv_open2.h: Update comment. * lib/loop_unicode.h (mb_to_uc_write_replacement): Test the DISCARD_UNCONVERTIBLE bit of discard_ilseq. (unicode_loop_convert): Test the respective bit of discard_ilseq. (unicode_loop_reset): Test the DISCARD_UNCONVERTIBLE bit of discard_ilseq. * lib/loop_wchar.h (wchar_from_loop_convert, wchar_to_loop_convert): Test the DISCARD_INVALID bit of discard_ilseq. * man/iconv_open.3: Mention the //NON_IDENTICAL_DISCARD suffix. Mark as conforming to POSIX:2024. * man/iconv.3: Likewise. * man/iconv_close.3: Mark as conforming to POSIX:2024. * man/iconv.1: Likewise. * man/iconvctl.3: Document ICONV_GET_DISCARD_INVALID, ICONV_SET_DISCARD_INVALID, ICONV_GET_DISCARD_NON_IDENTICAL, ICONV_SET_DISCARD_NON_IDENTICAL. Revise the description of ICONV_GET_DISCARD_ILSEQ, ICONV_SET_DISCARD_ILSEQ. * tests/test-discard.c (test_default, test_translit, test_ignore, test_ignore_translit): Test also the ICONV_GET_DISCARD_INVALID, ICONV_GET_DISCARD_NON_IDENTICAL accessors. (test_nid, test_nid_translit, test_invd, test_invd_translit): New functions. (main): Add test cases with //NON_IDENTICAL_DISCARD suffix. * NEWS: Mention the change.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260
.\" Copyright (c) Free Software Foundation, Inc.
.\"
.\" This is free documentation; you can redistribute it and/or
.\" modify it under the terms of the GNU General Public License as
.\" published by the Free Software Foundation; either version 3 of
.\" the License, or (at your option) any later version.
.\"
.\" References consulted:
.\" GNU glibc-2 source code and manual
.\" OpenGroup's Single Unix specification http://www.UNIX-systems.org/online.html
.\"
.TH ICONV_OPEN 3 "December 15, 2024" "GNU"
.SH NAME
iconv_open \- allocate descriptor for character set conversion
.SH SYNOPSIS
.nf
.B #include <iconv.h>
.sp
.BI "iconv_t iconv_open (const char* " tocode ", const char* " fromcode );
.fi
.SH DESCRIPTION
The \fBiconv_open\fP function allocates a conversion descriptor suitable
for converting byte sequences from character encoding \fIfromcode\fP to
character encoding \fItocode\fP.
.PP
The values permitted for \fIfromcode\fP and \fItocode\fP and the supported
combinations are system dependent. For the libiconv library, the following
encodings are supported, in all combinations.
.TP
European languages
.nf
.fi
ASCII, ISO\-8859\-{1,2,3,4,5,7,9,10,13,14,15,16},
KOI8\-R, KOI8\-U, KOI8\-RU,
CP{1250,1251,1252,1253,1254,1257}, CP{850,866,1131},
Mac{Roman,CentralEurope,Iceland,Croatian,Romania},
Mac{Cyrillic,Ukraine,Greek,Turkish},
Macintosh
.TP
Semitic languages
.nf
.fi
ISO\-8859\-{6,8}, CP{1255,1256}, CP862, Mac{Hebrew,Arabic}
.TP
Japanese
.nf
.fi
EUC\-JP, SHIFT_JIS, CP932, ISO\-2022\-JP, ISO\-2022\-JP\-2, ISO\-2022\-JP\-1,
ISO-2022\-JP\-MS
.TP
Chinese
.nf
.fi
EUC\-CN, HZ, GBK, CP936, GB18030, GB18030:2022, EUC\-TW, BIG5, CP950,
BIG5\-HKSCS, BIG5\-HKSCS:2004, BIG5\-HKSCS:2001, BIG5\-HKSCS:1999,
ISO\-2022\-CN, ISO\-2022\-CN\-EXT
.TP
Korean
.nf
.fi
EUC\-KR, CP949, ISO\-2022\-KR, JOHAB
.TP
Armenian
.nf
.fi
ARMSCII\-8
.TP
Georgian
.nf
.fi
Georgian\-Academy, Georgian\-PS
.TP
Tajik
.nf
.fi
KOI8\-T
.TP
Kazakh
.nf
.fi
PT154, RK1048
.TP
Thai
.nf
.fi
TIS\-620, CP874, MacThai
.TP
Laotian
.nf
.fi
MuleLao\-1, CP1133
.TP
Vietnamese
.nf
.fi
VISCII, TCVN, CP1258
.TP
Platform specifics
.nf
.fi
HP\-ROMAN8, NEXTSTEP
.TP
Full Unicode
.nf
.fi
UTF\-8
.nf
.fi
UCS\-2, UCS\-2BE, UCS\-2LE
.nf
.fi
UCS\-4, UCS\-4BE, UCS\-4LE
.nf
.fi
UTF\-16, UTF\-16BE, UTF\-16LE
.nf
.fi
UTF\-32, UTF\-32BE, UTF\-32LE
.nf
.fi
UTF\-7
.nf
.fi
C99, JAVA
.TP
Full Unicode, in terms of \fBuint16_t\fP or \fBuint32_t\fP
(with machine dependent endianness and alignment)
.nf
.fi
UCS\-2\-INTERNAL, UCS\-4\-INTERNAL
.TP
Locale dependent, in terms of \fBchar\fP or \fBwchar_t\fP
(with machine dependent endianness and alignment, and with semantics
depending on the OS and the current LC_CTYPE locale facet)
.nf
.fi
char, wchar_t
.PP
When configured with the option \fB\-\-enable\-extra\-encodings\fP, it also
provides support for a few extra encodings:
.TP
European languages
.nf
CP{437,737,775,852,853,855,857,858,860,861,863,865,869,1125}
.fi
.TP
Semitic languages
.nf
.fi
CP864
.TP
Japanese
.nf
.fi
EUC\-JISX0213, Shift_JISX0213, ISO\-2022\-JP\-3
.TP
Chinese
.nf
.fi
BIG5\-2003 (experimental)
.TP
Turkmen
.nf
.fi
TDS565
.TP
Platform specifics
.nf
.fi
ATARIST, RISCOS\-LATIN1
.PP
.TP
EBCDIC compatible (not ASCII compatible, very rarely used)
.nf
.fi
European languages:
.nf
.fi
IBM-{037,273,277,278,280,282,284,285,297,423,500,870,871,875,880},
IBM-{905,924,1025,1026,1047,1112,1122,1123,1140,1141,1142,1143},
IBM-{1144,1145,1146,1147,1148,1149,1153,1154,1155,1156,1157,1158},
IBM-{1165,1166,4971}
.nf
.fi
Semitic languages:
.nf
.fi
IBM-{424,425,12712,16804}
.nf
.fi
Persian:
.nf
.fi
IBM-1097
.nf
.fi
Thai:
.nf
.fi
IBM-{838,1160}
.nf
.fi
Laotian:
.nf
.fi
IBM-1132
.nf
.fi
Vietnamese:
.nf
.fi
IBM-{1130,1164}
.nf
.fi
Indic languages:
.nf
.fi
IBM-1137
.PP
The empty encoding name "" is equivalent to "char": it denotes the
locale dependent character encoding.
.PP
When the string "//TRANSLIT" is appended to \fItocode\fP, transliteration
is activated. This means that when a character cannot be represented in the
target character set, it can be approximated through one or several characters
that look similar to the original character.
.PP
When the string "//IGNORE" is appended to \fItocode\fP,
invalid multibyte sequences in the input and
characters that cannot be represented in the target character set
will be silently discarded.
.PP
When the string "//NON_IDENTICAL_DISCARD" is appended to \fItocode\fP,
characters that cannot be represented in the target character set
will be silently discarded.
.PP
The resulting conversion descriptor can be used with \fBiconv\fP any number
of times. It remains valid until deallocated using \fBiconv_close\fP.
.PP
A conversion descriptor contains a conversion state. After creation using
\fBiconv_open\fP, the state is in the initial state. Using \fBiconv\fP
modifies the descriptor's conversion state. (This implies that a conversion
descriptor can not be used in multiple threads simultaneously.) To bring the
state back to the initial state, use \fBiconv\fP with NULL as \fIinbuf\fP
argument.
.SH "RETURN VALUE"
The \fBiconv_open\fP function returns a freshly allocated conversion
descriptor. In case of error, it sets \fBerrno\fP and returns (iconv_t)(\-1).
.SH ERRORS
The following error can occur, among others:
.TP
.B EINVAL
The conversion from \fIfromcode\fP to \fItocode\fP is not supported by the
implementation.
.SH "CONFORMING TO"
POSIX:2024
.SH "SEE ALSO"
.BR iconv (3)
.BR iconvctl (3)
.BR iconv_close (3)