|
9f28054d
|
2024-12-15T12:52:18
|
|
Implement the //NON_IDENTICAL_DISCARD suffix from POSIX:2024.
* include/iconv.h.in (ICONV_GET_DISCARD_INVALID,
ICONV_SET_DISCARD_INVALID, ICONV_GET_DISCARD_NON_IDENTICAL,
ICONV_SET_DISCARD_NON_IDENTICAL): New macros.
* lib/converters.h (struct conv_struct): Change type of discard_ilseq to
'unsigned int'.
(DISCARD_INVALID, DISCARD_UNCONVERTIBLE): New macros.
* lib/iconv.c (iconv_open, iconv_open_into): Change type of
discard_ilseq to 'unsigned int'.
(iconvctl): Implement ICONV_GET_DISCARD_INVALID,
ICONV_SET_DISCARD_INVALID, ICONV_GET_DISCARD_NON_IDENTICAL,
ICONV_SET_DISCARD_NON_IDENTICAL. Change the implementation of
ICONV_GET_DISCARD_ILSEQ, ICONV_SET_DISCARD_ILSEQ to test/set both bits.
* lib/iconv_open1.h: Update comment. Recognize //NON_IDENTICAL_DISCARD.
* lib/iconv_open2.h: Update comment.
* lib/loop_unicode.h (mb_to_uc_write_replacement): Test the
DISCARD_UNCONVERTIBLE bit of discard_ilseq.
(unicode_loop_convert): Test the respective bit of discard_ilseq.
(unicode_loop_reset): Test the DISCARD_UNCONVERTIBLE bit of
discard_ilseq.
* lib/loop_wchar.h (wchar_from_loop_convert, wchar_to_loop_convert):
Test the DISCARD_INVALID bit of discard_ilseq.
* man/iconv_open.3: Mention the //NON_IDENTICAL_DISCARD suffix. Mark as
conforming to POSIX:2024.
* man/iconv.3: Likewise.
* man/iconv_close.3: Mark as conforming to POSIX:2024.
* man/iconv.1: Likewise.
* man/iconvctl.3: Document ICONV_GET_DISCARD_INVALID,
ICONV_SET_DISCARD_INVALID, ICONV_GET_DISCARD_NON_IDENTICAL,
ICONV_SET_DISCARD_NON_IDENTICAL. Revise the description of
ICONV_GET_DISCARD_ILSEQ, ICONV_SET_DISCARD_ILSEQ.
* tests/test-discard.c (test_default, test_translit, test_ignore,
test_ignore_translit): Test also the ICONV_GET_DISCARD_INVALID,
ICONV_GET_DISCARD_NON_IDENTICAL accessors.
(test_nid, test_nid_translit, test_invd, test_invd_translit): New
functions.
(main): Add test cases with //NON_IDENTICAL_DISCARD suffix.
* NEWS: Mention the change.
|
|
|
ef2f457f
|
2024-12-15T10:37:13
|
|
Support multiple suffixes in iconv_open, like glibc does.
* lib/iconv_open1.h: Use a loop when looking for the common suffixes.
* tests/test-discard.c: New file.
* tests/Makefile.in (check): Run test-discard.
(test-discard, test-discard.@OBJEXT@): New targets.
(clean): Remove test-discard.
(SOURCE_FILES): Add test-discard.c.
* NEWS: Mention the change.
|
|
|
a4c1470b
|
2024-12-13T23:55:13
|
|
Distinguish byte-order state and shift-state.
Reported by Tomas Kalibera <tomas.kalibera@gmail.com> in
<https://lists.gnu.org/archive/html/bug-gnu-libiconv/2024-12/msg00000.html>.
* lib/converters.h (struct conv_struct): Add field 'ibyteorder'.
* lib/iconv_open2.h: Initialize the ibyteorder field.
* lib/ucs2.h (ucs2_mbtowc): Use the ibyteorder field instead of the
istate field.
* lib/ucs4.h (ucs4_mbtowc): Likewise.
* lib/utf16.h (utf16_mbtowc): Likewise.
* lib/utf32.h (utf32_mbtowc): Likewise.
* tests/test-bom-state.c: New file.
* tests/Makefile.in (check): Run test-bom-state.
(test-bom-state, test-bom-state.@OBJEXT@): New targets.
(clean): Remove test-bom-state.
(SOURCE_FILES): Add test-bom-state.c.
* NEWS: Mention the change.
|
|
|
5448df58
|
2023-05-29T13:59:43
|
|
GB18030: Help transitioning away from PUA code points.
* lib/gb18030ext.h (gb18030_2005_ext_wctomb): Remove function.
(gb18030ext_wctomb): Renamed from gb18030_2022_ext_wctomb.
* lib/gb18030uni.h (gb18030_2005_uni_wctomb): Map 6 Ext-B code points to
4-bytes sequences.
(gb18030_2022_uni_wctomb): Small refactoring.
* lib/gb18030_2005.h (gb18030_2005_pua2charset): Map 6 PUA code points
to 4-bytes sequences instead of 2-bytes sequences.
(gb18030_2005_wctomb): Update accordingly. Invoke gb18030ext_wctomb
instead of gb18030_2005_ext_wctomb.
* lib/gb18030_2022.h (gb18030_2022_wctomb): Invoke gb18030ext_wctomb
instead of gb18030_2022_ext_wctomb.
* tests/GB18030-2005.IRREVERSIBLE.TXT: Update the inverse mappings of 6
Ext-B code points and 6 PUA code points.
* NEWS: Mention it.
|
|
|
6549d20c
|
2023-05-21T00:29:37
|
|
Implement GB18030 version 2022.
* lib/encodings.def (GB18030): Add alias GB18030:2005.
(GB18030:2022): New encoding.
* lib/gb18030ext.h (gb18030_2005_ext_2uni_pagefe): Renamed from
gb18030ext_2uni_pagefe.
(gb18030_2022_ext_2uni_pagefe): New array.
(gb18030_2005_ext_mbtowc): Renamed from gb18030ext_mbtowc.
(gb18030_2022_ext_mbtowc): New function.
(gb18030_2005_ext_wctomb): Renamed from gb18030ext_wctomb.
(gb18030_2022_ext_wctomb): New function.
* lib/gb18030uni.h (gb18030_2022_charset2uni_pua1,
gb18030_2022_charset2uni_pua2): New arrays.
(gb18030_2005_uni_mbtowc): Renamed from gb18030uni_mbtowc.
(gb18030_2022_uni_mbtowc): New function.
(gb18030_2022_uni2charset_pua1, gb18030_2022_uni2charset_pua2): New
arrays.
(gb18030_2005_uni_wctomb): Renamed from gb18030uni_wctomb.
(gb18030_2022_uni_wctomb): New function.
* lib/gb18030_2005.h: Renamed from lib/gb18030.h. Update comments.
(gb18030_2005_mbtowc): Renamed from gb18030_mbtowc.
(gb18030_2005_pua2charset): Renamed from gb18030_pua2charset.
(gb18030_2005_wctomb): Renamed from gb18030_wctomb.
* lib/gb18030_2022.h: New file, based on lib/gb18030_2005.h.
* lib/converters.h: Don't include gb18030.h. Include gb18030_2005.h,
gb18030_2022.h.
* lib/Makefile.in (SOURCE_FILES): Remove gb18030.h. Add gb18030_2005.h,
gb18030_2022.h.
* tests/GB18030-2005-BMP.TXT: Renamed from tests/GB18030-BMP.TXT.
* tests/GB18030-2005.IRREVERSIBLE.TXT: Renamed from
tests/GB18030.IRREVERSIBLE.TXT.
* tests/GB18030-2022-BMP.TXT: New file.
* tests/Makefile.in (check): Test GB18030:2005 instead of GB18030. Also
test GB18030:2022.
(clean): Don't remove GB18030.TXT. Instead, remove GB18030-2005.TXT and
GB18030-2022.TXT.
(SOURCE_FILES): Update. Add GB18030-2022-BMP.TXT.
* README: Mention the new encoding.
* man/iconv_open.3: Likewise.
* NEWS: Likewise.
|
|
|
af375a34
|
2023-05-12T22:50:18
|
|
Integrate the last change.
* src/Makefile.in (SOURCE_FILES): Add zos-tag.h.
* tests/Makefile.in (SOURCE_FILES): Add check-tag.
* NEWS: Mention the new functionality.
|
|
153d4079
|
2023-05-12T22:41:08
|
|
On z/OS, set a charset tag on iconv's output file.
For the concept of charset tags as external metadata on z/OS files, see
<https://lists.gnu.org/archive/html/bug-gnu-libiconv/2023-04/msg00021.html>.
* src/zos-tag.h: New file.
* src/iconv.c: Include zos-tag.h.
(convert): Add a 'tocode' parameter. On z/OS, turn off auto-conversion
and tag the output file.
(main): Update callers.
* tests/check-ebcdic: On z/OS, make all test files initially untagged.
* tests/check-tag: New file.
* tests/Makefile.in (check): Pass the host_os to check-ebcdic. Invoke
check-tag.
|
|
|
19b6af5e
|
2023-04-03T04:12:01
|
|
Allow overriding the newline conversion for EBCDIC encodings.
Reported by Mike Fulton <mikefultonpersonal@gmail.com> in
<https://lists.gnu.org/archive/html/bug-gnu-libiconv/2023-04/msg00009.html>.
* include/iconv.h.in (ICONV_SURFACE_NONE,
ICONV_SURFACE_EBCDIC_ZOS_UNIX): New macros.
(ICONV_GET_FROM_SURFACE, ICONV_SET_FROM_SURFACE, ICONV_GET_TO_SURFACE,
ICONV_SET_TO_SURFACE): New macros.
* lib/converters.h (struct conv_struct): Add the fields isurface,
osurface.
(swap_x15_x25): New macro.
* lib/iconv.c (iconv_open, iconv_open_into): Add local variables
from_surface, to_surface.
(ALL_SURFACES): New macro.
(iconvctl): Adjust ICONV_TRIVIALP implementation. Implement the
ICONV_{GET,SET}_{FROM,TO}_SURFACE requests.
* lib/iconv_open1.h: Parse a /ZOS_UNIX surface specifier. Set
from_surface, to_surface.
* lib/iconv_open2.h: Copy the values of from_surface, to_surface into
the conversion descriptor.
* lib/ebcdic*.h (*_mbtowc): Test the isurface. If requested, call
swap_x15_x25 right after fetching an input byte.
(*_wctomb): Test the osurface. If requested, call swap_x15_x25 right
before storing an output byte.
* man/iconvctl.3 (REQUEST VALUES): Document the
ICONV_{GET,SET}_{FROM,TO}_SURFACE requests.
* src/iconv.c (main): If ICONV_EBCDIC_ZOS_UNIX is set, set the from/to
surfaces accordingly.
* man/iconv.1 (ENVIRONMENT): New section.
* tests/check-ebcdic: New file.
* tests/Makefile.in (check): Invoke it.
(SOURCE_FILES): Add it.
* NEWS: Mention the new functionality.
|
|
|
8c9fa489
|
2022-04-13T18:33:16
|
|
Fix undefined behaviour.
* tests/test-shiftseq.c (main2): Make input array larger.
|
|
|
429e0425
|
2022-02-12T13:49:13
|
|
Update source file list (regression 2022-01-23).
* lib/Makefile.in (SOURCE_FILES): Add encodings_zos.def, ebcdic*.h.
(GENERATED_FILES): Add aliases_zos.h, canonical_zos.h.
* tests/Makefile.in (SOURCE_FILES): Add IBM-*.TXT.
|
|
|
68ac8a9f
|
2022-01-23T23:37:30
|
|
New EBCDIC encodings.
Reported by Ulrich Schwab and Calvin Buckley via Jack Woehr.
* NOTES: Mention how to enable EBCDIC encodings.
* tests/IBM-*.TXT: New files.
* tools/8bit_tab_to_h.c (main): Emit copyright header with year 2022.
* tools/Makefile: Add rules for generating ebcdic*.h.
* lib/ebcdic*.h: New files, automatically generated by tools/Makefile.
* lib/ebcdic838.h: Tweak reverse mapping manually.
* lib/ebcdic1160.h: Likewise.
* lib/converters.h: Include all ebcdic*.h.
* lib/encodings_zos.def: New file.
* lib/genaliases2.c: Include encodings_zos.def.
* lib/genflags.c: Likewise.
* Makefile.devel (lib/aliases_zos.h lib/canonical_zos.h): New rule.
(lib/flags.h, totally-clean): Update.
* lib/aliases2.h: Include aliases_zos.h.
* lib/iconv.c (USE_ZOS): New macro.
Include encodings_zos.def, canonical_zos.h.
* README, man/iconv_open.3: Document the IBM-* encodings.
* tests/Makefile.in (check-extra-yes): Also test the EBCDIC encodings.
|
|
|
05b20c93
|
2021-09-19T23:13:57
|
|
tests: Avoid check-subst failures in QEMU user-mode environments.
* Makefile.devel (GNULIB_MODULES): Add stdbool.
(gnulib-imported-files): Copy also tests/qemu.h.
* tests/is-native.c: New file.
* tests/Makefile.in (is-native, is-native.@OBJEXT@): New targets.
(clean): Remove the 'is-native' program.
(SOURCE_FILES): Add is-native.c.
(IMPORTED_FILES): Add qemu.h.
* tests/check-subst: Invoke 'locale charmap' only in native
environments.
|
|
|
91f96be0
|
2021-06-06T11:51:12
|
|
Change the license of the library from LGPL 2.0 to LGPL 2.1.
|
|
|
3acb1179
|
2020-04-04T14:58:34
|
|
Change the license of the library from LGPL 2.0 to LGPL 2.1.
|
|
|
1badfd5d
|
2019-01-27T22:57:45
|
|
Accommodate a shell that is not in /bin/sh.
|
|
|
7a3418bb
|
2018-09-22T14:23:53
|
|
Update from gnulib. Move mapping tables into the code.
|
|
|
e54fc9c1
|
2018-09-17T18:28:56
|
|
Prefer https URLs where possible.
|
|
|
67802283
|
2016-12-03T18:52:49
|
|
Create tarballs through an Automake-like "make dist" command.
|
|
|
0a04404c
|
2016-11-17T23:13:53
|
|
UTF-8: Reject surrogates and out-of-range code points.
|
|
|
500b967b
|
2016-10-04T20:49:25
|
|
Extend CP1255 mapping.
|
|
|
da2aa29c
|
2016-10-01T16:37:58
|
|
Update to current gnulib.
|
|
|
8dfc4c81
|
2012-05-13T20:08:21
|
|
Tweak the GB18030 converter to map 0x8135F437 to U+E7C7.
|
|
|
48f31c74
|
2012-02-12T20:54:51
|
|
Replace FSF snail-mail address with URL.
|
|
|
6b00cc70
|
2012-01-26T12:37:51
|
|
Modernize quoting.
|
|
|
a625a199
|
2012-01-15T12:19:04
|
|
Support for MSVC 9. Add a comment.
|
|
|
47bec3b3
|
2012-06-30T17:28:51
|
|
Improve ISO-2022-CP-MS versus CP932.
|
|
|
3a33986e
|
2011-10-24T02:39:35
|
|
New encoding ISO-2022-CP-MS.
|
|
|
e258a174
|
2011-09-02T21:34:29
|
|
Fix "make check" failure when $(srcdir) != $(builddir).
|
|
|
2665c092
|
2011-08-07T20:09:23
|
|
Remove all .cvsignore files. Not needed any more since the switch to git.
|
|
|
61400e7d
|
2011-08-07T18:33:39
|
|
Avoid a test failure on Solaris 2.6 and HP-UX 11.00.
|
|
|
cb5f9192
|
2011-08-07T18:26:51
|
|
Add comments.
|
|
|
ac379c19
|
2011-08-07T04:54:14
|
|
Upgrade the GB18030 converter to the version from 2005.
|
|
|
33b05e19
|
2011-08-06T19:01:19
|
|
Fix conversion bug in CP1258 converter.
|
|
|
fd7d5707
|
2010-11-24T03:33:29
|
|
Implement newer release of BIG5-HKSCS.
|
|
|
4a2f4d20
|
2009-12-11T12:16:52
|
|
Untabify.
|
|
|
ddd735bf
|
2009-11-23T02:22:51
|
|
Add reference to the Solaris bug.
|
|
|
2cf3ee65
|
2009-07-08T00:22:57
|
|
Fix an endless loop in the conversion to wchar_t.
|
|
|
459ce580
|
2009-01-24T23:16:06
|
|
New converter for CP1131.
|
|
|
a8e62402
|
2008-09-26T13:18:15
|
|
Use ICONV_CONST, to avoid warning.
|
|
|
ef2cb470
|
2008-09-26T13:17:21
|
|
Link with libicrt.a.
|
|
|
bb8f7987
|
2008-09-07T23:28:41
|
|
More consistent behaviour when invalid input is preceded by a shift sequence.
|
|
|
323c84a1
|
2008-04-14T11:19:33
|
|
More cleanup needed on MacOS X 10.5.
|
|
|
0531ec16
|
2007-10-07T18:30:41
|
|
Upgrade copyright header to GPLv3.
|
|
|
614f279f
|
2007-05-25T23:41:00
|
|
Add support for the Kazakh RK1048 encoding.
|
|
|
3db451a5
|
2007-04-24T06:41:00
|
|
Tweak last patch.
|
|
|
5367f752
|
2007-04-23T22:47:42
|
|
Work around a /bin/printf bug on Solaris.
|
|
|
ed5a7ddc
|
2007-02-04T15:49:22
|
|
Add the Korean postal code mark to the EUC-KR and JOHAB mappings.
|
|
|
cdc426ef
|
2006-07-31T16:14:55
|
|
Ensure that the "646" -> "ASCII" conversion works already during
"make check", before "make install".
|
|
|
917bd6c8
|
2006-07-27T12:02:37
|
|
Remove CMD.EXE scripts; assume bash.
|
|
|
a7a84a32
|
2006-07-27T12:00:38
|
|
Remove OS/2 build support that doesn't assume GNU make and GNU bash.
|
|
|
f260e646
|
2006-07-26T11:42:48
|
|
Remove MSVC/nmake build support.
|
|
|
f574c3fa
|
2006-07-25T11:08:43
|
|
Remove VMS support.
|
|
|
c8dc4a2e
|
2006-07-24T12:43:07
|
|
Use the CFLAGS also when linking.
|
|
|
5e08f217
|
2006-07-14T13:24:59
|
|
Treat *.stackdump files like core files.
|
|
|
0707a728
|
2006-07-14T13:18:10
|
|
Fix cleanup on cygwin and mingw.
|
|
|
16ebfb3f
|
2006-05-20T14:45:03
|
|
Test also the stateful parts of BIG5-HKSCS:1999, BIG5-HKSCS:2001,
BIG5-HKSCS:2004.
|
|
|
a6eee921
|
2006-05-20T14:44:27
|
|
Implement newer releases of BIG5-HKSCS.
|
|
|
5dd73c03
|
2006-05-20T14:43:52
|
|
Nop.
|
|
|
a4f903b6
|
2006-05-20T14:42:46
|
|
Test of stateful part of BIG5-HKSCS:2004.
|
|
|
133fdacb
|
2006-05-20T14:42:37
|
|
Test of stateful part of BIG5-HKSCS:2001.
|
|
|
55c8238b
|
2006-05-20T14:42:27
|
|
Test of stateful part of BIG5-HKSCS:1999.
|
|
|
7edefe50
|
2006-05-18T12:48:00
|
|
Implement newer releases of BIG5-HKSCS.
|
|
|
61e2c29c
|
2006-05-18T12:42:11
|
|
Expected mapping for BIG5-HKSCS:2004.
|
|
|
326a8003
|
2006-05-18T12:41:50
|
|
Expected mapping for BIG5-HKSCS:2001.
|
|
|
89e72f1b
|
2006-05-18T12:41:27
|
|
Update for 2006 edition of HKSCS:1999:
- 4 mistakes fixed.
- 4 characters map to pairs of Unicode characters.
- Many mappings now go to assigned Unicode characters, rather than to
the PUA.
|
|
|
d76fdd8d
|
2006-05-18T12:39:37
|
|
The 2006 version of HKSCS:1999 has cleared up some things.
|
|
|
643e86f2
|
2006-05-18T12:31:00
|
|
Moved here from BIG5-HKSCS.IRREVERSIBLE.TXT.
|
|
|
1d338634
|
2006-05-18T12:30:49
|
|
Move to BIG5-HKSCS-1999.IRREVERSIBLE.TXT.
|
|
|
e2ef1afa
|
2006-05-18T12:30:23
|
|
Moved here from BIG5-HKSCS.TXT.
|
|
|
d6858dc1
|
2006-05-18T12:30:10
|
|
Move to BIG5-HKSCS-1999.TXT.
|
|
|
8663fe9b
|
2006-05-16T19:15:03
|
|
Use EXEEXT, for Cygwin.
|
|
|
422b3b1d
|
2006-01-23T13:25:49
|
|
New feature: character-dependent substitutions.
|
|
|
6ec6f23c
|
2006-01-23T13:25:36
|
|
Unit test for substitution options.
|
|
|
cfef64ad
|
2005-12-28T14:19:38
|
|
Check the whole range of GB18030.
|
|
|
b63068d1
|
2005-12-28T14:18:56
|
|
Moved here from tests/GB18030.TXT.
|
|
|
8bb31880
|
2005-12-28T14:18:04
|
|
Move to tests/GB18030-BMP.TXT.
|
|
|
bd8c8607
|
2005-12-28T14:17:30
|
|
Creates a large portion of the GB18030.TXT table.
|
|
|
e27366b6
|
2005-12-28T14:15:32
|
|
Test the entire Unicode range.
|
|
|
e08e610f
|
2005-12-28T14:14:53
|
|
Extend GB18030 converter to cover the entire Unicode PUA.
|
|
|
0b8f69a5
|
2005-12-20T12:04:10
|
|
Add private area mappings to CP949.
|
|
|
b7709195
|
2005-12-15T12:53:32
|
|
Add private area mappings to CP950.
|
|
|
05a280da
|
2005-12-15T12:43:45
|
|
CP936 is now different from GBK.
|
|
|
c39e370c
|
2005-12-14T12:14:22
|
|
Add 5 irreversible mappings to the CP932 converter.
|
|
|
912d099a
|
2005-12-14T11:25:21
|
|
Include Euro sign.
|
|
|
bc2042c5
|
2005-09-19T15:04:57
|
|
Tweak BIG5-2003 mapping to match Ping Yeh's explanations.
|
|
|
38981e01
|
2005-05-23T10:06:44
|
|
Implement BIG5-2003 encoding.
|
|
|
c16f555c
|
2005-05-23T10:04:30
|
|
Add EURO SIGN to EUC-TW converter.
|
|
|
9341db94
|
2005-05-06T11:04:28
|
|
Support for PT154 encoding.
|
|
|
45bd190c
|
2005-05-19T17:14:19
|
|
Update FSF postal address.
|
|
|
bb98761e
|
2005-03-29T13:55:27
|
|
Implement and document ATARIST converter.
|
|
|
218d46eb
|
2005-03-14T11:36:42
|
|
More reliable error checking.
|
|
|
e6c475ab
|
2004-07-22T12:50:01
|
|
New revision of ISO-8859-7.
|
|
|
7736228d
|
2004-07-22T12:49:09
|
|
New encoding ISO-8859-11.
|
|
|
6e0919a6
|
2004-08-02T11:01:40
|
|
Update to JISX 0213 plane 1 to 2004 version (ISO-IR-233).
|
|
|
9c620c01
|
2004-01-23T12:17:54
|
|
Use binary mode for the testsuite. Needed for mingw build.
|
|
|
12d862d7
|
2004-01-22T19:33:40
|
|
Make --binary the default.
|
|
|
9b958b9f
|
2003-05-18T11:28:29
|
|
Make it possible to create DLLs with debug information.
|
|
|
9042f1c1
|
2003-05-07T19:52:28
|
|
Support for VMS.
|
|
|
d2a22aa6
|
2003-05-17T14:36:51
|
|
Tweaks for Woe32 portability.
|
|
|
1be7e00e
|
2003-04-10T10:32:39
|
|
Improved MSVC support.
|