kc3-lang/libiconv/tests

Branch :

Log

Author Commit Date CI Message
9f28054d 2024-12-15 12:52:18 Implement the //NON_IDENTICAL_DISCARD suffix from POSIX:2024. * include/iconv.h.in (ICONV_GET_DISCARD_INVALID, ICONV_SET_DISCARD_INVALID, ICONV_GET_DISCARD_NON_IDENTICAL, ICONV_SET_DISCARD_NON_IDENTICAL): New macros. * lib/converters.h (struct conv_struct): Change type of discard_ilseq to 'unsigned int'. (DISCARD_INVALID, DISCARD_UNCONVERTIBLE): New macros. * lib/iconv.c (iconv_open, iconv_open_into): Change type of discard_ilseq to 'unsigned int'. (iconvctl): Implement ICONV_GET_DISCARD_INVALID, ICONV_SET_DISCARD_INVALID, ICONV_GET_DISCARD_NON_IDENTICAL, ICONV_SET_DISCARD_NON_IDENTICAL. Change the implementation of ICONV_GET_DISCARD_ILSEQ, ICONV_SET_DISCARD_ILSEQ to test/set both bits. * lib/iconv_open1.h: Update comment. Recognize //NON_IDENTICAL_DISCARD. * lib/iconv_open2.h: Update comment. * lib/loop_unicode.h (mb_to_uc_write_replacement): Test the DISCARD_UNCONVERTIBLE bit of discard_ilseq. (unicode_loop_convert): Test the respective bit of discard_ilseq. (unicode_loop_reset): Test the DISCARD_UNCONVERTIBLE bit of discard_ilseq. * lib/loop_wchar.h (wchar_from_loop_convert, wchar_to_loop_convert): Test the DISCARD_INVALID bit of discard_ilseq. * man/iconv_open.3: Mention the //NON_IDENTICAL_DISCARD suffix. Mark as conforming to POSIX:2024. * man/iconv.3: Likewise. * man/iconv_close.3: Mark as conforming to POSIX:2024. * man/iconv.1: Likewise. * man/iconvctl.3: Document ICONV_GET_DISCARD_INVALID, ICONV_SET_DISCARD_INVALID, ICONV_GET_DISCARD_NON_IDENTICAL, ICONV_SET_DISCARD_NON_IDENTICAL. Revise the description of ICONV_GET_DISCARD_ILSEQ, ICONV_SET_DISCARD_ILSEQ. * tests/test-discard.c (test_default, test_translit, test_ignore, test_ignore_translit): Test also the ICONV_GET_DISCARD_INVALID, ICONV_GET_DISCARD_NON_IDENTICAL accessors. (test_nid, test_nid_translit, test_invd, test_invd_translit): New functions. (main): Add test cases with //NON_IDENTICAL_DISCARD suffix. * NEWS: Mention the change.
ef2f457f 2024-12-15 10:37:13 Support multiple suffixes in iconv_open, like glibc does. * lib/iconv_open1.h: Use a loop when looking for the common suffixes. * tests/test-discard.c: New file. * tests/Makefile.in (check): Run test-discard. (test-discard, test-discard.@OBJEXT@): New targets. (clean): Remove test-discard. (SOURCE_FILES): Add test-discard.c. * NEWS: Mention the change.
a4c1470b 2024-12-13 23:55:13 Distinguish byte-order state and shift-state. Reported by Tomas Kalibera <tomas.kalibera@gmail.com> in <https://lists.gnu.org/archive/html/bug-gnu-libiconv/2024-12/msg00000.html>. * lib/converters.h (struct conv_struct): Add field 'ibyteorder'. * lib/iconv_open2.h: Initialize the ibyteorder field. * lib/ucs2.h (ucs2_mbtowc): Use the ibyteorder field instead of the istate field. * lib/ucs4.h (ucs4_mbtowc): Likewise. * lib/utf16.h (utf16_mbtowc): Likewise. * lib/utf32.h (utf32_mbtowc): Likewise. * tests/test-bom-state.c: New file. * tests/Makefile.in (check): Run test-bom-state. (test-bom-state, test-bom-state.@OBJEXT@): New targets. (clean): Remove test-bom-state. (SOURCE_FILES): Add test-bom-state.c. * NEWS: Mention the change.
5448df58 2023-05-29 13:59:43 GB18030: Help transitioning away from PUA code points. * lib/gb18030ext.h (gb18030_2005_ext_wctomb): Remove function. (gb18030ext_wctomb): Renamed from gb18030_2022_ext_wctomb. * lib/gb18030uni.h (gb18030_2005_uni_wctomb): Map 6 Ext-B code points to 4-bytes sequences. (gb18030_2022_uni_wctomb): Small refactoring. * lib/gb18030_2005.h (gb18030_2005_pua2charset): Map 6 PUA code points to 4-bytes sequences instead of 2-bytes sequences. (gb18030_2005_wctomb): Update accordingly. Invoke gb18030ext_wctomb instead of gb18030_2005_ext_wctomb. * lib/gb18030_2022.h (gb18030_2022_wctomb): Invoke gb18030ext_wctomb instead of gb18030_2022_ext_wctomb. * tests/GB18030-2005.IRREVERSIBLE.TXT: Update the inverse mappings of 6 Ext-B code points and 6 PUA code points. * NEWS: Mention it.
6549d20c 2023-05-21 00:29:37 Implement GB18030 version 2022. * lib/encodings.def (GB18030): Add alias GB18030:2005. (GB18030:2022): New encoding. * lib/gb18030ext.h (gb18030_2005_ext_2uni_pagefe): Renamed from gb18030ext_2uni_pagefe. (gb18030_2022_ext_2uni_pagefe): New array. (gb18030_2005_ext_mbtowc): Renamed from gb18030ext_mbtowc. (gb18030_2022_ext_mbtowc): New function. (gb18030_2005_ext_wctomb): Renamed from gb18030ext_wctomb. (gb18030_2022_ext_wctomb): New function. * lib/gb18030uni.h (gb18030_2022_charset2uni_pua1, gb18030_2022_charset2uni_pua2): New arrays. (gb18030_2005_uni_mbtowc): Renamed from gb18030uni_mbtowc. (gb18030_2022_uni_mbtowc): New function. (gb18030_2022_uni2charset_pua1, gb18030_2022_uni2charset_pua2): New arrays. (gb18030_2005_uni_wctomb): Renamed from gb18030uni_wctomb. (gb18030_2022_uni_wctomb): New function. * lib/gb18030_2005.h: Renamed from lib/gb18030.h. Update comments. (gb18030_2005_mbtowc): Renamed from gb18030_mbtowc. (gb18030_2005_pua2charset): Renamed from gb18030_pua2charset. (gb18030_2005_wctomb): Renamed from gb18030_wctomb. * lib/gb18030_2022.h: New file, based on lib/gb18030_2005.h. * lib/converters.h: Don't include gb18030.h. Include gb18030_2005.h, gb18030_2022.h. * lib/Makefile.in (SOURCE_FILES): Remove gb18030.h. Add gb18030_2005.h, gb18030_2022.h. * tests/GB18030-2005-BMP.TXT: Renamed from tests/GB18030-BMP.TXT. * tests/GB18030-2005.IRREVERSIBLE.TXT: Renamed from tests/GB18030.IRREVERSIBLE.TXT. * tests/GB18030-2022-BMP.TXT: New file. * tests/Makefile.in (check): Test GB18030:2005 instead of GB18030. Also test GB18030:2022. (clean): Don't remove GB18030.TXT. Instead, remove GB18030-2005.TXT and GB18030-2022.TXT. (SOURCE_FILES): Update. Add GB18030-2022-BMP.TXT. * README: Mention the new encoding. * man/iconv_open.3: Likewise. * NEWS: Likewise.
af375a34 2023-05-12 22:50:18 Integrate the last change. * src/Makefile.in (SOURCE_FILES): Add zos-tag.h. * tests/Makefile.in (SOURCE_FILES): Add check-tag. * NEWS: Mention the new functionality.
153d4079 2023-05-12 22:41:08 On z/OS, set a charset tag on iconv's output file. For the concept of charset tags as external metadata on z/OS files, see <https://lists.gnu.org/archive/html/bug-gnu-libiconv/2023-04/msg00021.html>. * src/zos-tag.h: New file. * src/iconv.c: Include zos-tag.h. (convert): Add a 'tocode' parameter. On z/OS, turn off auto-conversion and tag the output file. (main): Update callers. * tests/check-ebcdic: On z/OS, make all test files initially untagged. * tests/check-tag: New file. * tests/Makefile.in (check): Pass the host_os to check-ebcdic. Invoke check-tag.
19b6af5e 2023-04-03 04:12:01 Allow overriding the newline conversion for EBCDIC encodings. Reported by Mike Fulton <mikefultonpersonal@gmail.com> in <https://lists.gnu.org/archive/html/bug-gnu-libiconv/2023-04/msg00009.html>. * include/iconv.h.in (ICONV_SURFACE_NONE, ICONV_SURFACE_EBCDIC_ZOS_UNIX): New macros. (ICONV_GET_FROM_SURFACE, ICONV_SET_FROM_SURFACE, ICONV_GET_TO_SURFACE, ICONV_SET_TO_SURFACE): New macros. * lib/converters.h (struct conv_struct): Add the fields isurface, osurface. (swap_x15_x25): New macro. * lib/iconv.c (iconv_open, iconv_open_into): Add local variables from_surface, to_surface. (ALL_SURFACES): New macro. (iconvctl): Adjust ICONV_TRIVIALP implementation. Implement the ICONV_{GET,SET}_{FROM,TO}_SURFACE requests. * lib/iconv_open1.h: Parse a /ZOS_UNIX surface specifier. Set from_surface, to_surface. * lib/iconv_open2.h: Copy the values of from_surface, to_surface into the conversion descriptor. * lib/ebcdic*.h (*_mbtowc): Test the isurface. If requested, call swap_x15_x25 right after fetching an input byte. (*_wctomb): Test the osurface. If requested, call swap_x15_x25 right before storing an output byte. * man/iconvctl.3 (REQUEST VALUES): Document the ICONV_{GET,SET}_{FROM,TO}_SURFACE requests. * src/iconv.c (main): If ICONV_EBCDIC_ZOS_UNIX is set, set the from/to surfaces accordingly. * man/iconv.1 (ENVIRONMENT): New section. * tests/check-ebcdic: New file. * tests/Makefile.in (check): Invoke it. (SOURCE_FILES): Add it. * NEWS: Mention the new functionality.
8c9fa489 2022-04-13 18:33:16 Fix undefined behaviour. * tests/test-shiftseq.c (main2): Make input array larger.
429e0425 2022-02-12 13:49:13 Update source file list (regression 2022-01-23). * lib/Makefile.in (SOURCE_FILES): Add encodings_zos.def, ebcdic*.h. (GENERATED_FILES): Add aliases_zos.h, canonical_zos.h. * tests/Makefile.in (SOURCE_FILES): Add IBM-*.TXT.