kc3-lang/libxml2/parserInternals.c

Branch :

Log

Author Commit Date CI Message
e75e878e 2024-05-20 13:58:22 doc: Update and fix documentation
ec2f68f1 2024-05-14 15:58:44 parser: Report malloc failure in xmlNewInputFromFile
4ff2dccf 2024-05-10 02:04:52 SAX2: Warn if URI resolution failed
84a71860 2024-02-26 15:14:28 xmlreader: Fix xmlTextReaderConstEncoding Regression from commit f1c1f5c6. Fixes #697.
c444c96e 2024-02-13 12:22:28 parser: Report OOM errors even after parser was stopped
63986c45 2024-01-22 21:02:16 parser: Report fatal error if document entity couldn't be loaded Only lower error level when loading entities. Fixes #667.
12f0bb94 2024-01-05 01:14:28 parser: Synchronize more options
3efbe916 2024-01-05 00:11:29 parser: Mark 'token' member as unused in xmlParserCtxt
b82fd81d 2024-01-04 23:25:06 parser: Rework xmlCtxtParseDocument Make xmlCtxtParseDocument take a parser input which can be popped after parsing.
d0eb5a7e 2024-01-03 18:12:29 parser: Remove xmlErrEncodingInt Convert the last user to xmlFatalErr.
f30b9b23 2024-01-03 18:11:44 fuzz: Add assertion in xmlCopyCharMultibyte This is an internal function that should never receive out-of-range codepoints.
0821efc8 2024-01-02 18:33:57 encoding: Check whether encoding handlers support input/output The "HTML" encoding handler doesn't support input which could lead to a wrong error report.
85f99023 2024-01-02 17:52:43 parser: Fix buffer size checks Don't test size of remaining data. This causes false positives with memory buffers. Also impose XML_MAX_HUGE_LENGTH limit when parsing with XML_PARSE_HUGE.
16b0dbc1 2023-12-29 18:47:30 parser: Fix XML_ERR_UNSUPPORTED_ENCODING errors Commit 45157261 added the check in the wrong place. Also allow unsupported encoding in xmlNewInputInternal. Fixes #654.
e45a4d71 2023-12-29 00:00:21 io: Always forward IO errors to global handler The HTTP module raises errors without context. This won't be fixed, so send them to the global error handler.
7e0bbbc1 2023-12-27 18:33:30 parser: New input API Provide a new set of functions to create xmlParserInputs. These can be used for the document entity or from external entity loaders. - Don't require xmlParserInputBuffer. - All functions take a base URI. - All functions take an encoding as string. - xmlNewInputURL also takes a public ID. - xmlNewInputMemory takes a size_t. - Optimization hints for memory buffers. Improve documentation. Only call xmlInitParser before allocating a new parser context. Call xmlCtxtUseOptions as early as possible.
45157261 2023-12-27 21:30:13 parser: Downgrade XML_ERR_UNSUPPORTED_ENCODING to warning If the actual encoding is UTF-8 or ASCII, we don't want to fail.
24b7144f 2023-12-27 15:50:58 parser: More refactoring of entity parsing Remove xmlCreateEntityParserCtxtInternal. Rework xmlNewEntityInputStream.
6a9a88a1 2023-12-26 03:13:05 parser: Move progressive flag into input struct
d944a415 2023-12-26 02:10:35 parser: Fix in-parameter-entity and in-external-dtd checks Use in ctxt->input->entity instead of ctxt->inputNr to determine whether we are inside a parameter entity. Stop using ctxt->external to check whether we're in an external DTD. This is signaled by ctxt->inSubset == 2.
c1bddd4c 2023-12-23 01:09:17 parser: Mark 'length' member of xmlParserInput as unused
955c177f 2023-12-23 00:58:36 parser: Stop using 'directory' struct member This was only used as a pointless fallback for URI resolution.
60841beb 2023-12-25 18:31:22 parser: Make XML_IO_NETWORK_ATTEMPT behave as before Always reported to generic error, not to parser context for backward compatibility. Several downstream test suites rely on this behavior.
a2693410 2023-12-23 00:35:30 io: Move some code from xmlIO.c to parserInternals.c Move everything related to parser contexts to parserInternals.c.
c8f1f4a2 2023-12-21 17:30:38 doc: Improve documentation of error handlers
13043691 2023-12-20 00:33:34 parser: Rename xmlErrParser to xmlCtxtErr
8d0aaf4b 2023-12-19 20:47:36 parser: Remove xmlErrEncoding Use xmlFatalErr or xmlCtxtErrIO.
9fbe46ba 2023-12-19 20:10:10 io: Consolidate error messages
23345a1c 2023-12-19 19:52:28 io: Report IO errors through xmlCtxtErrIO This is also a new public API function to be used in external entity loaders.
7e511f35 2023-12-19 15:41:37 io: Pass error codes from xmlFileOpenReal to xmlNewInputFromFile This allows to report the reason why opening a file failed to the parser context and improve error messages. Now we can also remove the stat call before opening a file.
531d06ad 2023-12-18 22:48:24 error: Stop printing some errors by default Unfortunately, it's long-standing behavior for libxml2 to print all reported errors to stderr by default. This default behavior is now partially disabled. If no error handler is set, only parser and validation errors are passed to a generic error handler or printed to stderr. Other errors are still available via xmlGetLastError and can be captured with a structured error handler.
54c70ed5 2023-12-18 19:31:29 parser: Improve error handling Introduce xmlCtxtSetErrorHandler allowing to set a structured error for a parser context. There already was the "serror" SAX handler but this always receives the parser context as argument. Start to use xmlRaiseMemoryError. Remove useless arguments from memory error functions. Rename xmlErrMemory to xmlCtxtErrMemory. Remove a few calls to xmlGenericError. Remove support for runtime entity debugging.
69d88566 2023-12-15 14:46:35 malloc-fail: Fix OOM error handling in xmlSwitchInputEncoding Make sure that in->buffer won't be set to NULL if a memory allocation fails. Short-lived regression.
79a9938d 2023-12-13 17:25:37 parser: Don't grow or shrink pull parser memory buffers Readd check for memory buffers without a read callback to avoid XML_MAX_LOOKUP_LIMIT errors if users provide a custom input buffer. Regressed with commit 834b8123 and later changes.
f19a9510 2023-12-10 17:50:22 parser: Report malloc failures Fix many places where malloc failures aren't reported. Make xmlErrMemory public. This is useful for custom external entity loaders. Introduce new API function xmlSwitchEncodingName. Change the way how we store whether the the parser is stopped. This used to be signaled by setting ctxt->instate to XML_PARSER_EOF which was misdesigned and error-prone. Set ctxt->disableSAX to 2 instead and introduce a macro PARSER_STOPPED. Also stop to remove parser inputs in xmlHaltParser. This allows to remove many checks of ctxt->instate. Introduce xmlErrParser to handle errors if a parser context is available.
bd5ad030 2023-12-10 14:56:21 encoding: Report malloc failures Introduce new API functions that return a separate error code if a memory allocation fails. - xmlOpenCharEncodingHandler - xmlLookupCharEncodingHandler Fix a few places where malloc failures weren't reported.
43b511fa 2023-11-26 14:31:39 parser: Make CRLF increment line number Partial revert of cb927e85 fixing CRLFs not incrementing the line number. This requires to rework xmlParseQNameHashed. The original implementation prompted the change to xmlCurrentChar which really shouldn't modify the 'cur' pointer as side effect. But the NEXTL macro relies on this behavior. Ultimately, we should reintroduce the change to xmlCurrentChar and fix the NEXTL macro. This will lead to single CRs incrementing the line number as well which seems more consistent. Fixes #628.
ff6c3188 2023-11-23 15:22:59 include: Remove useless 'const' from function arguments
02856674 2023-10-22 15:56:46 parser: Remove redundant IS_CHAR check in xmlCurrentChar
c082ef46 2023-08-09 16:59:36 parser: Stop switching to ISO-8859-1 on encoding errors Use U+FFFD Replacement Character if invalid UTF-8 is encountered in recovery mode. Also rewrite xmlNextChar and xmlCurrentChar. Fixes #598.
572ecc17 2023-10-22 13:59:55 parser: Fix buffer shrinking when push parsing Short-lived regression from b76d81da.
b76d81da 2023-10-06 11:50:29 parser: Fix regression when push parsing parameter entities Short-lived regression from 834b8123. Also shrink parameter entity buffers when push parsing.
eb69c1d3 2023-10-02 12:16:05 parser: Fix initialization of namespace data Move initialization to xmlInitSAXParserCtxt. Also add missing XML_HIDDEN to xmlParserNsFree. Fixes #597.
e0dd330b 2023-09-29 00:18:44 parser: Use hash tables to avoid quadratic behavior Use a hash table to lookup namespaces by prefix. The hash table stores an index into the namespace table. Auxiliary data for namespaces is stored in a separate array along the main namespace table. Use a hash table to verify attribute uniqueness. The hash table stores an index into the attribute table. Reuse hash value from the dictionary to avoid computing them twice. See #346.
cb927e85 2023-09-25 14:35:43 parser: Don't skip CR in xmlCurrentChar Skip over carriage returns later in xmlNextChar.
b9db3d7d 2023-09-22 15:45:20 parser: Simplify xmlStringCurrentChar Start to move away from using this function.
8c084ebd 2023-09-21 22:57:33 doc: Make apibuild.py happy
9b5cce7a 2023-09-21 00:44:50 include: Remove more unnecessary includes
699299ca 2023-09-20 18:54:39 globals: Stop including globals.h
11a1839d 2023-09-20 17:54:48 globals: Move remaining globals back to correct header files This undoes a lot of damage.
4e1c13eb 2023-09-18 14:45:10 debug: Remove debugging code This is barely useful these days and only clutters the code base.
e48f2695 2023-08-29 17:41:18 parser: Remove push parser debugging code
ed3bd052 2023-08-20 20:48:10 parser: Allow to set maximum amplification factor
f1c1f5c6 2023-08-16 19:43:02 parser: Revert change to doc->encoding Fixes #579.
61b8e097 2023-08-16 19:20:47 parser: Never use UTF-8 encoding handler
b973ceaf 2023-08-09 18:37:20 parser: Fix mistake in xmlDetectEncoding Short-lived regression.
95e81a36 2023-08-08 15:21:31 parser: Decode all data in xmlCharEncInput Even with flush set to true, xmlCharEncInput didn't guarantee to decode all data. This complicated the push parser. Remove the flush flag and always decode all available data. Also fix ICU code where the flush flag has a different meaning. Always set flush to false and retry even with empty input buffers.
834b8123 2023-08-08 15:21:28 parser: Stream data when reading from memory Don't create a copy of the whole input buffer. Read the data chunk by chunk to save memory. Historically, it was probably envisioned to read data from memory without additional copying. This doesn't work reliably with the current design of the XML parser which requires a terminating null byte at the end of input buffers. This lead to xmlReadMemory interfaces, which expect pointer and size arguments, being changed to make a zero-terminated copy of the input buffer. Interfaces based on xmlReadDoc, which actually expect a zero-terminated string and would make zero-copy operation work, were then simplified to rely on xmlReadMemoryi, resulting in an unnecessary copy. To avoid copying (possibly gigabytes) of memory temporarily, we now stream in-memory input just like content read from files in a chunk-by-chunk fashion (using a somewhat outdated INPUT_CHUNK size of 250 bytes). As a side effect, we also avoid another copy of the whole input when handling non-UTF-8 data which was made possible by some earlier commits. Interfaces expecting zero-terminated strings now make use of strnlen which unfortunately isn't part of the standard C library and only mandated since POSIX 2008.
59fa0bb3 2023-08-08 15:21:14 parser: Simplify input pointer updates The base member always points to the beginning of the buffer.
4ee08155 2023-08-08 15:19:51 encoding: Move rawconsumed accounting to xmlCharEncInput
ec7be506 2023-08-08 15:19:46 parser: Rework encoding detection Introduce XML_INPUT_HAS_ENCODING flag for xmlParserInput which is set when xmlSwitchEncoding is called. The parser can use the flag to reliably detect whether an encoding was already set via user override, BOM or other auto-detection. In this case, the encoding declaration won't be used to switch the encoding. Before, an inscrutable mix of ctxt->charset, ctxt->input->encoding and ctxt->input->buf->encoder was used. Introduce private helper functions to switch encodings used by both the XML and HTML parser: - xmlDetectEncoding which skips over the BOM, allowing to remove the BOM checks from other encoding functions. - xmlSetDeclaredEncoding, replacing htmlCheckEncodingDirect, which warns about encoding mismatches. If users override the encoding, store the declared instead of the actual encoding in xmlDoc. In this case, the actual encoding is known and the raw value from the doc is more useful. Also use the input flags to store the ISO-8859-1 fallback state. Restrict the fallback to cases where no encoding was specified. (The fallback is only useful in recovery mode and these days broken UTF-8 is probably more likely than ISO-8859-1, so it might eventually be removed completely.) The 'charset' member of xmlParserCtxt is now unused. The 'encoding' member of xmlParserInput is now unused. The 'standalone' member of xmlParserInput is renamed to 'flags'. A new parser state XML_PARSER_XML_DECL is added for the push parser.
131d0dc0 2023-08-08 15:19:39 parser: Don't use 'standalone' member of xmlParserInput The standalone declaration is only parsed in the main input stream.
88447447 2023-06-23 23:04:30 parser: Fix typo in previous commit
9d0541dd 2023-06-22 18:06:53 parser: Make xmlSwitchEncoding always skip the BOM Chromium calls xmlSwitchEncoding from the start document handler and relies on this function to skip the BOM. Commit 98840d40 changed the behavior when switching to UTF-16 since inspecting the input buffer at this point is fragile. Revert part of the commit to also skip a potential (decoded UTF-8) BOM when switching to UTF-16. Make sure that we do this only at the start of an input stream to avoid U-FEFF characters being lost. BOM handling should ultimately be moved to the parsing code to avoid such bugs. See https://bugs.chromium.org/p/chromium/issues/detail?id=1451026
b236b7a5 2023-06-08 21:53:05 parser: Halt parser when growing buffer results in OOM Fix short-lived regression from previous commit. It might be safer to make xmlBufSetInputBaseCur use the original buffer even in case of errors. Found by OSS-Fuzz.
20f5c734 2023-06-07 14:05:34 parser: Recover more input from encoding errors Don't halt the parser in xmlParserGrow to allow more input to be recovered in case of encoding errors. Fixes #543.
e0f3016f 2023-05-18 17:31:44 parser: Fix regression when push parsing UTF-8 sequences Partial UTF-8 sequences are allowed when push parsing. Fixes #542.
9dae389c 2023-05-09 13:28:06 parser: Fix "huge input lookup" error with push parser Fix parsing of larger documents without XML_PARSE_HUGE. Should fix #538.
320f5084 2023-04-30 18:25:09 parser: Improve handling of encoding and IO errors Make sure that xmlCharEncInput, xmlParserInputBufferPush and xmlParserInputBufferGrow set the correct error code in the xmlParserInputBuffer. Handle errors when calling these functions.
fc69cf56 2023-04-30 17:51:29 parser: Move xmlFatalErr to parserInternals.c
3ffcc03b 2023-03-13 19:38:41 parser: Deprecate more internal functions
9282b084 2023-04-19 21:55:24 parser: Fix regression in memory pull parser with encoding Revert another change from commit 98840d40. Decode the whole buffer when reading from memory and switching to the initial encoding. Add some comments about potential improvements.
a19fa11e 2023-04-13 15:11:47 parser: Fix regression when switching input encodings Revert some changes from commit 98840d40. WebKit/Chromium can actually switch from ISO-8859-1 to UTF-16 in the middle of parsing. This is a bad idea, but we have to keep supporting this use case.
921796b0 2023-04-12 13:43:28 parser: Don't grow push parser buffers This should fix a short-lived regression when push parsing with encodings.
0e42adce 2023-03-30 13:53:24 parser: Halt parser if switching encodings fails Avoids buffer overread in htmlParseHTMLAttribute. Found by OSS-Fuzz.
36602292 2023-03-26 14:11:31 parser: Fix buffer overread in xmlDetectEBCDIC Short-lived regression found by OSS-Fuzz.
7fbd454d 2023-03-21 13:26:36 parser: Grow input buffer earlier when reading characters Make more bytes available after invoking CUR_CHAR or NEXT.
98840d40 2023-03-21 19:07:12 parser: Rework EBCDIC code page detection To detect EBCDIC code pages, we used to switch the encoding twice and had to be very careful not to decode data after the XML declaration before the second switch. This relied on a hard-coded expected size of the XML declaration and was complicated and unreliable. Now we convert the first 200 bytes to EBCDIC-US and parse the encoding declaration manually.
04d1bedd 2023-03-21 13:08:44 parser: Rework shrinking of input buffers Don't try to grow the input buffer in xmlParserShrink. This makes sure that no memory allocations are made and the function always succeeds. Remove unnecessary invocations of SHRINK. Invoke SHRINK at the end of DTD parsing loops. Shrink before growing.
1a91392c 2023-03-16 17:48:57 parser: More fixes to xmlParserGrow xmlHaltParser must be called after reporting an error. Switch to xmlBufSetInputBaseCur.
ca2bfece 2023-03-15 16:18:11 malloc-fail: Fix buffer overread when reading from input Found by OSS-Fuzz, see #344.
b167c731 2023-03-14 14:42:36 parser: Fix short-lived regression causing infinite loops Fix 3eb6bf03. We really have to halt the parser, so the input buffer gets reset.
e7c3a4ca 2023-03-13 19:19:46 parser: Deprecate some parser input functions
2099441f 2023-03-13 17:51:13 parser: Stop calling xmlParserInputShrink Introduce xmlParserShrink which takes a parser context to simplify error handling.
457fc622 2023-03-13 16:51:14 malloc-fail: Fix null deref in xmlParserInputShrink Found by OSS-Fuzz.
3eb6bf03 2023-03-12 16:47:15 parser: Stop calling xmlParserInputGrow Introduce xmlParserGrow which takes a parser context to simplify error handling.
2355eac5 2023-01-22 14:52:06 malloc-fail: Fix null deref if growing input buffer fails Also add some error checks. Found with libFuzzer, see #344.
077df27e 2022-12-22 15:22:01 parser: Fix integer overflow of input ID Applies a patch from Chromium. Also stop incrementing input ID of subcontexts. This isn't necessary. Fixes #465.
ce76ebfd 2022-12-19 20:56:23 entities: Stop counting entities This was only used in the old version of xmlParserEntityCheck.
463bbeec 2022-12-19 18:39:45 entities: Rework entity amplification checks This commit implements robust detection of entity amplification attacks, better known as the "billion laughs" attack. We now limit the size of the document after substitution of entities to 10 times the size before expansion. This guarantees linear behavior by definition. There already was a similar check before, but the accounting of "sizeentities" (size of external entities) and "sizeentcopy" (size of all copies created by entity references) wasn't accurate. We also need saturation arithmetic since we're historically limited to "unsigned long" which is 32-bit on many platforms. A maximum of 10 MB of substitutions is always allowed. This should make use cases like DITA work which have caused problems in the past. The old checks based on the number of entities were removed. This is accounted for by adding a fixed cost to each entity reference. Entity amplification checks are now enabled even if XML_PARSE_HUGE is set. This option is mainly used to allow larger text nodes. Most users were unaware that it also disabled entity expansion checks. Some of the limits might be adjusted later. If this change turns out to affect legitimate use cases, we can add a separate parser option to disable the checks. Fixes #294. Fixes #345.
a8b31e68 2022-11-21 21:35:01 parser: Fix progress check when parsing character data Skip over zero bytes to guarantee progress. Short-lived regression.
691a7719 2022-11-20 19:55:12 parser: Fix 'consumed' accounting when switching encodings
249cee4b 2022-11-13 20:19:13 io: Fix a few integer overflows in I/O statistics There are still many places where arithmetic on "consumed" stats isn't checked for overflow, affecting platforms with a 32-bit long type.
6b570619 2022-11-13 19:44:00 io: Rearrange code in xmlSwitchInputEncodingInt No functional change.
46cd7d22 2022-11-13 16:30:46 io: Remove xmlInputReadCallbackNop In some cases, for example when using encoders, the read callback was set to NULL, in other cases it was set to xmlInputReadCallbackNop. xmlGROW only tested for xmlInputReadCallbackNop, resulting in errors when parsing large encoded content from memory. Always use a NULL callback for memory buffers to avoid ambiguities. Fixes #262.
9feafbc5 2022-11-13 16:56:10 io: Check for memory buffer early in xmlParserInputGrow
6843fc72 2022-09-01 02:58:00 Remove or annotate char casts
ad338ca7 2022-09-01 01:18:30 Remove explicit integer casts Remove explicit integer casts as final operation - in assignments - when passing arguments - when returning values Remove casts - to the same type - from certain range-bound values The main motivation is that these explicit casts don't change the result of operations and only render UBSan's implicit-conversion checks useless. Removing these casts allows UBSan to detect cases where truncation or sign-changes occur unexpectedly. Document some explicit casts as truncating and add a few missing ones.
65dc8a63 2022-09-01 00:13:19 Make xmlNewSAXParserCtx take a const sax handler Also improve documentation.
0f568c0b 2022-08-26 01:22:33 Consolidate private header files Private functions were previously declared - in header files in the root directory - in public headers guarded with IN_LIBXML - in libxml.h - redundantly in source files that used them. Consolidate all private header files in include/private.