kmx git

Commit	Date	Message
481d79d4	2022-12-19T15:26:46	entities: Add XML_ENT_PARSED flag To check whether an entity was already parsed, the code previously tested whether "checked" was non-zero or "children" was non-null. The "children" check could be unreliable because an empty entity also results in an empty (NULL) node list. Use a separate flag to make this check more reliable.
4b959ee1	2022-12-01T13:23:09	Remove hacky heuristic from b2dc5675e94aa6b5557ba63f7d66b0f08dd17e4d Checking whether the context is close to the parent context by hardcoding 250 is not portable (I noticed tests were failing on Morello since the value is 288 there due to pointers being 128 bits). Instead we should ensure that the XML_VCTXT_USE_PCTXT flag is not set in cases where the user data is not actually a parser context (or ideally add a separate field but that would be an ABI break. From what I can see in the source, the XML_VCTXT_USE_PCTXT is only set if the userData field points to a valid context, and if this is not the case the flag should be cleared when changing userData rather than relying on the offset between the two. Looking at the history, I think d7cb33cf44aa688f24215c9cd398c1a26f0d25ff fixed most of the need for this workaround, but it looks like there are a few more locations that need updating; This commit changes two more places to set/clear/copy the XML_VCTXT_USE_PCTXT flag, so this heuristic should not be needed anymore. I've also drop two = NULL assignment in xmllint since this is not needed after a call to memset(). There was also an uninitialized vctxt.flags (and other fields) in `xmlShellValidate()`, which I've fixed by adding a memset() call.
c62c0d82	2022-12-01T12:58:11	Correctly relocate internal pointers after realloc() Adding an offset to a deallocated pointer and assuming that it can be dereferenced is undefined behaviour. When running libxml2 on CHERI-enabled systems such as Arm Morello this results in the creation of an out-of-bounds pointer that cannot be dereferenced and therefore crashes at runtime. The effect of this UB is not just limited to architectures such as CHERI, incorrect relocation of pointers after realloc can in fact cause FORTIFY_SOURCE errors with recent GCC: https://developers.redhat.com/articles/2022/09/17/gccs-new-fortification-level
c16fd705	2022-11-25T14:52:37	xpath: Make init function private
53ab3840	2022-11-25T14:26:59	encoding: Make init function private
05c3a458	2022-11-25T14:15:43	tests: Check that xmlInitParser doesn't allocate memory
78c0391b	2022-11-25T13:55:39	parser: Register atexit handler in locked section
ed053c50	2022-11-25T12:27:14	dict: Make init/cleanup functions private
7010d877	2022-11-25T12:06:27	threads: Rework initialization Make init/cleanup functions private. Merge xmlOnceInit into xmlInitThreadsInternal.
9dbf1374	2022-11-24T20:52:57	parser: Make some module init/cleanup functions private
cecd364d	2022-11-24T16:38:47	parser: Don't call *DefaultSAXHandlerInit from xmlInitParser Change the default handler definitions to match the result after calling the initialization functions. This makes sure that no thread-local variables are accessed when calling xmlInitParser.
b1f9c193	2022-11-22T21:39:01	parser: Fix push parser with unterminated CDATA sections Short-lived regression found by OSS-Fuzz.
0e193f0d	2022-11-21T22:09:19	parser: Remove dangerous check in xmlParseCharData If this check succeeds, xmlParseCharData could be called over and over again without making progress, resulting in an infinite loop. It's only important to check for XML_PARSER_EOF which is done later. Related to #441.
94ca36c2	2022-11-21T22:07:11	parser: Restore parser state in xmlParseCDSect Fixes #441.
a8b31e68	2022-11-21T21:35:01	parser: Fix progress check when parsing character data Skip over zero bytes to guarantee progress. Short-lived regression.
c63900fb	2022-11-21T20:11:35	parser: Check terminate flag when push parsing CDATA sections Found by OSS-Fuzz.
a781ee33	2022-11-21T20:10:42	Revert "parser: Add overflow checks to xmlParseLookup functions" This reverts commit bfc55d688427972d093be010a8c2ef265375fcb2. It's better to fix the root cause.
bfc55d68	2022-11-21T18:29:54	parser: Add overflow checks to xmlParseLookup functions Short-lived regression found by OSS-Fuzz.
9e4a46ac	2022-11-20T22:03:08	parser: Merge misc, prolog and epilog cases in push parser
55fb8f72	2022-11-20T15:35:49	parser: Fix push parser with 1-3 byte initial chunk Make sure that ctxt->charset is initialized properly.
68a6518c	2022-11-15T18:23:33	parser: Rewrite push parser boundary checks Remove inaccurate xmlParseCheckTransition check. Remove non-incremental xmlParseGetLasts check. Add functions that check for several boundary constructs more accurately, keeping track of progress in ctxt->checkIndex. Fixes #439.
2059df53	2022-11-14T22:27:58	buf: Deprecate static/immutable buffers
4955e0c9	2022-11-14T20:16:22	io: Don't shrink memory input buffers
117bab22	2022-11-14T20:15:59	parser: Don't call xmlSHRINK from push parser xmlSHRINK also calls xmlParserInputGrow which isn't needed in the push parser.
f00739c1	2022-11-14T00:18:39	parser: Ignore cdata argument in xmlParseCharData It never could be used to parse CDATA sections.
e4f56a72	2022-11-13T23:42:10	parser: Simplify xmlParseConditionalSections
3582b07b	2022-11-13T22:57:32	parser: Fix content parser progress checks This is another attempt at fixing parser progress checks. Instead of relying on in->consumed, which could overflow, change some content parser functions to make guaranteed progress on certain byte sequences.
f7ad338e	2022-11-13T21:59:23	parser: Fix attribute parser progress checks This is another attempt at fixing parser progress checks. Instead of relying on in->consumed, which could overflow, make the attribute parser functions return a NULL name only if they don't make progress.
f61b8a62	2022-11-13T21:47:03	parser: Fix DTD parser progress checks This is another attempt at fixing parser progress checks. Instead of relying on in->consumed, which could overflow, change some DTD parser functions to make guaranteed progress on certain byte sequences.
46cd7d22	2022-11-13T16:30:46	io: Remove xmlInputReadCallbackNop In some cases, for example when using encoders, the read callback was set to NULL, in other cases it was set to xmlInputReadCallbackNop. xmlGROW only tested for xmlInputReadCallbackNop, resulting in errors when parsing large encoded content from memory. Always use a NULL callback for memory buffers to avoid ambiguities. Fixes #262.
a70f7d47	2022-11-04T14:03:31	parser: Fix error message in xmlParseCommentComplex Fixes #421.
afc7e3a7	2022-11-02T16:11:00	malloc-fail: Fix memory leak in xmlParseReference Found with libFuzzer, see #344.
e129c1d1	2022-11-02T16:02:39	malloc-fail: Fix infinite loop in xmlSkipBlankChars Found with libFuzzer, see #344.
865e142c	2022-11-02T15:46:11	malloc-fail: Fix memory leak in xmlCreatePushParserCtxt Found with libFuzzer, see #344.
ffaec758	2022-08-25T17:43:08	Fix integer overflows with XML_PARSE_HUGE Also impose size limits when XML_PARSE_HUGE is set. Limit size of names to XML_MAX_TEXT_LENGTH (10 million bytes) and other content to XML_MAX_HUGE_LENGTH (1 billion bytes). Move some the length checks to the end of the respective loop to make them strict. xmlParseEntityValue didn't have a length limitation at all. But without XML_PARSE_HUGE, this should eventually trigger an error in xmlGROW. Thanks to Maddie Stone working with Google Project Zero for the report!
1a2d8ddc	2022-10-11T13:02:47	parser: Fix potential memory leak in xmlParseAttValueInternal Fix memory leak in case xmlParseAttValueInternal is called with a NULL `len` a non-NULL `alloc` argument. This static function is never called with such arguments internally, but the misleading code should be fixed nevertheless. Fixes #422.
a9669679	2022-09-09T01:44:00	error: Don't use initGenericErrorDefaultFunc The code in xmlInitParser did only set the error handler if it was NULL which should never happen.
59f2f60e	2022-09-02T00:27:57	Remove "runtime debugging" This doesn't seem useful as configuration option.
884e142d	2022-09-01T22:44:02	Fix --with-schemas --without-xpath build xmlXPathInit must be called for schemas.
6843fc72	2022-09-01T02:58:00	Remove or annotate char casts
2cac6269	2022-09-01T03:14:13	Don't use sizeof(xmlChar) or sizeof(char)
ad338ca7	2022-09-01T01:18:30	Remove explicit integer casts Remove explicit integer casts as final operation - in assignments - when passing arguments - when returning values Remove casts - to the same type - from certain range-bound values The main motivation is that these explicit casts don't change the result of operations and only render UBSan's implicit-conversion checks useless. Removing these casts allows UBSan to detect cases where truncation or sign-changes occur unexpectedly. Document some explicit casts as truncating and add a few missing ones.
0f568c0b	2022-08-26T01:22:33	Consolidate private header files Private functions were previously declared - in header files in the root directory - in public headers guarded with IN_LIBXML - in libxml.h - redundantly in source files that used them. Consolidate all private header files in include/private.
48f84ea8	2022-08-25T21:31:08	Remove internal macros from parserInternals.h Replace MOVETO_ENDTAG with code that updates line and column numbers.
58fc89e8	2022-08-25T20:57:30	Deprecate internal parser functions
34a050cd	2022-08-24T16:35:58	Move some HTML functions to correct header file
fd85b566	2022-08-24T15:12:24	Mark more parser functions as deprecated No compiler warnings generated yet.
0e49f882	2022-08-24T05:25:37	Mark most SAX1 functions as deprecated No compiler warnings generated yet.
9a82b94a	2022-08-24T04:21:58	Introduce xmlNewSAXParserCtxt and htmlNewSAXParserCtxt Add API functions to create a parser context with a custom SAX handler without having to mess with ctxt->sax manually.
5b2d07a7	2022-08-20T17:00:50	Use xmlStrlen in *CtxtReadDoc xmlStrlen handles buffers larger than INT_MAX more gracefully.
4ad71c2d	2022-08-20T16:19:34	Fix xmlCtxtReadDoc with encoding xmlCtxtReadDoc used to create an input stream involving xmlNewStringInputStream. This would create a stream without an input buffer, causing problems with encodings (see #34). After commit aab584dc3, an error was returned even with UTF-8 encodings which happened to work before. Make xmlCtxtReadDoc call xmlCtxtReadMemory which doesn't suffer from these issues. Also fix htmlCtxtReadDoc. Fixes #397.
5930fe01	2022-07-18T20:59:45	Reset nsNr in xmlCtxtReset
ca2c91f1	2022-06-28T19:24:14	Fix memory leak in xmlLoadEntityContent error path Free the input stream if pushing it fails. Found by OSS-Fuzz. https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43743
ecba4cbd	2022-06-28T19:22:31	Avoid double-free if malloc fails in inputPush It's the caller's responsibility to free the input stream if this function fails.
3e7b4f37	2022-05-20T23:28:25	Avoid calling xmlSetTreeDoc Create text nodes with xmlNewDocText or set the document directly to avoid xmlSetTreeDoc being called when the node is inserted.
44e9118c	2022-04-08T12:33:17	Prevent integer-overflow in htmlSkipBlankChars() and xmlSkipBlankChars() * HTMLparser.c: (htmlSkipBlankChars): * parser.c: (xmlSkipBlankChars): - Cap the return value at INT_MAX. - The commit range that OSS-Fuzz listed for the fix didn't make any changes to xmlSkipBlankChars(), so it seems like this issue may still exist. Found by OSS-Fuzz Issue 44803.
21561e83	2016-05-20T15:21:43	Mark more static data as `const` Similar to 8f5710379, mark more static data structures with `const` keyword. Also fix placement of `const` in encoding.c. Original patch by Sarah Wilkin.
92bff866	2022-03-29T14:18:31	Fix calls to deprecated init/cleanup functions Only use xmlInitParser/xmlCleanupParser.
96849544	2022-03-22T19:10:51	Revert "Continue to parse entity refs in recovery mode" This reverts commit 84823b86344fb530790a8787b80abf62715ea885 which exposed several other, potentially serious bugs. Fixes #356.
7d02c729	2022-03-06T00:49:02	Fix parser progress checks Testing the current input pointer for modification is unreliable since the input buffer could have been freed and realloced. Check whether the input id and the up-to-date number of bytes consumed match.
84823b86	2022-03-05T22:48:11	Continue to parse entity refs in recovery mode There doesn't seem to be a good reason to abort in xmlParseReference if a well-formedness error was detected. Removing this check allows to parse entity references after an error in recovery mode. Fixes #270.
d99ddd9b	2022-03-05T21:46:40	Improve buffer allocation scheme In most places, we really need the double-it scheme to avoid quadratic behavior. The hybrid scheme still can cause many reallocations and the bounded scheme doesn't seem to provide meaningful protection in xmlreader.c.
ebb17970	2022-03-04T02:31:59	Remove unneeded #includes
776d15d3	2022-03-02T00:29:17	Don't check for standard C89 headers Don't check for - ctype.h - errno.h - float.h - limits.h - math.h - signal.h - stdarg.h - stdlib.h - string.h - time.h Stop including non-standard headers - malloc.h - strings.h
89d9ef3e	2022-03-01T15:14:00	Reset last error in xmlCleanupGlobals Before, we tried to reset the last error in xmlCleanupParser. But if xmlCleanupParser wasn't called from the main thread, this would reset the thread-local error object. xmlCleanupGlobals has access to the error object of the main thread and can reset it reliably.
2489c1d0	2022-02-28T22:42:10	Remove useless __CYGWIN__ checks From what I can tell, some really early Cygwin versions from around 1998-2000 used to erroneously define _WIN32. This was eventually fixed, but these days, the `defined(_WIN32) && !defined(__CYGWIN__)` idiom is unnecessary. Now, we only check for __CYGWIN__ in xmlexports.h when deciding whether to use __declspec.
c41bc10d	2022-02-22T19:57:12	Fix unused variable warnings with disabled features
346c3a93	2022-02-20T18:46:42	Remove elfgcchack.h The same optimization can be enabled with -fno-semantic-interposition since GCC 5. clang has always used this option by default.
9edc20c1	2022-02-07T20:38:30	Fix double counting of CRLF in comments Fixes #151.
96535657	2022-02-07T15:26:33	Make sure to grow input buffer in xmlParseMisc Otherwise, large amount of whitespace could lead to documents not being parsed correctly. Fixes #299.
d85245f9	2022-01-16T21:39:04	Fix regression with PEs in external DTD Fix a regression introduced with commit a28f7d87. In some cases, parameter entity references in external DTDs wouldn't be expanded. Fixes #306.
46c658b0	2021-08-06T08:48:24	move current position before possible calling of ctxt->sax->characters.
fe564967	2021-07-14T14:35:17	Fix memory leak in xmlCreateIOParserCtxt Found by Coverity. https://bugzilla.redhat.com/show_bug.cgi?id=1938806
a7b9f3eb	2021-05-20T13:38:54	fix: avoid segfault at exit when using custom memory functions This extends the fix introduced by 956534e to Windows processes dynamically loading libxml2. Closes #256.
8598060b	2021-05-13T14:55:12	Patch for security issue CVE-2021-3541 This is relapted to parameter entities expansion and following the line of the billion laugh attack. Somehow in that path the counting of parameters was missed and the normal algorithm based on entities "density" was useless.
bfd2f430	2021-05-09T18:56:57	Fix null deref in legacy SAX1 parser Always call nameNsPush instead of namePush. The latter is unused now and should probably be removed from the public API. I can't see how it could be used reasonably from client code and the unprefixed name has always polluted the global namespace. Fixes a null pointer dereference introduced with de5b624f when parsing in SAX1 mode. Found by OSS-Fuzz.
ce00c36e	2021-05-08T21:20:05	Store per-element parser state in a struct Make the parser context's "pushTab" point to an array of structs instead of void pointers. This avoids casting unrelated types to void pointers, improving readability and portability, and allows for more efficient packing. Ultimately, the struct could be extended to include the contents of "nameTab" and "spaceTab", further simplifying the code. Historically, "pushTab" was only used by the push parser (hence the name), so the change to the public headers should be safe. Also remove an unused parameter from xmlParseEndTag2.
de5b624f	2021-05-08T20:21:29	Fix handling of unexpected EOF in xmlParseContent Readd the XML_ERR_TAG_NOT_FINISHED error on unexpected EOF which was removed in commit 62150ed2. This commit also introduced a regression for direct users of xmlParseContent. Unclosed tags weren't checked.
3e80560d	2021-05-07T10:51:38	Fix line numbers in error messages for mismatched tags Commit 62150ed2 introduced a small regression in the error messages for mismatched tags. This typically only affected messages after the first mismatch, but with custom SAX handlers all line numbers would be off. This also fixes line numbers in the SAX push parser which were never handled correctly.
babe7503	2021-05-01T16:53:33	Propagate error in xmlParseElementChildrenContentDeclPriv Check return value of recursive calls to xmlParseElementChildrenContentDeclPriv and return immediately in case of errors. Otherwise, struct xmlElementContent could contain unexpected null pointers, leading to a null deref when post-validating documents which aren't well-formed and parsed in recovery mode. Fixes #243.
c3fd8c42	2021-03-13T17:19:32	Fix exponential behavior with recursive entities Fix another case where only recursion depth was limited, but entities would still be expanded over and over again. The test case discovered by fuzzing only affected parsing in recovery mode with XML_PARSE_RECOVER. Found by OSS-Fuzz.
afad3721	2021-01-31T09:53:56	parser.c: shrink the input buffer when appropriate Fixes GNOME/libxml2#200 Also see discussions at: - GNOME/libxml2#192 - https://gitlab.gnome.org/nwellnhof/libxml2/-/commit/99bda1e - https://github.com/sparklemotion/nokogiri/issues/2132
79301d3d	2020-12-18T12:50:21	Fix timeout when handling recursive entities Abort parsing early to avoid an almost infinite loop in certain error cases involving recursive entities. Found with libFuzzer.
45da175c	2020-12-18T12:14:52	Fix memory leak in xmlParseElementMixedContentDecl Free parsed content if malloc fails to avoid a memory leak. Found with libFuzzer.
c0c26ff2	2020-10-11T16:33:07	parser.c: xmlParseCharData peek behavior fixed wrt newlines Previously, xmlParseCharData and xmlParseComment would consider 0xA to be unhandleable when seen as the first byte of an input chunk, and fall back to xmlParseCharDataComplex and xmlParseCommentComplex, which have different memory and performance characteristics. Fixes GNOME/libxml2#192
7929f057	2020-08-30T10:34:01	Fix SEGV in xmlSAXParseFileWithData Fixes #181.
99fc048d	2020-08-14T14:18:50	Don't use SAX1 if all element handlers are NULL Running xmllint with "--sax --noout" installs a SAX2 handler with all callbacks set to NULL. In this case or similar situations, we don't want to switch to SAX1 parsing.
b82fa3dd	2020-08-09T14:50:46	Fix column number accounting in xmlParse*NameAndCompare Thanks to Frederic Vancraeyveldt for the report.
438e595a	2020-08-09T14:43:53	Stop counting nbChars in parser context The value was inaccurate and never used.
956534e0	2020-08-04T19:27:13	Check for custom free function in global destructor Calling a custom deallocation function in the global destructor could cause all kinds of unexpected problems. See for example https://github.com/sparklemotion/nokogiri/issues/2059 Only clean up if memory is managed with malloc/free.
0e5c4fec	2020-07-13T15:20:45	Reset XML parser input before reporting errors Apply changes to htmlParseChunk() in 13ba5b61 and 3f18e748 to xmlParseChunk().
43a8836c	2020-05-31T18:46:21	Fix rebuilding docs, by hiding __attribute__((...)) behind a macro. When enabled via `./configure --enable-rebuild-docs`, `make -C doc libxml2-api.xml` will invoke apibuild.py to rebuild libxml2-api.xml from the sources. But the code added in 9fa3200cb366c726f7c8ef234282603bb9e8816d made it error out with ``` Parsing ../parser.c Parse Error: parsing type : expecting a name ('Got token ', ('sep', '(')) ('Last token: ', ('sep', '(')) ('Token queue: ', [('name', 'destructor'), ('sep', ')'), ('sep', ')')]) ('Line 14689 end: ', '') ```
a28f7d87	2020-06-10T13:41:13	Never expand parameter entities in text declaration When parsing the text declaration of external DTDs or entities, make sure that parameter entities are not expanded. This also fixes a memory leak in certain error cases. The change to xmlSkipBlankChars assumes that the parser state is maintained correctly when parsing external DTDs or parameter entities, and might expose bugs in the code that were hidden previously. Found by OSS-Fuzz.
2e8cc66d	2020-05-30T15:40:08	xmlParseBalancedChunkMemory must not be called with NULL doc There is no way to avoid memory leaks without a document to hold the namespace list.
a0a8059b	2020-05-30T15:33:03	Revert "Fix memory leak in xmlParseBalancedChunkMemoryRecover" This reverts commit 5a02583c7e683896d84878bd90641d8d9b0d0549. Fixes #161.
9fa3200c	2020-03-31T23:18:25	Call xmlCleanupParser on ELF destruction Fixes #153.
20c60886	2020-03-08T17:19:42	Fix typos Resolves #133.
1a3e584a	2020-01-21T22:12:42	Merge code paths loading external entities Merge xmlParseCtxtExternalEntity into xmlParseExternalEntityPrivate.
f9ea1a24	2020-02-11T16:17:34	Fix copying of entities in xmlParseReference Before, reader mode would end up in a branch that didn't handle entities with multiple children and failed to update ent->last, so the hack copying the "extra" reader data wouldn't trigger. Consequently, some empty nodes in entities are correctly detected now in the test suite. (The detection of empty nodes in entities is still buggy, though.)
c7c526d6	2020-01-13T18:49:01	Fix memory leak when shared libxml.dll is unloaded When a multiple modules (process/plugins) all link to libxml2.dll they will in fact share a single loaded instance of it. It is unsafe for any of them to call xmlCleanupParser, as this would deinitialize the shared state and break others that might still have ongoing use. However, on windows atexit is per-module (rather process-wide), so if used within libxml2 it is possible to register a clean up when all users are done and libxml2.dll is about to actually unload. This allows multiple plugins to link with and share libxml2 without a premature cleanup if one is unloaded, while still cleaning up if all such callers are themselves unloaded.

481d79d4

2022-12-19T15:26:46

entities: Add XML_ENT_PARSED flag To check whether an entity was already parsed, the code previously tested whether "checked" was non-zero or "children" was non-null. The "children" check could be unreliable because an empty entity also results in an empty (NULL) node list. Use a separate flag to make this check more reliable.

4b959ee1

2022-12-01T13:23:09

Remove hacky heuristic from b2dc5675e94aa6b5557ba63f7d66b0f08dd17e4d Checking whether the context is close to the parent context by hardcoding 250 is not portable (I noticed tests were failing on Morello since the value is 288 there due to pointers being 128 bits). Instead we should ensure that the XML_VCTXT_USE_PCTXT flag is not set in cases where the user data is not actually a parser context (or ideally add a separate field but that would be an ABI break. From what I can see in the source, the XML_VCTXT_USE_PCTXT is only set if the userData field points to a valid context, and if this is not the case the flag should be cleared when changing userData rather than relying on the offset between the two. Looking at the history, I think d7cb33cf44aa688f24215c9cd398c1a26f0d25ff fixed most of the need for this workaround, but it looks like there are a few more locations that need updating; This commit changes two more places to set/clear/copy the XML_VCTXT_USE_PCTXT flag, so this heuristic should not be needed anymore. I've also drop two = NULL assignment in xmllint since this is not needed after a call to memset(). There was also an uninitialized vctxt.flags (and other fields) in `xmlShellValidate()`, which I've fixed by adding a memset() call.

c62c0d82

2022-12-01T12:58:11

Correctly relocate internal pointers after realloc() Adding an offset to a deallocated pointer and assuming that it can be dereferenced is undefined behaviour. When running libxml2 on CHERI-enabled systems such as Arm Morello this results in the creation of an out-of-bounds pointer that cannot be dereferenced and therefore crashes at runtime. The effect of this UB is not just limited to architectures such as CHERI, incorrect relocation of pointers after realloc can in fact cause FORTIFY_SOURCE errors with recent GCC: https://developers.redhat.com/articles/2022/09/17/gccs-new-fortification-level

c16fd705

2022-11-25T14:52:37

xpath: Make init function private

53ab3840

2022-11-25T14:26:59

encoding: Make init function private

05c3a458

2022-11-25T14:15:43

tests: Check that xmlInitParser doesn't allocate memory

78c0391b

2022-11-25T13:55:39

parser: Register atexit handler in locked section

ed053c50

2022-11-25T12:27:14

dict: Make init/cleanup functions private

7010d877

2022-11-25T12:06:27

threads: Rework initialization Make init/cleanup functions private. Merge xmlOnceInit into xmlInitThreadsInternal.

9dbf1374

2022-11-24T20:52:57

parser: Make some module init/cleanup functions private

cecd364d

2022-11-24T16:38:47

parser: Don't call *DefaultSAXHandlerInit from xmlInitParser Change the default handler definitions to match the result after calling the initialization functions. This makes sure that no thread-local variables are accessed when calling xmlInitParser.

b1f9c193

2022-11-22T21:39:01

parser: Fix push parser with unterminated CDATA sections Short-lived regression found by OSS-Fuzz.

0e193f0d

2022-11-21T22:09:19

parser: Remove dangerous check in xmlParseCharData If this check succeeds, xmlParseCharData could be called over and over again without making progress, resulting in an infinite loop. It's only important to check for XML_PARSER_EOF which is done later. Related to #441.

94ca36c2

2022-11-21T22:07:11

parser: Restore parser state in xmlParseCDSect Fixes #441.

a8b31e68

2022-11-21T21:35:01

parser: Fix progress check when parsing character data Skip over zero bytes to guarantee progress. Short-lived regression.

c63900fb

2022-11-21T20:11:35

parser: Check terminate flag when push parsing CDATA sections Found by OSS-Fuzz.

a781ee33

2022-11-21T20:10:42

Revert "parser: Add overflow checks to xmlParseLookup functions" This reverts commit bfc55d688427972d093be010a8c2ef265375fcb2. It's better to fix the root cause.

bfc55d68

2022-11-21T18:29:54

parser: Add overflow checks to xmlParseLookup functions Short-lived regression found by OSS-Fuzz.

9e4a46ac

2022-11-20T22:03:08

parser: Merge misc, prolog and epilog cases in push parser

55fb8f72

2022-11-20T15:35:49

parser: Fix push parser with 1-3 byte initial chunk Make sure that ctxt->charset is initialized properly.

68a6518c

2022-11-15T18:23:33

parser: Rewrite push parser boundary checks Remove inaccurate xmlParseCheckTransition check. Remove non-incremental xmlParseGetLasts check. Add functions that check for several boundary constructs more accurately, keeping track of progress in ctxt->checkIndex. Fixes #439.

2059df53

2022-11-14T22:27:58

buf: Deprecate static/immutable buffers

4955e0c9

2022-11-14T20:16:22

io: Don't shrink memory input buffers

117bab22

2022-11-14T20:15:59

parser: Don't call xmlSHRINK from push parser xmlSHRINK also calls xmlParserInputGrow which isn't needed in the push parser.

f00739c1

2022-11-14T00:18:39

parser: Ignore cdata argument in xmlParseCharData It never could be used to parse CDATA sections.

e4f56a72

2022-11-13T23:42:10

parser: Simplify xmlParseConditionalSections

3582b07b

2022-11-13T22:57:32

parser: Fix content parser progress checks This is another attempt at fixing parser progress checks. Instead of relying on in->consumed, which could overflow, change some content parser functions to make guaranteed progress on certain byte sequences.

f7ad338e

2022-11-13T21:59:23

parser: Fix attribute parser progress checks This is another attempt at fixing parser progress checks. Instead of relying on in->consumed, which could overflow, make the attribute parser functions return a NULL name only if they don't make progress.

f61b8a62

2022-11-13T21:47:03

parser: Fix DTD parser progress checks This is another attempt at fixing parser progress checks. Instead of relying on in->consumed, which could overflow, change some DTD parser functions to make guaranteed progress on certain byte sequences.

46cd7d22

2022-11-13T16:30:46

io: Remove xmlInputReadCallbackNop In some cases, for example when using encoders, the read callback was set to NULL, in other cases it was set to xmlInputReadCallbackNop. xmlGROW only tested for xmlInputReadCallbackNop, resulting in errors when parsing large encoded content from memory. Always use a NULL callback for memory buffers to avoid ambiguities. Fixes #262.

a70f7d47

2022-11-04T14:03:31

parser: Fix error message in xmlParseCommentComplex Fixes #421.

afc7e3a7

2022-11-02T16:11:00

malloc-fail: Fix memory leak in xmlParseReference Found with libFuzzer, see #344.

e129c1d1

2022-11-02T16:02:39

malloc-fail: Fix infinite loop in xmlSkipBlankChars Found with libFuzzer, see #344.

865e142c

2022-11-02T15:46:11

malloc-fail: Fix memory leak in xmlCreatePushParserCtxt Found with libFuzzer, see #344.

ffaec758

2022-08-25T17:43:08

Fix integer overflows with XML_PARSE_HUGE Also impose size limits when XML_PARSE_HUGE is set. Limit size of names to XML_MAX_TEXT_LENGTH (10 million bytes) and other content to XML_MAX_HUGE_LENGTH (1 billion bytes). Move some the length checks to the end of the respective loop to make them strict. xmlParseEntityValue didn't have a length limitation at all. But without XML_PARSE_HUGE, this should eventually trigger an error in xmlGROW. Thanks to Maddie Stone working with Google Project Zero for the report!

1a2d8ddc

2022-10-11T13:02:47

parser: Fix potential memory leak in xmlParseAttValueInternal Fix memory leak in case xmlParseAttValueInternal is called with a NULL `len` a non-NULL `alloc` argument. This static function is never called with such arguments internally, but the misleading code should be fixed nevertheless. Fixes #422.

a9669679

2022-09-09T01:44:00

error: Don't use initGenericErrorDefaultFunc The code in xmlInitParser did only set the error handler if it was NULL which should never happen.

59f2f60e

2022-09-02T00:27:57

Remove "runtime debugging" This doesn't seem useful as configuration option.

884e142d

2022-09-01T22:44:02

Fix --with-schemas --without-xpath build xmlXPathInit must be called for schemas.

6843fc72

2022-09-01T02:58:00

Remove or annotate char casts

2cac6269

2022-09-01T03:14:13

Don't use sizeof(xmlChar) or sizeof(char)

ad338ca7

2022-09-01T01:18:30

Remove explicit integer casts Remove explicit integer casts as final operation - in assignments - when passing arguments - when returning values Remove casts - to the same type - from certain range-bound values The main motivation is that these explicit casts don't change the result of operations and only render UBSan's implicit-conversion checks useless. Removing these casts allows UBSan to detect cases where truncation or sign-changes occur unexpectedly. Document some explicit casts as truncating and add a few missing ones.

0f568c0b

2022-08-26T01:22:33

Consolidate private header files Private functions were previously declared - in header files in the root directory - in public headers guarded with IN_LIBXML - in libxml.h - redundantly in source files that used them. Consolidate all private header files in include/private.

48f84ea8

2022-08-25T21:31:08

Remove internal macros from parserInternals.h Replace MOVETO_ENDTAG with code that updates line and column numbers.

58fc89e8

2022-08-25T20:57:30

Deprecate internal parser functions

34a050cd

2022-08-24T16:35:58

Move some HTML functions to correct header file

fd85b566

2022-08-24T15:12:24

Mark more parser functions as deprecated No compiler warnings generated yet.

0e49f882

2022-08-24T05:25:37

Mark most SAX1 functions as deprecated No compiler warnings generated yet.

9a82b94a

2022-08-24T04:21:58

Introduce xmlNewSAXParserCtxt and htmlNewSAXParserCtxt Add API functions to create a parser context with a custom SAX handler without having to mess with ctxt->sax manually.

5b2d07a7

2022-08-20T17:00:50

Use xmlStrlen in *CtxtReadDoc xmlStrlen handles buffers larger than INT_MAX more gracefully.

4ad71c2d

2022-08-20T16:19:34

Fix xmlCtxtReadDoc with encoding xmlCtxtReadDoc used to create an input stream involving xmlNewStringInputStream. This would create a stream without an input buffer, causing problems with encodings (see #34). After commit aab584dc3, an error was returned even with UTF-8 encodings which happened to work before. Make xmlCtxtReadDoc call xmlCtxtReadMemory which doesn't suffer from these issues. Also fix htmlCtxtReadDoc. Fixes #397.

5930fe01

2022-07-18T20:59:45

Reset nsNr in xmlCtxtReset

ca2c91f1

2022-06-28T19:24:14

Fix memory leak in xmlLoadEntityContent error path Free the input stream if pushing it fails. Found by OSS-Fuzz. https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43743

ecba4cbd

2022-06-28T19:22:31

Avoid double-free if malloc fails in inputPush It's the caller's responsibility to free the input stream if this function fails.

3e7b4f37

2022-05-20T23:28:25

Avoid calling xmlSetTreeDoc Create text nodes with xmlNewDocText or set the document directly to avoid xmlSetTreeDoc being called when the node is inserted.

44e9118c

2022-04-08T12:33:17

Prevent integer-overflow in htmlSkipBlankChars() and xmlSkipBlankChars() * HTMLparser.c: (htmlSkipBlankChars): * parser.c: (xmlSkipBlankChars): - Cap the return value at INT_MAX. - The commit range that OSS-Fuzz listed for the fix didn't make any changes to xmlSkipBlankChars(), so it seems like this issue may still exist. Found by OSS-Fuzz Issue 44803.

21561e83

2016-05-20T15:21:43

Mark more static data as `const` Similar to 8f5710379, mark more static data structures with `const` keyword. Also fix placement of `const` in encoding.c. Original patch by Sarah Wilkin.

92bff866

2022-03-29T14:18:31

Fix calls to deprecated init/cleanup functions Only use xmlInitParser/xmlCleanupParser.

96849544

2022-03-22T19:10:51

Revert "Continue to parse entity refs in recovery mode" This reverts commit 84823b86344fb530790a8787b80abf62715ea885 which exposed several other, potentially serious bugs. Fixes #356.

7d02c729

2022-03-06T00:49:02

Fix parser progress checks Testing the current input pointer for modification is unreliable since the input buffer could have been freed and realloced. Check whether the input id and the up-to-date number of bytes consumed match.

84823b86

2022-03-05T22:48:11

Continue to parse entity refs in recovery mode There doesn't seem to be a good reason to abort in xmlParseReference if a well-formedness error was detected. Removing this check allows to parse entity references after an error in recovery mode. Fixes #270.

d99ddd9b

2022-03-05T21:46:40

Improve buffer allocation scheme In most places, we really need the double-it scheme to avoid quadratic behavior. The hybrid scheme still can cause many reallocations and the bounded scheme doesn't seem to provide meaningful protection in xmlreader.c.

ebb17970

2022-03-04T02:31:59

Remove unneeded #includes

776d15d3

2022-03-02T00:29:17

Don't check for standard C89 headers Don't check for - ctype.h - errno.h - float.h - limits.h - math.h - signal.h - stdarg.h - stdlib.h - string.h - time.h Stop including non-standard headers - malloc.h - strings.h

89d9ef3e

2022-03-01T15:14:00

Reset last error in xmlCleanupGlobals Before, we tried to reset the last error in xmlCleanupParser. But if xmlCleanupParser wasn't called from the main thread, this would reset the thread-local error object. xmlCleanupGlobals has access to the error object of the main thread and can reset it reliably.

2489c1d0

2022-02-28T22:42:10

Remove useless __CYGWIN__ checks From what I can tell, some really early Cygwin versions from around 1998-2000 used to erroneously define _WIN32. This was eventually fixed, but these days, the `defined(_WIN32) && !defined(__CYGWIN__)` idiom is unnecessary. Now, we only check for __CYGWIN__ in xmlexports.h when deciding whether to use __declspec.

c41bc10d

2022-02-22T19:57:12

Fix unused variable warnings with disabled features

346c3a93

2022-02-20T18:46:42

Remove elfgcchack.h The same optimization can be enabled with -fno-semantic-interposition since GCC 5. clang has always used this option by default.

9edc20c1

2022-02-07T20:38:30

Fix double counting of CRLF in comments Fixes #151.

96535657

2022-02-07T15:26:33

Make sure to grow input buffer in xmlParseMisc Otherwise, large amount of whitespace could lead to documents not being parsed correctly. Fixes #299.

d85245f9

2022-01-16T21:39:04

Fix regression with PEs in external DTD Fix a regression introduced with commit a28f7d87. In some cases, parameter entity references in external DTDs wouldn't be expanded. Fixes #306.

46c658b0

2021-08-06T08:48:24

move current position before possible calling of ctxt->sax->characters.

fe564967

2021-07-14T14:35:17

Fix memory leak in xmlCreateIOParserCtxt Found by Coverity. https://bugzilla.redhat.com/show_bug.cgi?id=1938806

a7b9f3eb

2021-05-20T13:38:54

fix: avoid segfault at exit when using custom memory functions This extends the fix introduced by 956534e to Windows processes dynamically loading libxml2. Closes #256.

8598060b

2021-05-13T14:55:12

Patch for security issue CVE-2021-3541 This is relapted to parameter entities expansion and following the line of the billion laugh attack. Somehow in that path the counting of parameters was missed and the normal algorithm based on entities "density" was useless.

bfd2f430

2021-05-09T18:56:57

Fix null deref in legacy SAX1 parser Always call nameNsPush instead of namePush. The latter is unused now and should probably be removed from the public API. I can't see how it could be used reasonably from client code and the unprefixed name has always polluted the global namespace. Fixes a null pointer dereference introduced with de5b624f when parsing in SAX1 mode. Found by OSS-Fuzz.

ce00c36e

2021-05-08T21:20:05

Store per-element parser state in a struct Make the parser context's "pushTab" point to an array of structs instead of void pointers. This avoids casting unrelated types to void pointers, improving readability and portability, and allows for more efficient packing. Ultimately, the struct could be extended to include the contents of "nameTab" and "spaceTab", further simplifying the code. Historically, "pushTab" was only used by the push parser (hence the name), so the change to the public headers should be safe. Also remove an unused parameter from xmlParseEndTag2.

de5b624f

2021-05-08T20:21:29

Fix handling of unexpected EOF in xmlParseContent Readd the XML_ERR_TAG_NOT_FINISHED error on unexpected EOF which was removed in commit 62150ed2. This commit also introduced a regression for direct users of xmlParseContent. Unclosed tags weren't checked.

3e80560d

2021-05-07T10:51:38

Fix line numbers in error messages for mismatched tags Commit 62150ed2 introduced a small regression in the error messages for mismatched tags. This typically only affected messages after the first mismatch, but with custom SAX handlers all line numbers would be off. This also fixes line numbers in the SAX push parser which were never handled correctly.

babe7503

2021-05-01T16:53:33

Propagate error in xmlParseElementChildrenContentDeclPriv Check return value of recursive calls to xmlParseElementChildrenContentDeclPriv and return immediately in case of errors. Otherwise, struct xmlElementContent could contain unexpected null pointers, leading to a null deref when post-validating documents which aren't well-formed and parsed in recovery mode. Fixes #243.

c3fd8c42

2021-03-13T17:19:32

Fix exponential behavior with recursive entities Fix another case where only recursion depth was limited, but entities would still be expanded over and over again. The test case discovered by fuzzing only affected parsing in recovery mode with XML_PARSE_RECOVER. Found by OSS-Fuzz.

afad3721

2021-01-31T09:53:56

parser.c: shrink the input buffer when appropriate Fixes GNOME/libxml2#200 Also see discussions at: - GNOME/libxml2#192 - https://gitlab.gnome.org/nwellnhof/libxml2/-/commit/99bda1e - https://github.com/sparklemotion/nokogiri/issues/2132

79301d3d

2020-12-18T12:50:21

Fix timeout when handling recursive entities Abort parsing early to avoid an almost infinite loop in certain error cases involving recursive entities. Found with libFuzzer.

45da175c

2020-12-18T12:14:52

Fix memory leak in xmlParseElementMixedContentDecl Free parsed content if malloc fails to avoid a memory leak. Found with libFuzzer.

c0c26ff2

2020-10-11T16:33:07

parser.c: xmlParseCharData peek behavior fixed wrt newlines Previously, xmlParseCharData and xmlParseComment would consider 0xA to be unhandleable when seen as the first byte of an input chunk, and fall back to xmlParseCharDataComplex and xmlParseCommentComplex, which have different memory and performance characteristics. Fixes GNOME/libxml2#192

7929f057

2020-08-30T10:34:01

Fix SEGV in xmlSAXParseFileWithData Fixes #181.

99fc048d

2020-08-14T14:18:50

Don't use SAX1 if all element handlers are NULL Running xmllint with "--sax --noout" installs a SAX2 handler with all callbacks set to NULL. In this case or similar situations, we don't want to switch to SAX1 parsing.

b82fa3dd

2020-08-09T14:50:46

Fix column number accounting in xmlParse*NameAndCompare Thanks to Frederic Vancraeyveldt for the report.

438e595a

2020-08-09T14:43:53

Stop counting nbChars in parser context The value was inaccurate and never used.

956534e0

2020-08-04T19:27:13

Check for custom free function in global destructor Calling a custom deallocation function in the global destructor could cause all kinds of unexpected problems. See for example https://github.com/sparklemotion/nokogiri/issues/2059 Only clean up if memory is managed with malloc/free.

0e5c4fec

2020-07-13T15:20:45

Reset XML parser input before reporting errors Apply changes to htmlParseChunk() in 13ba5b61 and 3f18e748 to xmlParseChunk().

43a8836c

2020-05-31T18:46:21

Fix rebuilding docs, by hiding __attribute__((...)) behind a macro. When enabled via `./configure --enable-rebuild-docs`, `make -C doc libxml2-api.xml` will invoke apibuild.py to rebuild libxml2-api.xml from the sources. But the code added in 9fa3200cb366c726f7c8ef234282603bb9e8816d made it error out with ``` Parsing ../parser.c Parse Error: parsing type : expecting a name ('Got token ', ('sep', '(')) ('Last token: ', ('sep', '(')) ('Token queue: ', [('name', 'destructor'), ('sep', ')'), ('sep', ')')]) ('Line 14689 end: ', '') ```

a28f7d87

2020-06-10T13:41:13

Never expand parameter entities in text declaration When parsing the text declaration of external DTDs or entities, make sure that parameter entities are not expanded. This also fixes a memory leak in certain error cases. The change to xmlSkipBlankChars assumes that the parser state is maintained correctly when parsing external DTDs or parameter entities, and might expose bugs in the code that were hidden previously. Found by OSS-Fuzz.

2e8cc66d

2020-05-30T15:40:08

xmlParseBalancedChunkMemory must not be called with NULL doc There is no way to avoid memory leaks without a document to hold the namespace list.

a0a8059b

2020-05-30T15:33:03

Revert "Fix memory leak in xmlParseBalancedChunkMemoryRecover" This reverts commit 5a02583c7e683896d84878bd90641d8d9b0d0549. Fixes #161.

9fa3200c

2020-03-31T23:18:25

Call xmlCleanupParser on ELF destruction Fixes #153.

20c60886

2020-03-08T17:19:42

Fix typos Resolves #133.

1a3e584a

2020-01-21T22:12:42

Merge code paths loading external entities Merge xmlParseCtxtExternalEntity into xmlParseExternalEntityPrivate.

f9ea1a24

2020-02-11T16:17:34

Fix copying of entities in xmlParseReference Before, reader mode would end up in a branch that didn't handle entities with multiple children and failed to update ent->last, so the hack copying the "extra" reader data wouldn't trigger. Consequently, some empty nodes in entities are correctly detected now in the test suite. (The detection of empty nodes in entities is still buggy, though.)

c7c526d6

2020-01-13T18:49:01

Fix memory leak when shared libxml.dll is unloaded When a multiple modules (process/plugins) all link to libxml2.dll they will in fact share a single loaded instance of it. It is unsafe for any of them to call xmlCleanupParser, as this would deinitialize the shared state and break others that might still have ongoing use. However, on windows atexit is per-module (rather process-wide), so if used *within* libxml2 it is possible to register a clean up when all users are done and libxml2.dll is about to actually unload. This allows multiple plugins to link with and share libxml2 without a premature cleanup if one is unloaded, while still cleaning up if *all* such callers are themselves unloaded.

kc3-lang/libxml2/parser.c

parser.c

Log