kmx git

Commit	Date	Message
03a238f9	2017-09-29T13:00:50	Use __linux__ macro in generated code
4b4d3d85	2017-10-06T09:00:53	Release of libxml2-2.9.6 * configure.ac doc/xml.html doc/news.html: updated for release
0f3b843b	2017-06-01T23:12:19	Fix XPath stack frame logic Move the calls to xmlXPathSetFrame and xmlXPathPopFrame around in xmlXPathCompOpEvalPositionalPredicate to make sure that the context object on the stack is actually protected. Otherwise, memory corruption can occur when calling sloppily coded XPath extension functions. Fixes bug 783160.
3157cf4e	2017-09-20T16:13:29	Report undefined XPath variable error message Commit c851970 removed a redundant error message if XPath evaluation failed. This uncovered a case where an undefined XPath variable error wasn't reported correctly. Thanks to Petr Pisar for the report. Fixes bug 787941.
b90d8989	2017-09-19T15:45:35	Fix regression with librsvg Instead of using xmlCreateIOParserCtxt, librsvg pushes its own xmlParserInput on top of a memory push parser. This incorrect use of the API confuses several parser checks and, since 2.9.5, completely breaks documents with internal subsets. Work around the problem with internal subsets. Thanks to Petr Sumbera for the report: https://mail.gnome.org/archives/xml/2017-September/msg00011.html Also see https://bugzilla.gnome.org/show_bug.cgi?id=787895
2cdaaab1	2017-09-14T21:30:51	Change preprocessor OS tests to __linux__ "linux" without leading underscores is deprecated and less reliable.
abbda93c	2017-09-11T01:14:16	Handle more invalid entity values in recovery mode In attribute content, don't emit entity references if there are problems with the entity value. Otherwise some illegal entity values like <!ENTITY a '&#x123456789;'> would later cause problems like integer overflow. Make xmlStringLenDecodeEntities return NULL on more error conditions including invalid char refs and errors from recursive calls. Remove some fragile error checks based on lastError that shouldn't be needed now. Clear the entity content in xmlParseAttValueComplex if an error was found. Found by OSS-Fuzz. Should fix bug 783052. Also see https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3343
c709f3f2	2017-09-07T19:52:39	Fix structured validation errors Also pass 'str2'. Fixes bug 777473.
07e227ed	2017-09-07T18:55:46	Fix memory leak in LZMA decompressor
e2a9122b	2017-09-07T18:36:01	Set memory limit for LZMA decompression Otherwise malicious LZMA compressed files could consume large amounts of memory when decompressed. According to the xz man page, files compressed with `xz -9` currently require 65 MB to decompress, so set the limit to 100 MB. Should fix bug 786696.
0fcab658	2017-09-07T18:25:11	Handle illegal entity values in recovery mode Make xmlParseEntityValue always return NULL on error. Otherwise some illegal entity values like <!ENTITY e '&%#4294967298;'> would later cause problems like integer overflow. Found by OSS-Fuzz. Should fix bug 783052. Also see https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=592 https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2732
ca8635bf	2017-09-07T15:46:12	Fix debug dump of streaming XPath expressions The debug function xmlXPathDebugDumpCompExpr would crash when called with a "streaming" expression. Fixes bug 787368.
3b3d7502	2017-09-06T01:31:31	Fix memory leak in nanoftp Found by OSS-Fuzz (testcase 5487741645094912). Free the user and password strings in xmlNanoFTPFreeCtxt. Also see https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2727
83fb4119	2017-09-05T23:45:04	Fix memory leaks in SAX1 parser Found by OSS-Fuzz. I could only reproduce this with the (obsolete) SAX1 parser. One leak is caused by duplicate namespaced attribute names and can be reproduced in memory mode (testcase 4556417027538944): $ cat file <d xmlns:a="ns" a:x="v" xmlns:b="ns" b:x="v"/> $ xmllint --sax1 --memory file The other is caused by ATTLISTs with a normalized default for "xmlns" if they're processed after the entity recursion limit was hit (testcase 5580750034305024). $ cat file <!DOCTYPE d [ <!ENTITY a '<d>&a;'> <!ATTLIST d xmlns NMTOKEN 't'> ]> <d>&a; $ xmllint --sax1 --valid file Also see https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2461
2960178f	2017-09-04T15:38:47	Release of libxml2-2.9.5 * configure.ac, doc/xslt.html: updated for the release * doc/*, python/setup.py, testapi.c: regenerated
69936b12	2017-08-30T14:16:01	Revert "Print error messages for truncated UTF-8 sequences" This reverts commit 79c8a6b which caused a serious regression in streaming mode. Also reverts part of commit 52ceced "Fix infinite loops with push parser in recovery mode". Fixes bug 786554.
e29e50e5	2017-08-29T12:40:54	Updating the spec file to reflect Fedora 24 new python subpackages namings
8f571037	2017-08-28T21:27:03	Add const in five places to move 1 KiB to .rdata xmlUnicodeBlocks is logically const but was not marked as such. This fixes that, thus moving it to the read-only data segment.
3daee3f1	2017-08-28T21:12:14	Problem resolving relative URIs Raised by Matthias Pigulla <mp@webfactory.de> In a nutshell we had that bug on URI composition after some fixes in the area of localhost empty shortcuts : ./testURI --base file:///some/where file Without patch: file:/some/file With patch: file:///some/file
f19385a5	2017-08-28T20:40:19	Fix a couple of misleading indentation errors Raised by gcc as potential error, no semantic change needed but fixed the indentation
dbb828fe	2017-08-28T20:38:53	Fix missing part of comment for function xmlXPathEvalExpression()
56ec9cef	2017-08-28T20:15:38	Adding README.zOS to list of extra files for the release
afb0f93c	2017-08-28T17:45:47	Revert "Allow build on Visual Studio 2017 community edition for WIN32 on a 64 bit machine." This reverts commit 605899cf22d185f537eb5efb97a55be30467eb17.
605899cf	2017-08-16T12:22:44	Allow build on Visual Studio 2017 community edition for WIN32 on a 64 bit machine. This patch hold the necessary changes to build libxml2 with Visual Studio 2017 for WIN32 on a 64 bit machine. Today 64 bit windows installation are very common, and Visual Studio 2017 community is the most recent version i have access to. I have chosen 32 bit and SDK 8.1 to be as close as possible to Igor Zlatkovic work.
1056e82e	2017-08-28T14:45:19	Description of work needed to compile on zOS building on zOS requires Makefile surgery that we cannot carrry in the common branch, this documents the changes needed to compile on that platform
454e397e	2017-08-28T14:30:43	Porting libxml2 on zOS encoding of code First set of patches for zOS - entities.c parser.c tree.c xmlschemas.c xmlschemastypes.c xpath.c xpointer.c: ask conversion of code to ISO Latin 1 to avoid having the compiler assume EBCDIC codepoint for characters. - xmlmodule.c: make sure we have support for modules - xmlIO.c: zOS path names are special avoid dsome of the expectstions from Unix/Windows
3aca7f31	2017-08-21T13:06:29	Fix unwanted warnings when switching encodings Revert part of commit 46dc989 "Don't switch encoding for internal parameter entities" that caused spurious warnings. Fixes bug 786267.
27f310d4	2017-07-27T14:06:44	Fix signature of xmlSchemaAugmentImportedIDC this is used in a callback which will pass a name, the name is ignored but it's best to have the signature of the function match, pointed out by Claude Petit * xmlschemas.c: fix xmlSchemaAugmentImportedIDC() signature no functional change
d1f90624	2017-07-27T11:20:02	small changes for OS/400 OS/400 version V5R3 is not supported by IBM anymore. In addition, the iSeries system I have here for development has been changed and the new system is not able to compile for an OS version lower than V6R1. Thus I made some updates to the libxml2 os400 scripts accordingly: - Oldest supported OS version is now V6R1. - Adjust ILE/RPG wrappers comments. - Update copyright year range. - Do not log compiler informational messages.
899a5d9f	2017-07-25T14:59:49	Detect infinite recursion in parameter entities When expanding a parameter entity in a DTD, infinite recursion could lead to an infinite loop or memory exhaustion. Thanks to Wei Lei for the first of many reports. Fixes bug 759579.
8a0c6698	2017-07-04T17:13:06	Fix NULL pointer deref in xmlFAParseCharClassEsc Found with libFuzzer.
52ceced6	2017-07-01T17:49:30	Fix infinite loops with push parser in recovery mode Make sure that the input pointer advances in case of errors. Otherwise, the push parser can loop infinitely. Found with libFuzzer.
fb56f80e	2017-07-04T18:38:03	Heap-buffer-overflow read of size 1 in xmlFAParsePosCharGroup Credit to OSS-Fuzz. Add a check to xmlFAParseCharRange() for the end of the buffer to prevent reading past the end of it. This fixes Bug 784017.
f435365b	2017-06-20T16:19:33	Send xmllint usage error to stderr Fixes bug 781459.
3eef3f39	2017-06-20T16:13:57	Fix NULL deref in xmlParseExternalEntityPrivate If called from xmlParseExternalEntity, oldctxt is NULL which leads to a NULL deref if an error occurs. This only affects external code that calls xmlParseExternalEntity. Patch from David Kilzer with minor changes. Fixes bug 780159.
872fea94	2017-06-19T00:24:12	Get rid of "blanks wrapper" for parameter entities Now that replacement of parameter entities goes exclusively through xmlSkipBlankChars, we can account for the surrounding space characters there and remove the "blanks wrapper" hack.
d9e43c7d	2017-06-19T18:01:23	Make sure not to call IS_BLANK_CH when parsing the DTD This is required to get rid of the "blanks wrapper" hack. Checking the return value of xmlSkipBlankChars is more efficient, too.
453dff1e	2017-06-19T17:55:20	Remove unnecessary calls to xmlPopInput It's enough if xmlPopInput is called from xmlSkipBlankChars. Since the replacement text of a parameter entity is surrounded with space characters, that's the only place where the replacement can end in a well-formed document. This is also required to get rid of the "blanks wrapper" hack.
aa267cd1	2017-06-18T23:29:51	Simplify handling of parameter entity references There are only two places where parameter entity references must be handled. For the internal subset in xmlParseInternalSubset. For the external subset or content from other external PEs in xmlSkipBlankChars. Make sure that xmlSkipBlankChars skips over sequences of PEs and whitespace. Rely on xmlSkipBlankChars instead of calling xmlParsePEReference directly when in the external subset or a conditional section. xmlParserHandlePEReference is unused now.
24246c76	2017-06-20T12:56:36	Fix xmlHaltParser Pop all extra input streams before resetting the input. Otherwise, a call to xmlPopInput could make input available again. Also set input->end to input->cur. Changes the test output for some error tests. Unfortunately, some fuzzed test cases were added to the test suite without manual cleanup. This makes it almost impossible to review the impact of later changes on the test output.
e5107772	2017-06-19T15:32:56	Fix pathological performance when outputting charrefs If a character can't be represented in the output encoding, it is converted to a character reference. This used to to replace the character in the input stream by calling xmlBufAddHead or xmlBufferAddHead. These functions shifted the entire input array around, leading to quadratic performance when converting a run of non-representable characters. This is most pronounced when dumping to memory. Output the charref directly instead. Found with libFuzzer.
c9ccbd6a	2017-06-19T14:57:43	Deduplicate code in encoding.c Introduce static functions xmlEncInputChunk and xmlEncOutputChunk that handle the internal/iconv/ICU branching.
5a0ae66d	2017-06-17T23:20:38	Documentation fixes Fixes bug 347465, bug 599433, bug 624550, bug 698253.
8bbe4508	2017-06-17T16:15:09	Spelling and grammar fixes Fixes bug 743172, bug 743489, bug 769632, bug 782400 and a few other misspellings.
576912fa	2017-06-17T15:59:13	Make HTML parser functions take const pointers The 'cur' parameter of htmlParseDoc and htmlSAXParseDoc should be 'const xmlChar *'. Fixes bug 770650.
988a5a3b	2017-06-17T15:44:50	Build test programs only when needed Add test programs to 'check_PROGRAMS' instead of 'noinst_PROGRAMS'. Fixes bug 760457.
b9b4b6b5	2017-06-17T15:40:45	Fix doc/examples/index.py In my previous commit that silenced some test output I didn't realize that doc/examples/Makefile.am was autogenerated. Also make index.py output deterministic by sorting the glob results.
1f09aea2	2017-06-17T15:05:34	Fix compiler warnings in threads.c Use '#pragma weak' to declare weak functions.
629e47e7	2017-06-17T14:51:10	Fix empty-body warning in nanohttp.c
1a595cd1	2017-06-17T14:13:51	Fix cast-align warnings - Suppress warnings in xmlmemory.c by casting to 'void *'. - Remove unneeded cast in xmlschemas.c that caused a macro precedence error. - Add dummy fields to short structs in xmlschemas.c. This increases the size of the structs, but I can't see a better solution without using C11's _Alignof operator. There are still a couple of cast-align warnings in encoding.c. These are legitimate portability issues that can't be fixed without reworking the conversion functions.
81c01ee9	2017-06-17T14:12:53	Fix unused-parameter warnings
1ce1f785	2017-06-17T14:11:03	Fix invalid-source-encoding warnings in testWriter.c Use hex escapes instead of binary data in source file.
5f440d8c	2017-06-12T14:32:34	Rework entity boundary checks Make sure to finish all entities in the internal subset. Nevertheless, readd a sanity check in xmlParseStartTag2 that was lost in my previous commit. Also add a sanity check in xmlPopInput. Popping an input unexpectedly was the source of many recent memory bugs. The check doesn't mitigate such issues but helps with diagnosis. Always base entity boundary checks on the input ID, not the input pointer. The pointer could have been reallocated to the old address. Always throw a well-formedness error if a boundary check fails. In a few places, a validity error was thrown. Fix a few error codes and improve indentation.
46dc9890	2017-06-08T02:24:56	Don't switch encoding for internal parameter entities This is only needed for external entities. Trying to switch the encoding for internal entities could also cause a memory leak in recovery mode.
03904159	2017-06-05T21:16:00	Merge duplicate code paths handling PE references xmlParsePEReference is essentially a subset of xmlParserHandlePEReference, so make xmlParserHandlePEReference call xmlParsePEReference. The code paths in these functions differed slighty, but the code from xmlParserHandlePEReference seems more solid and tested.
dbaab1f3	2017-06-16T21:38:57	Test SAX2 callbacks with entity substitution This detects regressions like bug 760367.
3f0627a1	2017-06-16T21:30:42	Fix duplicate SAX callbacks for entity content Reset 'was_checked' to prevent entity from being parsed twice and SAX callbacks being invoked twice if XML_PARSE_NOENT was set. This regressed in version 2.9.3 and caused problems with WebKit. Fixes bug 760367.
01a4b81a	2017-06-16T21:27:47	Support catalog and threads tests under --without-sax1
3939178e	2017-06-12T20:06:37	Treat URIs with scheme as absolute in C14N Fixes bug 783656.
67f9f9d6	2017-06-12T19:25:01	Misc fixes for 'make tests' - Silence test output. - Clean up after doc/examples tests. - Adjust expected output for script tests. - Add missing results for relaxng/pattern3 There are still two test failures I can't comment on: - regexp/bug316338 - schemas/any4_0
0b2d5c48	2017-06-12T19:10:04	Initialize keepBlanks in HTML parser This caused failures in the HTML push tests but the fix required to change the expected output of the HTML SAX tests.
85c112a0	2017-06-12T18:26:11	Add test cases for bug 758518 test/HTML/758518-entity.html exposed a bug in pushParseTest() in runtest.c which assumed that an input file was at least 4 bytes long. That test case is only 3 bytes, so we now take the minimum of 4 bytes or the length of the test input. We also now use 'chunkSize' in place of the hard-coded value '1024' later in the function.
ed48d65b	2017-06-11T17:12:21	Fix copy-paste errors in error messages Thanks to David Kilzer for the review.
9a366a37	2017-06-11T12:40:01	Fix compiler warning in htmlParseElementInternal
f39e3be0	2017-06-11T12:35:59	Fix sanity check in htmlParseNameComplex - (cur - len) can overflow. - Throw an internal error. Fixes bug 780077.
79c8a6b1	2017-06-10T17:01:27	Print error messages for truncated UTF-8 sequences Before, truncated UTF-8 sequences at the end of a file were treated as EOF. Create an error message containing the offending bytes. xmlStringCurrentChar would also print characters from the input stream, not the string it's working on.
fb2f518c	2017-06-10T17:06:16	Fix potential infinite loop in xmlStringLenDecodeEntities Make sure that xmlParseStringPEReference advances the "str" pointer even if the parser was stopped. Otherwise xmlStringLenDecodeEntities can loop infinitely.
4ba8cc85	2017-06-10T02:33:58	Remove useless check in xmlParseAttributeListDecl Since we already successfully parsed the attribute name and other items, it is guaranteed that we made progress in the input stream. Comparing the input pointer to a previous value also looks fragile to me. What if the input buffer was reallocated and the new "cur" pointer happens to be the same as the old one? There are a couple of similar checks which also take "consumed" into account. This seems to be safer but I'm not convinced that it couldn't lead to false alarms in rare situations.
f9e7997e	2017-06-10T00:33:07	Reset parser input pointers on encoding failure Call xmlBufResetInput before bailing out if switching the encoding fails. Otherwise, the input pointers are left in an invalid state. This would typically lead to an internal error in xmlGROW but could also cause other unforeseen problems.
bedbef80	2017-06-09T15:10:13	Fix memory leak in xmlParseEntityDecl error path When parsing the entity value, it can happen that an external entity with an unsupported encoding is loaded and the parser is stopped. This would lead to a memory leak. A custom SAX callback could also stop the parser. Found with libFuzzer and ASan.
94f6ce83	2017-06-08T22:36:09	Allow zero sized memory input buffers Useful for a fuzz target I'm working on.
91e54967	2017-06-08T18:25:30	Fix xmlBuildRelativeURI for URIs starting with './' If the relative URI started with './', the 'pos' index was increased which also affected indexing into the base path. Aside from producing wrong results, this could also lead to a heap overread of the base path buffer. The data read from beyond the buffer was only compared to some char values, so this is mostly harmless. Inside libxml2, xmlBuildRelativeURI is only called from xinclude.c. Found with libFuzzer and ASan.
45ce1ee3	2017-06-09T15:27:25	Add TODO comment in xmlSwitchEncoding It would be nice if we could recover from unsupported encodings in external entities.
92b9e8c8	2017-06-06T12:56:28	Fix type confusion in xmlValidateOneNamespace Comment out code that casts xmlNsPtr to xmlAttrPtr. ID types on namespace declarations make no practical sense anyway. Fixes bug 780228. Found with libFuzzer and ASan.
8c82f5de	2017-06-07T18:32:49	Fix memory leak in xmlStringLenGetNodeList Avoid expanding the entity recursively. Use the same prevention mechanism as in xmlStringGetNodeList. xmlStringGetNodeList on the other hand wasn't fixing up the 'last' pointer. I think the memory leak can only be triggered in recovery mode. Found with libFuzzer and ASan.
94691dc8	2017-06-07T16:47:36	Fix NULL pointer deref in xmlDumpElementContent Can only be triggered in recovery mode. Fixes bug 758422 (CVE-2017-5969).
362b3229	2017-06-07T16:35:28	Fix memory leak in xmlBufAttrSerializeTxtContent The serializer sets doc->encoding to a temporary value and restores the original value when it's done. This overwrites the encoding value set in xmlBufAttrSerializeTxtContent, causing a memory leak. Don't mess with doc->encoding if invalid UTF-8 is encountered. Found with libFuzzer and ASan.
0db8dc9d	2017-06-07T16:07:33	Stop parser on unsupported encodings Otherwise, the push parser can loop infinitely in recover mode. Found with libFuzzer.
030b1f7a	2017-06-06T15:53:42	Revert "Add an XML_PARSE_NOXXE flag to block all entities loading even local" This reverts commit 2304078555896cf1638c628f50326aeef6f0e0d0. The new flag doesn't work and the change even broke the XML_PARSE_NONET option.
897dffba	2017-06-06T13:21:14	Check for integer overflow in memory debug code Fixes bug 783026. Thanks to Pranjal Jumde for the report.
932cc989	2017-06-03T02:01:29	Fix buffer size checks in xmlSnprintfElementContent xmlSnprintfElementContent failed to correctly check the available buffer space in two locations. Fixes bug 781333 (CVE-2017-9047) and bug 781701 (CVE-2017-9048). Thanks to Marcel Böhme and Thuan Pham for the report.
e2663054	2017-06-05T15:37:17	Fix handling of parameter-entity references There were two bugs where parameter-entity references could lead to an unexpected change of the input buffer in xmlParseNameComplex and xmlDictLookup being called with an invalid pointer. Percent sign in DTD Names ========================= The NEXTL macro used to call xmlParserHandlePEReference. When parsing "complex" names inside the DTD, this could result in entity expansion which created a new input buffer. The fix is to simply remove the call to xmlParserHandlePEReference from the NEXTL macro. This is safe because no users of the macro require expansion of parameter entities. - xmlParseNameComplex - xmlParseNCNameComplex - xmlParseNmtoken The percent sign is not allowed in names, which are grammatical tokens. - xmlParseEntityValue Parameter-entity references in entity values are expanded but this happens in a separate step in this function. - xmlParseSystemLiteral Parameter-entity references are ignored in the system literal. - xmlParseAttValueComplex - xmlParseCharDataComplex - xmlParseCommentComplex - xmlParsePI - xmlParseCDSect Parameter-entity references are ignored outside the DTD. - xmlLoadEntityContent This function is only called from xmlStringLenDecodeEntities and entities are replaced in a separate step immediately after the function call. This bug could also be triggered with an internal subset and double entity expansion. This fixes bug 766956 initially reported by Wei Lei and independently by Chromium's ClusterFuzz, Hanno Böck, and Marco Grassi. Thanks to everyone involved. xmlParseNameComplex with XML_PARSE_OLD10 ======================================== When parsing Names inside an expanded parameter entity with the XML_PARSE_OLD10 option, xmlParseNameComplex would call xmlGROW via the GROW macro if the input buffer was exhausted. At the end of the parameter entity's replacement text, this function would then call xmlPopInput which invalidated the input buffer. There should be no need to invoke GROW in this situation because the buffer is grown periodically every XML_PARSER_CHUNK_SIZE characters and, at least for UTF-8, in xmlCurrentChar. This also matches the code path executed when XML_PARSE_OLD10 is not set. This fixes bugs 781205 (CVE-2017-9049) and 781361 (CVE-2017-9050). Thanks to Marcel Böhme and Thuan Pham for the report. Additional hardening ==================== A separate check was added in xmlParseNameComplex to validate the buffer size.
7482f41f	2017-06-01T22:00:19	Check for integer overflow in xmlXPathFormatNumber Check for overflow before casting double to int. Found with afl-fuzz and UBSan.
863b5792	2017-06-01T17:53:16	Make Travis print UBSan stacktraces
a2b53178	2017-06-01T01:21:27	Add .travis.yml For now this is mainly useful if you work on a fork of the libxml2 mirror on GitHub: https://github.com/GNOME/libxml2 Start with two build setups: - GCC with as many GNU extensions disabled as possible, trying to emulate a C89 compiler on a POSIX system. - clang with ASan and UBSan. The Python tests don't set an exit code, so Travis won't detect failures. The same goes for "make tests", but we only run "make check" anyway.
83212ff4	2017-06-01T12:42:03	Fix expected error output in Python tests
855c19ef	2017-06-01T01:04:08	Avoid reparsing in xmlParseStartTag2 The code in xmlParseStartTag2 must handle the case that the input buffer was grown and reallocated which can invalidate pointers to attribute values. Before, this was handled by detecting changes of the input buffer "base" pointer and, in case of a change, jumping back to the beginning of the function and reparsing the start tag. The major problem of this approach is that whether an input buffer is reallocated is nondeterministic, resulting in seemingly random test failures. See the mailing list thread "runtest mystery bug: name2.xml error case regression test" from 2012, for example. If a reallocation was detected, the code also made no attempts to continue parsing in case of errors which makes a difference in the lax "recover" mode. Now we store the current input buffer "base" pointer for each (not separately allocated) attribute in the namespace URI field, which isn't used until later. After the whole start tag was parsed, the pointers to the attribute values are reconstructed using the offset between the new and the old input buffer. This relies on arithmetic on dangling pointers which is technically undefined behavior. But it seems like the easiest and most efficient fix and a similar approach is used in xmlParserInputGrow. This changes the error output of several tests, typically making it more verbose because we try harder to continue parsing in case of errors. (Another possible solution is to check not only the "base" pointer but the size of the input buffer as well. But this would result in even more reparsing.)
07b7428b	2017-06-01T00:19:14	Simplify control flow in xmlParseStartTag2 Remove some goto labels and deduplicate a bit of code after handling namespaces. Before: loop { parseAttribute if (ok) { if (defaultNamespace) { handleDefaultNamespace if (error) goto skip_default_ns; handleDefaultNamespace skip_default_ns: freeAttr nextAttr continue; } if (namespace) { handleNamespace if (error) goto skip_ns; handleNamespace skip_ns: freeAttr nextAttr; continue; } handleAttr } else { freeAttr } nextAttr } After: loop { parseAttribute if (!ok) goto next_attr; if (defaultNamespace) { handleDefaultNamespace if (error) goto next_attr; handleDefaultNamespace } else if (namespace) { handleNamespace if (error) goto next_attr; handleNamespace } else { handleAttr } next_attr: freeAttr nextAttr }
ac9a4560	2017-05-31T19:06:30	Disable LeakSanitizer when running API tests The autogenerated API tests leak memory.
ff34ba3e	2017-05-31T18:53:45	Avoid out-of-bound array access in API tests The API tests combine string buffers with arbitrary length values which makes ASan detect out-of-bound array accesses. Even without ASan, this could lead to unwanted test failures. Add a check for "len", "size", and "start" arguments, assuming they apply to the nearest char pointer. Skip the test if they exceed the buffer size. This is a somewhat naive heuristic but it seems to work well.
34e44567	2017-05-31T16:48:27	Fix undefined behavior in xmlRegExecPushStringInternal It's stupid, but the behavior of memcpy(NULL, NULL, 0) is undefined.
47496724	2017-05-31T16:46:39	Avoid spurious UBSan errors in parser.c If available, use a C99 flexible array member to avoid spurious UBSan errors.
f4029cd4	2016-04-21T16:37:26	Check XPath exponents for overflow Avoid undefined behavior and wrong results with huge exponents. Found with afl-fuzz and UBSan.
a58331a6	2017-05-29T21:02:21	Check for overflow in xmlXPathIsPositionalPredicate Avoid undefined behavior when casting from double to int. Found with afl-fuzz and UBSan.
a851868a	2017-05-29T20:14:42	Parse small XPath numbers more accurately Don't count leading zeros towards the fraction size limit. This allows to parse numbers like 0.0000000000000000000000000000000000000000000000000000000001 which is the only standard-conformant way to represent such numbers, as scientific notation isn't allowed in XPath 1.0. (It is allowed in XPath 2.0 and in libxml2 as an extension, though.) Overall accuracy is still bad, see bug 783238.
4bebb030	2016-04-21T13:41:09	Rework XPath rounding functions Use the C library's floor and ceil functions. The old code was overly complicated for no apparent reason and could result in undefined behavior when handling NaNs (found with afl-fuzz and UBSan). Fix wrong comment in xmlXPathRoundFunction. The implementation was already following the spec and rounding half up.
43f50f4d	2017-05-26T23:01:54	Fix white space in test output Quote echoed variable to avoid newlines being converted to space.
40f58521	2017-05-26T20:16:35	Fix axis traversal from attribute and namespace nodes When traversing the "preceding" axis from an attribute node, we must first go up to the attribute's containing element. Otherwise, text children of other attributes could be returned. This made it possible to hit a code path in xmlXPathNextAncestor which contained another bug: The attribute node was initialized with the context node instead of the current node. Normally, this code path is only hit via xmlXPathNextAncestorOrSelf in which case the current and context node are the same. The combination of the two bugs could result in an infinite loop, found with libFuzzer. Traversing the "following" and the "preceding" axis from namespace nodes should be handled similarly. This wasn't supported at all previously.
a07a4e96	2017-05-27T17:04:12	Fix spurious error message Commit c851970 introduced a spurious error message when evaluating XPath expressions with xmlXPathCompiledEvalToBoolean.
aed407c1	2017-05-25T16:57:14	Check for trailing characters in XPath expressions earlier Move the check for trailing characters from xmlXPathEval to xmlXPathEvalExpr. Otherwise, a valid portion of a syntactically invalid expression would be evaluated before returning an error.

03a238f9

2017-09-29T13:00:50

Use __linux__ macro in generated code

4b4d3d85

2017-10-06T09:00:53

Release of libxml2-2.9.6 * configure.ac doc/xml.html doc/news.html: updated for release

0f3b843b

2017-06-01T23:12:19

Fix XPath stack frame logic Move the calls to xmlXPathSetFrame and xmlXPathPopFrame around in xmlXPathCompOpEvalPositionalPredicate to make sure that the context object on the stack is actually protected. Otherwise, memory corruption can occur when calling sloppily coded XPath extension functions. Fixes bug 783160.

3157cf4e

2017-09-20T16:13:29

Report undefined XPath variable error message Commit c851970 removed a redundant error message if XPath evaluation failed. This uncovered a case where an undefined XPath variable error wasn't reported correctly. Thanks to Petr Pisar for the report. Fixes bug 787941.

b90d8989

2017-09-19T15:45:35

Fix regression with librsvg Instead of using xmlCreateIOParserCtxt, librsvg pushes its own xmlParserInput on top of a memory push parser. This incorrect use of the API confuses several parser checks and, since 2.9.5, completely breaks documents with internal subsets. Work around the problem with internal subsets. Thanks to Petr Sumbera for the report: https://mail.gnome.org/archives/xml/2017-September/msg00011.html Also see https://bugzilla.gnome.org/show_bug.cgi?id=787895

2cdaaab1

2017-09-14T21:30:51

Change preprocessor OS tests to __linux__ "linux" without leading underscores is deprecated and less reliable.

abbda93c

2017-09-11T01:14:16

Handle more invalid entity values in recovery mode In attribute content, don't emit entity references if there are problems with the entity value. Otherwise some illegal entity values like <!ENTITY a '&#x123456789;'> would later cause problems like integer overflow. Make xmlStringLenDecodeEntities return NULL on more error conditions including invalid char refs and errors from recursive calls. Remove some fragile error checks based on lastError that shouldn't be needed now. Clear the entity content in xmlParseAttValueComplex if an error was found. Found by OSS-Fuzz. Should fix bug 783052. Also see https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3343

c709f3f2

2017-09-07T19:52:39

Fix structured validation errors Also pass 'str2'. Fixes bug 777473.

07e227ed

2017-09-07T18:55:46

Fix memory leak in LZMA decompressor

e2a9122b

2017-09-07T18:36:01

Set memory limit for LZMA decompression Otherwise malicious LZMA compressed files could consume large amounts of memory when decompressed. According to the xz man page, files compressed with `xz -9` currently require 65 MB to decompress, so set the limit to 100 MB. Should fix bug 786696.

0fcab658

2017-09-07T18:25:11

Handle illegal entity values in recovery mode Make xmlParseEntityValue always return NULL on error. Otherwise some illegal entity values like <!ENTITY e '&%#4294967298;'> would later cause problems like integer overflow. Found by OSS-Fuzz. Should fix bug 783052. Also see https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=592 https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2732

ca8635bf

2017-09-07T15:46:12

Fix debug dump of streaming XPath expressions The debug function xmlXPathDebugDumpCompExpr would crash when called with a "streaming" expression. Fixes bug 787368.

3b3d7502

2017-09-06T01:31:31

Fix memory leak in nanoftp Found by OSS-Fuzz (testcase 5487741645094912). Free the user and password strings in xmlNanoFTPFreeCtxt. Also see https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2727

83fb4119

2017-09-05T23:45:04

Fix memory leaks in SAX1 parser Found by OSS-Fuzz. I could only reproduce this with the (obsolete) SAX1 parser. One leak is caused by duplicate namespaced attribute names and can be reproduced in memory mode (testcase 4556417027538944): $ cat file <d xmlns:a="ns" a:x="v" xmlns:b="ns" b:x="v"/> $ xmllint --sax1 --memory file The other is caused by ATTLISTs with a normalized default for "xmlns" if they're processed after the entity recursion limit was hit (testcase 5580750034305024). $ cat file <!DOCTYPE d [ <!ENTITY a '<d>&a;'> <!ATTLIST d xmlns NMTOKEN 't'> ]> <d>&a; $ xmllint --sax1 --valid file Also see https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2461

2960178f

2017-09-04T15:38:47

Release of libxml2-2.9.5 * configure.ac, doc/xslt.html: updated for the release * doc/*, python/setup.py, testapi.c: regenerated

69936b12

2017-08-30T14:16:01

Revert "Print error messages for truncated UTF-8 sequences" This reverts commit 79c8a6b which caused a serious regression in streaming mode. Also reverts part of commit 52ceced "Fix infinite loops with push parser in recovery mode". Fixes bug 786554.

e29e50e5

2017-08-29T12:40:54

Updating the spec file to reflect Fedora 24 new python subpackages namings

8f571037

2017-08-28T21:27:03

Add const in five places to move 1 KiB to .rdata xmlUnicodeBlocks is logically const but was not marked as such. This fixes that, thus moving it to the read-only data segment.

3daee3f1

2017-08-28T21:12:14

Problem resolving relative URIs Raised by Matthias Pigulla <mp@webfactory.de> In a nutshell we had that bug on URI composition after some fixes in the area of localhost empty shortcuts : ./testURI --base file:///some/where file Without patch: file:/some/file With patch: file:///some/file

f19385a5

2017-08-28T20:40:19

Fix a couple of misleading indentation errors Raised by gcc as potential error, no semantic change needed but fixed the indentation

dbb828fe

2017-08-28T20:38:53

Fix missing part of comment for function xmlXPathEvalExpression()

56ec9cef

2017-08-28T20:15:38

Adding README.zOS to list of extra files for the release

afb0f93c

2017-08-28T17:45:47

Revert "Allow build on Visual Studio 2017 community edition for WIN32 on a 64 bit machine." This reverts commit 605899cf22d185f537eb5efb97a55be30467eb17.

605899cf

2017-08-16T12:22:44

Allow build on Visual Studio 2017 community edition for WIN32 on a 64 bit machine. This patch hold the necessary changes to build libxml2 with Visual Studio 2017 for WIN32 on a 64 bit machine. Today 64 bit windows installation are very common, and Visual Studio 2017 community is the most recent version i have access to. I have chosen 32 bit and SDK 8.1 to be as close as possible to Igor Zlatkovic work.

1056e82e

2017-08-28T14:45:19

Description of work needed to compile on zOS building on zOS requires Makefile surgery that we cannot carrry in the common branch, this documents the changes needed to compile on that platform

454e397e

2017-08-28T14:30:43

Porting libxml2 on zOS encoding of code First set of patches for zOS - entities.c parser.c tree.c xmlschemas.c xmlschemastypes.c xpath.c xpointer.c: ask conversion of code to ISO Latin 1 to avoid having the compiler assume EBCDIC codepoint for characters. - xmlmodule.c: make sure we have support for modules - xmlIO.c: zOS path names are special avoid dsome of the expectstions from Unix/Windows

3aca7f31

2017-08-21T13:06:29

Fix unwanted warnings when switching encodings Revert part of commit 46dc989 "Don't switch encoding for internal parameter entities" that caused spurious warnings. Fixes bug 786267.

27f310d4

2017-07-27T14:06:44

Fix signature of xmlSchemaAugmentImportedIDC this is used in a callback which will pass a name, the name is ignored but it's best to have the signature of the function match, pointed out by Claude Petit * xmlschemas.c: fix xmlSchemaAugmentImportedIDC() signature no functional change

d1f90624

2017-07-27T11:20:02

small changes for OS/400 OS/400 version V5R3 is not supported by IBM anymore. In addition, the iSeries system I have here for development has been changed and the new system is not able to compile for an OS version lower than V6R1. Thus I made some updates to the libxml2 os400 scripts accordingly: - Oldest supported OS version is now V6R1. - Adjust ILE/RPG wrappers comments. - Update copyright year range. - Do not log compiler informational messages.

899a5d9f

2017-07-25T14:59:49

Detect infinite recursion in parameter entities When expanding a parameter entity in a DTD, infinite recursion could lead to an infinite loop or memory exhaustion. Thanks to Wei Lei for the first of many reports. Fixes bug 759579.

8a0c6698

2017-07-04T17:13:06

Fix NULL pointer deref in xmlFAParseCharClassEsc Found with libFuzzer.

52ceced6

2017-07-01T17:49:30

Fix infinite loops with push parser in recovery mode Make sure that the input pointer advances in case of errors. Otherwise, the push parser can loop infinitely. Found with libFuzzer.

fb56f80e

2017-07-04T18:38:03

Heap-buffer-overflow read of size 1 in xmlFAParsePosCharGroup Credit to OSS-Fuzz. Add a check to xmlFAParseCharRange() for the end of the buffer to prevent reading past the end of it. This fixes Bug 784017.

f435365b

2017-06-20T16:19:33

Send xmllint usage error to stderr Fixes bug 781459.

3eef3f39

2017-06-20T16:13:57

Fix NULL deref in xmlParseExternalEntityPrivate If called from xmlParseExternalEntity, oldctxt is NULL which leads to a NULL deref if an error occurs. This only affects external code that calls xmlParseExternalEntity. Patch from David Kilzer with minor changes. Fixes bug 780159.

872fea94

2017-06-19T00:24:12

Get rid of "blanks wrapper" for parameter entities Now that replacement of parameter entities goes exclusively through xmlSkipBlankChars, we can account for the surrounding space characters there and remove the "blanks wrapper" hack.

d9e43c7d

2017-06-19T18:01:23

Make sure not to call IS_BLANK_CH when parsing the DTD This is required to get rid of the "blanks wrapper" hack. Checking the return value of xmlSkipBlankChars is more efficient, too.

453dff1e

2017-06-19T17:55:20

Remove unnecessary calls to xmlPopInput It's enough if xmlPopInput is called from xmlSkipBlankChars. Since the replacement text of a parameter entity is surrounded with space characters, that's the only place where the replacement can end in a well-formed document. This is also required to get rid of the "blanks wrapper" hack.

aa267cd1

2017-06-18T23:29:51

Simplify handling of parameter entity references There are only two places where parameter entity references must be handled. For the internal subset in xmlParseInternalSubset. For the external subset or content from other external PEs in xmlSkipBlankChars. Make sure that xmlSkipBlankChars skips over sequences of PEs and whitespace. Rely on xmlSkipBlankChars instead of calling xmlParsePEReference directly when in the external subset or a conditional section. xmlParserHandlePEReference is unused now.

24246c76

2017-06-20T12:56:36

Fix xmlHaltParser Pop all extra input streams before resetting the input. Otherwise, a call to xmlPopInput could make input available again. Also set input->end to input->cur. Changes the test output for some error tests. Unfortunately, some fuzzed test cases were added to the test suite without manual cleanup. This makes it almost impossible to review the impact of later changes on the test output.

e5107772

2017-06-19T15:32:56

Fix pathological performance when outputting charrefs If a character can't be represented in the output encoding, it is converted to a character reference. This used to to replace the character in the input stream by calling xmlBufAddHead or xmlBufferAddHead. These functions shifted the entire input array around, leading to quadratic performance when converting a run of non-representable characters. This is most pronounced when dumping to memory. Output the charref directly instead. Found with libFuzzer.

c9ccbd6a

2017-06-19T14:57:43

Deduplicate code in encoding.c Introduce static functions xmlEncInputChunk and xmlEncOutputChunk that handle the internal/iconv/ICU branching.

5a0ae66d

2017-06-17T23:20:38

Documentation fixes Fixes bug 347465, bug 599433, bug 624550, bug 698253.

8bbe4508

2017-06-17T16:15:09

Spelling and grammar fixes Fixes bug 743172, bug 743489, bug 769632, bug 782400 and a few other misspellings.

576912fa

2017-06-17T15:59:13

Make HTML parser functions take const pointers The 'cur' parameter of htmlParseDoc and htmlSAXParseDoc should be 'const xmlChar *'. Fixes bug 770650.

988a5a3b

2017-06-17T15:44:50

Build test programs only when needed Add test programs to 'check_PROGRAMS' instead of 'noinst_PROGRAMS'. Fixes bug 760457.

b9b4b6b5

2017-06-17T15:40:45

Fix doc/examples/index.py In my previous commit that silenced some test output I didn't realize that doc/examples/Makefile.am was autogenerated. Also make index.py output deterministic by sorting the glob results.

1f09aea2

2017-06-17T15:05:34

Fix compiler warnings in threads.c Use '#pragma weak' to declare weak functions.

629e47e7

2017-06-17T14:51:10

Fix empty-body warning in nanohttp.c

1a595cd1

2017-06-17T14:13:51

Fix cast-align warnings - Suppress warnings in xmlmemory.c by casting to 'void *'. - Remove unneeded cast in xmlschemas.c that caused a macro precedence error. - Add dummy fields to short structs in xmlschemas.c. This increases the size of the structs, but I can't see a better solution without using C11's _Alignof operator. There are still a couple of cast-align warnings in encoding.c. These are legitimate portability issues that can't be fixed without reworking the conversion functions.

81c01ee9

2017-06-17T14:12:53

Fix unused-parameter warnings

1ce1f785

2017-06-17T14:11:03

Fix invalid-source-encoding warnings in testWriter.c Use hex escapes instead of binary data in source file.

5f440d8c

2017-06-12T14:32:34

Rework entity boundary checks Make sure to finish all entities in the internal subset. Nevertheless, readd a sanity check in xmlParseStartTag2 that was lost in my previous commit. Also add a sanity check in xmlPopInput. Popping an input unexpectedly was the source of many recent memory bugs. The check doesn't mitigate such issues but helps with diagnosis. Always base entity boundary checks on the input ID, not the input pointer. The pointer could have been reallocated to the old address. Always throw a well-formedness error if a boundary check fails. In a few places, a validity error was thrown. Fix a few error codes and improve indentation.

46dc9890

2017-06-08T02:24:56

Don't switch encoding for internal parameter entities This is only needed for external entities. Trying to switch the encoding for internal entities could also cause a memory leak in recovery mode.

03904159

2017-06-05T21:16:00

Merge duplicate code paths handling PE references xmlParsePEReference is essentially a subset of xmlParserHandlePEReference, so make xmlParserHandlePEReference call xmlParsePEReference. The code paths in these functions differed slighty, but the code from xmlParserHandlePEReference seems more solid and tested.

dbaab1f3

2017-06-16T21:38:57

Test SAX2 callbacks with entity substitution This detects regressions like bug 760367.

3f0627a1

2017-06-16T21:30:42

Fix duplicate SAX callbacks for entity content Reset 'was_checked' to prevent entity from being parsed twice and SAX callbacks being invoked twice if XML_PARSE_NOENT was set. This regressed in version 2.9.3 and caused problems with WebKit. Fixes bug 760367.

01a4b81a

2017-06-16T21:27:47

Support catalog and threads tests under --without-sax1

3939178e

2017-06-12T20:06:37

Treat URIs with scheme as absolute in C14N Fixes bug 783656.

67f9f9d6

2017-06-12T19:25:01

Misc fixes for 'make tests' - Silence test output. - Clean up after doc/examples tests. - Adjust expected output for script tests. - Add missing results for relaxng/pattern3 There are still two test failures I can't comment on: - regexp/bug316338 - schemas/any4_0

0b2d5c48

2017-06-12T19:10:04

Initialize keepBlanks in HTML parser This caused failures in the HTML push tests but the fix required to change the expected output of the HTML SAX tests.

85c112a0

2017-06-12T18:26:11

Add test cases for bug 758518 test/HTML/758518-entity.html exposed a bug in pushParseTest() in runtest.c which assumed that an input file was at least 4 bytes long. That test case is only 3 bytes, so we now take the minimum of 4 bytes or the length of the test input. We also now use 'chunkSize' in place of the hard-coded value '1024' later in the function.

ed48d65b

2017-06-11T17:12:21

Fix copy-paste errors in error messages Thanks to David Kilzer for the review.

9a366a37

2017-06-11T12:40:01

Fix compiler warning in htmlParseElementInternal

f39e3be0

2017-06-11T12:35:59

Fix sanity check in htmlParseNameComplex - (cur - len) can overflow. - Throw an internal error. Fixes bug 780077.

79c8a6b1

2017-06-10T17:01:27

Print error messages for truncated UTF-8 sequences Before, truncated UTF-8 sequences at the end of a file were treated as EOF. Create an error message containing the offending bytes. xmlStringCurrentChar would also print characters from the input stream, not the string it's working on.

fb2f518c

2017-06-10T17:06:16

Fix potential infinite loop in xmlStringLenDecodeEntities Make sure that xmlParseStringPEReference advances the "str" pointer even if the parser was stopped. Otherwise xmlStringLenDecodeEntities can loop infinitely.

4ba8cc85

2017-06-10T02:33:58

Remove useless check in xmlParseAttributeListDecl Since we already successfully parsed the attribute name and other items, it is guaranteed that we made progress in the input stream. Comparing the input pointer to a previous value also looks fragile to me. What if the input buffer was reallocated and the new "cur" pointer happens to be the same as the old one? There are a couple of similar checks which also take "consumed" into account. This seems to be safer but I'm not convinced that it couldn't lead to false alarms in rare situations.

f9e7997e

2017-06-10T00:33:07

Reset parser input pointers on encoding failure Call xmlBufResetInput before bailing out if switching the encoding fails. Otherwise, the input pointers are left in an invalid state. This would typically lead to an internal error in xmlGROW but could also cause other unforeseen problems.

bedbef80

2017-06-09T15:10:13

Fix memory leak in xmlParseEntityDecl error path When parsing the entity value, it can happen that an external entity with an unsupported encoding is loaded and the parser is stopped. This would lead to a memory leak. A custom SAX callback could also stop the parser. Found with libFuzzer and ASan.

94f6ce83

2017-06-08T22:36:09

Allow zero sized memory input buffers Useful for a fuzz target I'm working on.

91e54967

2017-06-08T18:25:30

Fix xmlBuildRelativeURI for URIs starting with './' If the relative URI started with './', the 'pos' index was increased which also affected indexing into the base path. Aside from producing wrong results, this could also lead to a heap overread of the base path buffer. The data read from beyond the buffer was only compared to some char values, so this is mostly harmless. Inside libxml2, xmlBuildRelativeURI is only called from xinclude.c. Found with libFuzzer and ASan.

45ce1ee3

2017-06-09T15:27:25

Add TODO comment in xmlSwitchEncoding It would be nice if we could recover from unsupported encodings in external entities.

92b9e8c8

2017-06-06T12:56:28

Fix type confusion in xmlValidateOneNamespace Comment out code that casts xmlNsPtr to xmlAttrPtr. ID types on namespace declarations make no practical sense anyway. Fixes bug 780228. Found with libFuzzer and ASan.

8c82f5de

2017-06-07T18:32:49

Fix memory leak in xmlStringLenGetNodeList Avoid expanding the entity recursively. Use the same prevention mechanism as in xmlStringGetNodeList. xmlStringGetNodeList on the other hand wasn't fixing up the 'last' pointer. I think the memory leak can only be triggered in recovery mode. Found with libFuzzer and ASan.

94691dc8

2017-06-07T16:47:36

Fix NULL pointer deref in xmlDumpElementContent Can only be triggered in recovery mode. Fixes bug 758422 (CVE-2017-5969).

362b3229

2017-06-07T16:35:28

Fix memory leak in xmlBufAttrSerializeTxtContent The serializer sets doc->encoding to a temporary value and restores the original value when it's done. This overwrites the encoding value set in xmlBufAttrSerializeTxtContent, causing a memory leak. Don't mess with doc->encoding if invalid UTF-8 is encountered. Found with libFuzzer and ASan.

0db8dc9d

2017-06-07T16:07:33

Stop parser on unsupported encodings Otherwise, the push parser can loop infinitely in recover mode. Found with libFuzzer.

030b1f7a

2017-06-06T15:53:42

Revert "Add an XML_PARSE_NOXXE flag to block all entities loading even local" This reverts commit 2304078555896cf1638c628f50326aeef6f0e0d0. The new flag doesn't work and the change even broke the XML_PARSE_NONET option.

897dffba

2017-06-06T13:21:14

Check for integer overflow in memory debug code Fixes bug 783026. Thanks to Pranjal Jumde for the report.

932cc989

2017-06-03T02:01:29

Fix buffer size checks in xmlSnprintfElementContent xmlSnprintfElementContent failed to correctly check the available buffer space in two locations. Fixes bug 781333 (CVE-2017-9047) and bug 781701 (CVE-2017-9048). Thanks to Marcel Böhme and Thuan Pham for the report.

e2663054

2017-06-05T15:37:17

Fix handling of parameter-entity references There were two bugs where parameter-entity references could lead to an unexpected change of the input buffer in xmlParseNameComplex and xmlDictLookup being called with an invalid pointer. Percent sign in DTD Names ========================= The NEXTL macro used to call xmlParserHandlePEReference. When parsing "complex" names inside the DTD, this could result in entity expansion which created a new input buffer. The fix is to simply remove the call to xmlParserHandlePEReference from the NEXTL macro. This is safe because no users of the macro require expansion of parameter entities. - xmlParseNameComplex - xmlParseNCNameComplex - xmlParseNmtoken The percent sign is not allowed in names, which are grammatical tokens. - xmlParseEntityValue Parameter-entity references in entity values are expanded but this happens in a separate step in this function. - xmlParseSystemLiteral Parameter-entity references are ignored in the system literal. - xmlParseAttValueComplex - xmlParseCharDataComplex - xmlParseCommentComplex - xmlParsePI - xmlParseCDSect Parameter-entity references are ignored outside the DTD. - xmlLoadEntityContent This function is only called from xmlStringLenDecodeEntities and entities are replaced in a separate step immediately after the function call. This bug could also be triggered with an internal subset and double entity expansion. This fixes bug 766956 initially reported by Wei Lei and independently by Chromium's ClusterFuzz, Hanno Böck, and Marco Grassi. Thanks to everyone involved. xmlParseNameComplex with XML_PARSE_OLD10 ======================================== When parsing Names inside an expanded parameter entity with the XML_PARSE_OLD10 option, xmlParseNameComplex would call xmlGROW via the GROW macro if the input buffer was exhausted. At the end of the parameter entity's replacement text, this function would then call xmlPopInput which invalidated the input buffer. There should be no need to invoke GROW in this situation because the buffer is grown periodically every XML_PARSER_CHUNK_SIZE characters and, at least for UTF-8, in xmlCurrentChar. This also matches the code path executed when XML_PARSE_OLD10 is not set. This fixes bugs 781205 (CVE-2017-9049) and 781361 (CVE-2017-9050). Thanks to Marcel Böhme and Thuan Pham for the report. Additional hardening ==================== A separate check was added in xmlParseNameComplex to validate the buffer size.

7482f41f

2017-06-01T22:00:19

Check for integer overflow in xmlXPathFormatNumber Check for overflow before casting double to int. Found with afl-fuzz and UBSan.

863b5792

2017-06-01T17:53:16

Make Travis print UBSan stacktraces

a2b53178

2017-06-01T01:21:27

Add .travis.yml For now this is mainly useful if you work on a fork of the libxml2 mirror on GitHub: https://github.com/GNOME/libxml2 Start with two build setups: - GCC with as many GNU extensions disabled as possible, trying to emulate a C89 compiler on a POSIX system. - clang with ASan and UBSan. The Python tests don't set an exit code, so Travis won't detect failures. The same goes for "make tests", but we only run "make check" anyway.

83212ff4

2017-06-01T12:42:03

Fix expected error output in Python tests

855c19ef

2017-06-01T01:04:08

Avoid reparsing in xmlParseStartTag2 The code in xmlParseStartTag2 must handle the case that the input buffer was grown and reallocated which can invalidate pointers to attribute values. Before, this was handled by detecting changes of the input buffer "base" pointer and, in case of a change, jumping back to the beginning of the function and reparsing the start tag. The major problem of this approach is that whether an input buffer is reallocated is nondeterministic, resulting in seemingly random test failures. See the mailing list thread "runtest mystery bug: name2.xml error case regression test" from 2012, for example. If a reallocation was detected, the code also made no attempts to continue parsing in case of errors which makes a difference in the lax "recover" mode. Now we store the current input buffer "base" pointer for each (not separately allocated) attribute in the namespace URI field, which isn't used until later. After the whole start tag was parsed, the pointers to the attribute values are reconstructed using the offset between the new and the old input buffer. This relies on arithmetic on dangling pointers which is technically undefined behavior. But it seems like the easiest and most efficient fix and a similar approach is used in xmlParserInputGrow. This changes the error output of several tests, typically making it more verbose because we try harder to continue parsing in case of errors. (Another possible solution is to check not only the "base" pointer but the size of the input buffer as well. But this would result in even more reparsing.)

07b7428b

2017-06-01T00:19:14

Simplify control flow in xmlParseStartTag2 Remove some goto labels and deduplicate a bit of code after handling namespaces. Before: loop { parseAttribute if (ok) { if (defaultNamespace) { handleDefaultNamespace if (error) goto skip_default_ns; handleDefaultNamespace skip_default_ns: freeAttr nextAttr continue; } if (namespace) { handleNamespace if (error) goto skip_ns; handleNamespace skip_ns: freeAttr nextAttr; continue; } handleAttr } else { freeAttr } nextAttr } After: loop { parseAttribute if (!ok) goto next_attr; if (defaultNamespace) { handleDefaultNamespace if (error) goto next_attr; handleDefaultNamespace } else if (namespace) { handleNamespace if (error) goto next_attr; handleNamespace } else { handleAttr } next_attr: freeAttr nextAttr }

ac9a4560

2017-05-31T19:06:30

Disable LeakSanitizer when running API tests The autogenerated API tests leak memory.

ff34ba3e

2017-05-31T18:53:45

Avoid out-of-bound array access in API tests The API tests combine string buffers with arbitrary length values which makes ASan detect out-of-bound array accesses. Even without ASan, this could lead to unwanted test failures. Add a check for "len", "size", and "start" arguments, assuming they apply to the nearest char pointer. Skip the test if they exceed the buffer size. This is a somewhat naive heuristic but it seems to work well.

34e44567

2017-05-31T16:48:27

Fix undefined behavior in xmlRegExecPushStringInternal It's stupid, but the behavior of memcpy(NULL, NULL, 0) is undefined.

47496724

2017-05-31T16:46:39

Avoid spurious UBSan errors in parser.c If available, use a C99 flexible array member to avoid spurious UBSan errors.

f4029cd4

2016-04-21T16:37:26

Check XPath exponents for overflow Avoid undefined behavior and wrong results with huge exponents. Found with afl-fuzz and UBSan.

a58331a6

2017-05-29T21:02:21

Check for overflow in xmlXPathIsPositionalPredicate Avoid undefined behavior when casting from double to int. Found with afl-fuzz and UBSan.

a851868a

2017-05-29T20:14:42

Parse small XPath numbers more accurately Don't count leading zeros towards the fraction size limit. This allows to parse numbers like 0.0000000000000000000000000000000000000000000000000000000001 which is the only standard-conformant way to represent such numbers, as scientific notation isn't allowed in XPath 1.0. (It is allowed in XPath 2.0 and in libxml2 as an extension, though.) Overall accuracy is still bad, see bug 783238.

4bebb030

2016-04-21T13:41:09

Rework XPath rounding functions Use the C library's floor and ceil functions. The old code was overly complicated for no apparent reason and could result in undefined behavior when handling NaNs (found with afl-fuzz and UBSan). Fix wrong comment in xmlXPathRoundFunction. The implementation was already following the spec and rounding half up.

43f50f4d

2017-05-26T23:01:54

Fix white space in test output Quote echoed variable to avoid newlines being converted to space.

40f58521

2017-05-26T20:16:35

Fix axis traversal from attribute and namespace nodes When traversing the "preceding" axis from an attribute node, we must first go up to the attribute's containing element. Otherwise, text children of other attributes could be returned. This made it possible to hit a code path in xmlXPathNextAncestor which contained another bug: The attribute node was initialized with the context node instead of the current node. Normally, this code path is only hit via xmlXPathNextAncestorOrSelf in which case the current and context node are the same. The combination of the two bugs could result in an infinite loop, found with libFuzzer. Traversing the "following" and the "preceding" axis from namespace nodes should be handled similarly. This wasn't supported at all previously.

a07a4e96

2017-05-27T17:04:12

Fix spurious error message Commit c851970 introduced a spurious error message when evaluating XPath expressions with xmlXPathCompiledEvalToBoolean.

aed407c1

2017-05-25T16:57:14

Check for trailing characters in XPath expressions earlier Move the check for trailing characters from xmlXPathEval to xmlXPathEvalExpr. Otherwise, a valid portion of a syntactically invalid expression would be evaluated before returning an error.

kc3-lang/libxml2

Log