xinclude.c


Log

Author Commit Date CI Message
Nick Wellnhofer afeff9c5 2024-12-21T20:47:40 xinclude: Allow build without XPath This disables XPath queries and makes the tests fail, but might be useful.
Nick Wellnhofer ae41cf9a 2024-12-15T23:05:06 xinclude: Check reallocations for overflow
Nick Wellnhofer d3a70754 2024-11-25T21:35:09 xinclude: Fix handling of catastrophic errors - Forward catastrophic errors from parser - Don't overwrite catastrophic errors
Nick Wellnhofer 0f4f8900 2024-11-17T20:13:14 parser: Rename inputPush to xmlCtxtPushInput
Nick Wellnhofer 322e733b 2024-07-18T19:27:43 xinclude: Fix fallback for text includes Fixes #772.
Nick Wellnhofer a221cd78 2024-07-07T03:01:51 buf: Rework xmlBuf code Always use what the old implementation called the "IO" allocation scheme, allowing to move the content pointer past the initial allocation. This is inexpensive and allows efficient shrinking. Optimize xmlBufGrow, reusing shrunken memory as much as possible. Simplify xmlBufAdd. Make xmlBufBackToBuffer return an error on overflow. Make "size" exclude the terminating NULL byte. Always provide an initial size. Reintroduce static buffers. Remove xmlBufResize and several other functions.
Nick Wellnhofer 6be79014 2024-07-15T14:18:26 Remove unused code
Nick Wellnhofer 2e63656e 2024-07-07T19:21:46 parser: Check return value of inputPush inputPush typically doesn't fail because we pre-allocate the input table. The return value should be checked nevertheless.
Nick Wellnhofer 16e7ecd4 2024-07-01T16:01:24 xinclude: Check URI length Don't report long URIs as OOM errors.
Nick Wellnhofer 598ee0d2 2024-06-26T01:18:55 error: Remove underscores from xmlRaiseError
Nick Wellnhofer 1dd5e76a 2024-06-17T21:06:46 xinclude: Don't remove root element Don't replace include element at root with empty nodeset.
Nick Wellnhofer 3c7c831c 2024-06-17T14:15:49 xinclude: Set XPath context doc Regressed in commit 78eab7a1.
Nick Wellnhofer 1ff48433 2024-06-17T12:54:46 xinclude: Load included documents with XML_PARSE_DTDLOAD We must detect ID attributes declared in external DTDs. Regressed in 12f0bb94.
Nick Wellnhofer 1aa37db0 2024-06-14T20:44:45 xinclude: Don't raise error on empty nodeset xmlXPtrEval returning NULL means an empty nodeset if no error code is set. See #733.
Nick Wellnhofer 669bd349 2024-06-12T18:20:01 xpointer: Remove support for XPointer locations The latest spec for what it essentially an XPath extension seems to be this working draft from 2002: https://www.w3.org/TR/xptr-xpointer/ The xpointer() scheme is listed as "being reviewed" in the XPointer registry since at least 2006. libxml2 seems to be the only modern software that tries to implement this spec, but the code has many bugs and quality issues. If you configure --with-legacy, old symbols are retained for ABI compatibility.
Nick Wellnhofer 52384043 2024-06-11T19:10:41 parser: Pass resource type to resource loader
Nick Wellnhofer fa2592b3 2024-06-11T14:40:28 xinclude: Add callbacks for resource loader
Nick Wellnhofer 15ed7566 2024-06-06T22:18:05 xinclude: Fix warning when building --without-xptr
Nick Wellnhofer e75e878e 2024-05-20T13:58:22 doc: Update and fix documentation
Nick Wellnhofer 577fb0e3 2024-04-27T16:31:01 xinclude: Report malloc failure in xmlXIncludeMergeEntities
Nick Wellnhofer 087a3464 2024-04-18T11:57:46 xinclude: Increase error count in xmlXIncludeErrMemory
Nick Wellnhofer 16ccb61d 2024-04-05T13:19:56 xinclude: Don't try to fix base of non-elements This would result in a spurious error.
Nick Wellnhofer 365976db 2024-03-30T17:17:21 xinclude: Don't use xmlHasNsProp This function can malloc and doesn't report failures. Another design mistake is that xmlUnsetNsProp requires an xmlNs struct.
Nick Wellnhofer f43197fc 2024-03-29T11:16:45 tree: Don't coalesce text nodes in xmlAdd{Prev,Next}Sibling Commit 9e1c72da from 2001 introduced a bug where xmlAddPrevSibling and xmlAddNextSibling would only try to merge text nodes with one of its new siblings. Commit 4ccd3eb8 fixed this bug but unfortunately, lxml and possibly other downstream code depend on text nodes not being merged. To avoid breaking downstream code while still having somewhat consistent API behavior, it's probably best to make these functions never coalesce text nodes.
Nick Wellnhofer 1ace0e85 2024-03-24T12:28:22 xinclude: Report malloc failure in xmlXIncludeAddNode
Nick Wellnhofer 0d04d79c 2024-03-22T12:26:28 xinclude: Report malloc failures in xmlXIncludeIncludeNode
Nick Wellnhofer 3bdd0d7b 2024-03-18T12:12:15 xinclude: Set errNo in xmlXIncludeErrMemory
Nick Wellnhofer 12f0bb94 2024-01-05T01:14:28 parser: Synchronize more options
Nick Wellnhofer 0821efc8 2024-01-02T18:33:57 encoding: Check whether encoding handlers support input/output The "HTML" encoding handler doesn't support input which could lead to a wrong error report.
Nick Wellnhofer e8fb3d63 2024-01-02T17:45:54 parser: Convert some "internal errors" to meaningful codes
Nick Wellnhofer 2a2fbe1e 2023-12-28T16:42:03 xinclude: Only set xml:base if necessary
Nick Wellnhofer 8a685a3d 2023-12-26T00:42:22 xinclude: Allow empty nodesets There's no reason to treat an empty nodeset as error.
Nick Wellnhofer 955c177f 2023-12-23T00:58:36 parser: Stop using 'directory' struct member This was only used as a pointless fallback for URI resolution.
Nick Wellnhofer 6e3a2ac6 2023-12-22T21:38:50 xinclude: Rework xml:base fixup The xml:base fixup was broken in more complex cases. Also avoid parsing and building the included URI multiple times.
Nick Wellnhofer 35a4bc50 2023-12-22T15:14:19 xinclude: Report to xmlGenericError
Nick Wellnhofer c8f1f4a2 2023-12-21T17:30:38 doc: Improve documentation of error handlers
Nick Wellnhofer 0a658c0f 2023-12-20T23:53:19 io: Don't use "-" to read from stdin To implement this feature on such a low level is a disaster waiting to happen. Remove these checks from the IO code and move them to xmllint. Note that the serialization API will still treat "-" as stdout.
Nick Wellnhofer 2829a21a 2023-12-18T19:43:55 xinclude: Improve error handling Introduce xmlXIncludeSetErrorHandler allowing to set a structured error handler for an XInclude context and forwarding errors from parser. Remove arguments from memory error handlers. Use xmlRaiseMemoryError.
Nick Wellnhofer 78eab7a1 2023-12-10T18:15:59 xinclude: Report malloc failures Fix many places where malloc failures aren't reported. Introduce a new API function xmlXIncludeGetLastError.
Nick Wellnhofer 80a0580f 2023-09-30T15:47:46 xinclude: Expand comment about fuzz timeouts
Nick Wellnhofer b9db3d7d 2023-09-22T15:45:20 parser: Simplify xmlStringCurrentChar Start to move away from using this function.
Nick Wellnhofer 699299ca 2023-09-20T18:54:39 globals: Stop including globals.h
Nick Wellnhofer 4e1c13eb 2023-09-18T14:45:10 debug: Remove debugging code This is barely useful these days and only clutters the code base.
Nick Wellnhofer 4aa08c80 2023-09-08T14:52:22 xinclude: Fix 'last' pointer in xmlXIncludeCopyNode Also set the 'last' pointer for the root node. Fixes https://gitlab.gnome.org/GNOME/libxslt/-/issues/93
Nick Wellnhofer 687a2b71 2023-05-08T17:05:13 xinclude: Lower initial table size when fuzzing We don't have test cases with many documents, so set the initial table size to 1 when fuzzing, so there is a chance to detect reallocation issues.
Nick Wellnhofer c40cbf07 2023-05-08T17:03:00 malloc-fail: Fix null deref after xmlXIncludeNewRef See #344.
Nick Wellnhofer 105ce73d 2023-05-08T16:45:28 xinclude: Fix false positives in inclusion loop detection xmlXIncludeRecurseDoc can realloc the cache.
Nick Wellnhofer ec05f04d 2023-02-16T12:40:02 malloc-fail: Fix memory leak in xmlXIncludeLoadTxt Found with libFuzzer, see #344.
Nick Wellnhofer c02df686 2023-02-16T12:10:36 malloc-fail: Fix memory leak in xmlXIncludeLoadDoc Found with libFuzzer, see #344.
Nick Wellnhofer 3b59fdf0 2023-02-15T13:28:24 malloc-fail: Fix memory leak in xmlXIncludeAddNode Found with libFuzzer, see #344.
Nick Wellnhofer e20f4d7a 2023-02-13T14:38:05 xinclude: Fix quadratic behavior in xmlXIncludeLoadTxt Also make text inclusions work with memory buffers, for example when using a custom entity loader, and fix a memory leak in case of invalid characters. Fixes #483.
Nick Wellnhofer a96312db 2023-02-03T14:55:53 xinclude: Avoid timeouts when fuzzing Fix the check for maximum number of inclusions.
Nick Wellnhofer be0ec005 2023-02-03T14:37:49 xinclude: Abort immediately if max depth was exceeded Avoids resource exhaustion if the maximum recursion depth was exceeded. Note that the XInclude engine offers no protection against other "billion laughs"-style amplification attacks as long as they stay below the maximum depth.
Nick Wellnhofer dc2dde1a 2023-02-04T15:00:54 malloc-fail: Fix null deref in xmlXIncludeLoadTxt Found with libFuzzer, see #344.
Nick Wellnhofer a3749551 2023-02-03T14:00:13 malloc-fail: Fix reallocation in xmlXIncludeNewRef Avoid null deref. Found with libFuzzer, see #344.
Nick Wellnhofer 5a19e216 2022-11-02T16:13:27 malloc-fail: Fix use-after-free in xmlXIncludeAddNode Found with libFuzzer, see #344.
Nick Wellnhofer b456e3bb 2022-10-30T20:28:20 xinclude: Always allow XPtr expressions in external documents
Nick Wellnhofer f14529ba 2022-10-30T12:32:14 xinclude: Fix more memory leaks in xmlXIncludeLoadDoc
Nick Wellnhofer eef0a739 2022-10-30T12:21:20 xinclude: Implement "streaming" mode When using xmlreader, XPointer expressions in XIncludes simply cannot work. Expressions can reference nodes which weren't parsed yet or which were already deleted. After fixing nested XIncludes, we reference includes which were parsed previously. When streaming, these nodes could have been deleted, leading to use-after-free errors. Disallow XPointer expressions and truncate the include table in streaming mode.
Nick Wellnhofer 7e078cc1 2022-10-30T12:03:51 xinclude: Clean up xmlXIncludeDoProcess Only scan new includes in the second pass. Always reset the "replace" flag. This only matters if xmlXIncludeDoProcess is called multiple times on the same document which currently only happens when called from xmlreader.
Nick Wellnhofer 075cee9e 2022-10-29T15:38:16 xinclude: Fix memory leak when fuzzing This only affects the fuzzing build mode.
Nick Wellnhofer b59ff587 2022-10-28T20:08:02 xinclude: Fix memory leak in xmlXIncludeLoadDoc Regressed in a recent commit.
Nick Wellnhofer 20e2fb4c 2022-10-23T17:52:29 xinclude: Avoid creation of subcontexts Don't create subcontext in xmlXIncludeRecurseDoc. Save and restore 'doc' and 'incTab' instead. Make xmlXIncludeLoadFallback call xmlXIncludeCopyNode which seems safer than xmlXIncludeDoProcess since the latter may modify the document. This should also be more performant since we need to copy the whole fallback subtree anyway. Also make sure to avoid replacements in fallback elements in xmlXIncludeDoProcess.
Nick Wellnhofer 010b682f 2022-10-23T16:02:48 xinclude: Rework XInclude cache Use a single cache stored in the "root" context, avoiding cache misses in some scenarios. Separate the cache from the "include" table. Rework recursion check.
Nick Wellnhofer 3b68aa47 2022-10-23T16:54:22 xinclude: Remove unused 'doc' argument from xmlXIncludeDoProcess
Nick Wellnhofer 0b09f132 2022-10-23T14:59:31 xinclude: Remove unused member 'url' from xmlXIncludeCtxt
Nick Wellnhofer 966f5ebd 2022-10-23T14:14:39 xinclude: Remove inefficient refcounting scheme xinclude.c used to implement a refcounting mechanism for documents which could result in documents being freed too early and reloaded. We really should keep parsed and processed documents around for as long as possible to avoid reprocessing.
Nick Wellnhofer fe0194d5 2022-10-23T13:57:33 xinlucde: Pass xmlXIncludeRefPtrs directly Stop passing indices into the include table.
Nick Wellnhofer 0fc5be0a 2022-10-23T13:55:29 xinclude: Remove dead code in xmlXIncludeLoadDoc The include table was already scanned for entries with a matching URI a couple of lines before.
Nick Wellnhofer 2fc8d123 2022-10-22T19:08:43 xinclude: Make xmlXIncludeCopyNode non-recursive Avoid call stack overflows. Also switch to xmlStaticCopyNode which avoids duplicate namespace definitions.
Nick Wellnhofer bc32c386 2022-10-22T17:48:25 xinclude: Don't pass documents to xmlXIncludeCopyNode Extract the root element directly in xmlXIncludeCopyXPointer.
Nick Wellnhofer 46e4cfb6 2022-10-22T19:20:20 xinclude: Remove unused source and target arguments source is unused, target is always ctxt->doc.
Nick Wellnhofer d2ed1e4f 2022-10-22T16:50:18 xinclude: Limit recursion depth This avoids call stack overflows.
Nick Wellnhofer 34496f26 2022-10-22T16:09:21 xinclude: Test for inclusion loops
Nick Wellnhofer bad30e2b 2022-10-21T17:27:23 xinclude: Copy included docs with xmlDocCopyNode All includes should already be expanded after calling xmlXIncludeRecurseDoc, so there's no need to invoke the more costly xmlXIncludeCopyNodeList. Now xmlXIncludeCopyNode is only used to copy XPointer fragments.
Nick Wellnhofer bc267cb9 2022-10-22T02:19:22 xinclude: Expand includes in xmlXIncludeCopyNode This should make nested includes work reliably. Fixes #424.
Nick Wellnhofer 5e4b751f 2022-10-22T02:17:39 xinclude: Make xmlXIncludeCopyNode return a node list Convert callers of xmlXIncludeCopyNode to handle node lists.
Nick Wellnhofer 64eaf4d4 2022-10-21T16:05:29 xinclude: Make xmlXIncludeCopyNode iterate all descendants
Nick Wellnhofer 5f664ab2 2022-10-21T16:17:48 xinclude: Rework xmlXIncludePreProcessNode Rename xmlXIncludePreProcessNode to xmlXIncludeExpandNode. Search the include table for an already existing entry. Also load the included items in xmlXIncludeExpandNode and remove the second phase in xmlXIncludeDoProcess.
Nick Wellnhofer 4d7524af 2022-10-21T16:46:38 xinclude: Ignore XML_XINCLUDE_START in xmlXIncludeCopyXPointer The XPath engine shouldn't return such nodes anyway.
Nick Wellnhofer 938105b5 2022-10-21T15:56:12 Revert "xinclude: Fix regression with nested includes" This reverts commit 7f04e297318b1b908cec20711f74f75625afed7f which caused memory errors. See #424.
Nick Wellnhofer 7f04e297 2022-10-18T18:40:00 xinclude: Fix regression with nested includes This reverts commits 74dcc10b and 87d20b55. Fixes #424.
Nick Wellnhofer 6843fc72 2022-09-01T02:58:00 Remove or annotate char casts
Nick Wellnhofer 0f568c0b 2022-08-26T01:22:33 Consolidate private header files Private functions were previously declared - in header files in the root directory - in public headers guarded with IN_LIBXML - in libxml.h - redundantly in source files that used them. Consolidate all private header files in include/private.
Nick Wellnhofer 3e7b4f37 2022-05-20T23:28:25 Avoid calling xmlSetTreeDoc Create text nodes with xmlNewDocText or set the document directly to avoid xmlSetTreeDoc being called when the node is inserted.
Nick Wellnhofer 0aa8652e 2022-05-20T14:54:49 Use xmlNewDocText in xmlXIncludeCopyRange Otherwise, the initial node of the copy could be a text node with a NULL document. This results in the NULL document being propagated to copies of other nodes, losing information about the dictionary in which node data is stored, and freeing a dict-allocated string. See discussion in !175.
Nick Wellnhofer 67070107 2022-04-20T23:17:14 Add configuration flag for XPointer locations support Add a new configuration flag that controls whether the outdated support for XPointer locations (ranges and points) is enabled. --with-xptr-locs # Autotools LIBXML2_WITH_XPTR_LOCS # CMake The latest spec for what it essentially an XPath extension seems to be this working draft from 2002: https://www.w3.org/TR/xptr-xpointer/ The xpointer() scheme is listed as "being reviewed" in the XPointer registry since at least 2006. libxml2 seems to be the only modern software that tries to implement this spec, but the code has many bugs and quality issues. The flag defaults to "off" and support for this extensions has to be requested explicitly. The relevant API functions are deprecated.
Nick Wellnhofer 41afa89f 2022-04-10T14:09:29 Fix short-lived regression in xmlStaticCopyNode Commit 7618a3b1 didn't account for coalesced text nodes. I think it would be better if xmlStaticCopyNode didn't try to coalesce text nodes at all. This code path can only be triggered if some other code doesn't coalesce text nodes properly. In this case, OSS-Fuzz found such behavior in xinclude.c.
Nick Wellnhofer 4a8c71eb 2022-03-04T03:35:57 Remove DOCBparser This code has been broken and deprecated since version 2.6.0, released in 2003. Because of a bug in commit 961b535c, DOCBparser.c was never compiled since 2012. I couldn't find a Debian package using any of its symbols, so it seems safe to remove this module.
Nick Wellnhofer bc06a522 2022-03-02T02:57:49 Fix recursion check in xinclude.c Compare the included URL with the document's URL to detect local inclusions. Fixes #348.
Nick Wellnhofer 346c3a93 2022-02-20T18:46:42 Remove elfgcchack.h The same optimization can be enabled with -fno-semantic-interposition since GCC 5. clang has always used this option by default.
David Kilzer 8f5ccada 2021-07-07T19:24:36 xmlAddChild() and xmlAddNextSibling() may not attach their second argument Use the return value of xmlAddChild() and xmlAddNextSibling() instead of the second argument directly. Found by OSS-Fuzz. Fixes #316
Nick Wellnhofer 1098c30a 2021-04-22T19:26:28 Fix user-after-free with `xmllint --xinclude --dropdtd` The --dropdtd option can leave dangling pointers in entity reference nodes. Make sure to skip these nodes when processing XIncludes. This also avoids scanning entity declarations and even modifying them inadvertently during XInclude processing. Move from a block list to an allow list approach to avoid descending into other node types that can't contain elements. Fixes #237.
Nick Wellnhofer 847a3a11 2020-09-28T12:28:29 Fix use-after-free when XIncluding text from Reader The XML Reader can free text nodes coming from the XInclude engine before parsing has finished. Cache a copy of the text string, not the included node to avoid use after free. Found by OSS-Fuzz.
Nick Wellnhofer 3fcf3193 2020-08-22T00:43:18 Fix regression introduced with commit d88df4b Revert the commit and use a different approach. Found by OSS-Fuzz.
Nick Wellnhofer 87d20b55 2020-08-19T13:52:08 Fix regression introduced with commit 74dcc10b The code wasn't dead after all, but I can see no reason in delaying the XPointer evaluation. This could lead to nodes included earlier appearing in XPointer results.
Nick Wellnhofer fbb7fa9a 2020-08-19T13:13:20 Fix memory leak in xmlXIncludeAddNode error paths Found by OSS-Fuzz.
Nick Wellnhofer 19cae17f 2020-08-19T13:07:28 Revert "Fix quadratic runtime in xi:fallback processing" This reverts commit 27119ec33c9f6b9830efa1e0da0acfa353dfa55a. Not copying fallback children didn't fix up namespaces and could lead to use-after-free errors. Found by OSS-Fuzz.
Nick Wellnhofer d63cfeca 2020-08-17T15:40:06 Add TODO comment in xinclude.c Add some thoughts on the major remaining problems with the XInclude implementation.
Nick Wellnhofer 74dcc10b 2020-08-17T03:24:56 Remove dead code in xinclude.c 'doc' is checked for NULL in xmlXIncludeLoadDoc, so several code paths can be eliminated.