|
e7b6ca15
|
2023-09-18T13:25:06
|
|
globals: Rework global state destruction on Windows
If DllMain is used, rely on it working as expected. The old code seemed
to attempt to free global state of other threads if, for some reason,
the DllMain mechanism didn't work.
In a static build, register a destructor with
RegisterWaitForSingleObject.
Make public functions xmlGetGlobalState and xmlInitializeGlobalState
no-ops.
Move initialization and registration of global state objects to
xmlInitGlobalState. Lookup global state with xmlGetThreadLocalStorage
which can be inlined nicely.
Also cleanup global state when using TLS. xmlLastError must be reset.
|
|
|
39a275a5
|
2023-09-18T21:25:35
|
|
globals: Define globals using macros
Declare and define globals and helper functions by (ab)using the
preprocessor.
|
|
|
bf6bd161
|
2023-09-18T19:53:31
|
|
globals: Introduce xmlCheckThreadLocalStorage
Checks whether (emulated) thread-local storage could be allocated.
|
|
|
89f49767
|
2023-09-18T18:44:32
|
|
globals: Make xmlGlobalState private
This removes a public struct but it seems impossible to use its members
in a sensible way from external code.
|
|
|
a07ec7c1
|
2023-09-18T17:39:13
|
|
threads: Move library initialization code to threads.c
This allows to consolidate the initialization code since the global init
lock was already implemented in threads.c.
|
|
|
4e1c13eb
|
2023-09-18T14:45:10
|
|
debug: Remove debugging code
This is barely useful these days and only clutters the code base.
|
|
|
c19771c1
|
2023-09-18T00:54:39
|
|
globals: Move code from threads.c to globals.c
Move all code that handles globals to the place where it belongs.
|
|
|
2a4b8114
|
2023-09-17T23:16:49
|
|
globals: Rename members of xmlGlobalState
This is a deliberate first step to remove some internals from the
public API and to avoid issues when redefining tokens.
|
|
|
edc2dd48
|
2023-09-04T16:07:23
|
|
dict: Update hash function
Update hash function from classic Jenkins OAAT (dict.c) and a variant of
DJB2 (hash.c) to "GoodOAAT" taken from the SMHasher repo. This hash
function passes all SMHasher tests.
|
|
|
57cfd221
|
2023-09-01T14:52:04
|
|
dict: Use xoroshiro64** as PRNG
Stop using rand_r. This enables hash randomization on all platforms.
|
|
|
778cca38
|
2023-08-20T22:50:57
|
|
legacy: Add stubs for disabled modules
When legacy support is requested, always enable stubs for FTP and
XPointer location modules which were removed from the standard
configuration. Going forward, the --with-legacy configuration option
should be used to provide maximum ABI compatibility.
Fixes #433.
|
|
|
ed3bd052
|
2023-08-20T20:48:10
|
|
parser: Allow to set maximum amplification factor
|
|
|
f1c1f5c6
|
2023-08-16T19:43:02
|
|
parser: Revert change to doc->encoding
Fixes #579.
|
|
|
95e81a36
|
2023-08-08T15:21:31
|
|
parser: Decode all data in xmlCharEncInput
Even with flush set to true, xmlCharEncInput didn't guarantee to decode
all data. This complicated the push parser.
Remove the flush flag and always decode all available data.
Also fix ICU code where the flush flag has a different meaning. Always
set flush to false and retry even with empty input buffers.
|
|
|
834b8123
|
2023-08-08T15:21:28
|
|
parser: Stream data when reading from memory
Don't create a copy of the whole input buffer. Read the data chunk by
chunk to save memory.
Historically, it was probably envisioned to read data from memory
without additional copying. This doesn't work reliably with the current
design of the XML parser which requires a terminating null byte at the
end of input buffers. This lead to xmlReadMemory interfaces, which
expect pointer and size arguments, being changed to make a
zero-terminated copy of the input buffer. Interfaces based on
xmlReadDoc, which actually expect a zero-terminated string and
would make zero-copy operation work, were then simplified to rely on
xmlReadMemoryi, resulting in an unnecessary copy.
To avoid copying (possibly gigabytes) of memory temporarily, we now
stream in-memory input just like content read from files in a
chunk-by-chunk fashion (using a somewhat outdated INPUT_CHUNK size of
250 bytes). As a side effect, we also avoid another copy of the whole
input when handling non-UTF-8 data which was made possible by some
earlier commits.
Interfaces expecting zero-terminated strings now make use of strnlen
which unfortunately isn't part of the standard C library and only
mandated since POSIX 2008.
|
|
|
59fa0bb3
|
2023-08-08T15:21:14
|
|
parser: Simplify input pointer updates
The base member always points to the beginning of the buffer.
|
|
|
ec7be506
|
2023-08-08T15:19:46
|
|
parser: Rework encoding detection
Introduce XML_INPUT_HAS_ENCODING flag for xmlParserInput which is set
when xmlSwitchEncoding is called. The parser can use the flag to
reliably detect whether an encoding was already set via user override,
BOM or other auto-detection. In this case, the encoding declaration
won't be used to switch the encoding.
Before, an inscrutable mix of ctxt->charset, ctxt->input->encoding
and ctxt->input->buf->encoder was used.
Introduce private helper functions to switch encodings used by both the
XML and HTML parser:
- xmlDetectEncoding which skips over the BOM, allowing to remove the
BOM checks from other encoding functions.
- xmlSetDeclaredEncoding, replacing htmlCheckEncodingDirect, which warns
about encoding mismatches.
If users override the encoding, store the declared instead of the actual
encoding in xmlDoc. In this case, the actual encoding is known and the
raw value from the doc is more useful.
Also use the input flags to store the ISO-8859-1 fallback state.
Restrict the fallback to cases where no encoding was specified. (The
fallback is only useful in recovery mode and these days broken UTF-8 is
probably more likely than ISO-8859-1, so it might eventually be removed
completely.)
The 'charset' member of xmlParserCtxt is now unused. The 'encoding'
member of xmlParserInput is now unused.
The 'standalone' member of xmlParserInput is renamed to 'flags'.
A new parser state XML_PARSER_XML_DECL is added for the push parser.
|
|
|
b8961df6
|
2023-05-09T03:25:24
|
|
SAX: Always validate xml:ids
The behavior shouldn't depend on mostly random configuration options.
|
|
|
8d5e33ef
|
2023-05-03T20:42:10
|
|
Fix compiler warning on GCC < 8
-Wcast-function-type is only available since GCC 8.
|
|
|
fc69cf56
|
2023-04-30T17:51:29
|
|
parser: Move xmlFatalErr to parserInternals.c
|
|
|
3ff6abbf
|
2023-02-22T17:11:20
|
|
encoding: Rework error codes
Use an enum instead of magic numbers. Fix a few error codes. Simplify
handling of "space" and "partial" errors.
See #506.
|
|
|
fa993130
|
2023-04-30T12:57:09
|
|
xpath: Remove remaining references to valueFrame
Fixes #529.
|
|
|
3ffcc03b
|
2023-03-13T19:38:41
|
|
parser: Deprecate more internal functions
|
|
|
98840d40
|
2023-03-21T19:07:12
|
|
parser: Rework EBCDIC code page detection
To detect EBCDIC code pages, we used to switch the encoding twice and
had to be very careful not to decode data after the XML declaration
before the second switch. This relied on a hard-coded expected size of
the XML declaration and was complicated and unreliable.
Now we convert the first 200 bytes to EBCDIC-US and parse the encoding
declaration manually.
|
|
|
04d1bedd
|
2023-03-21T13:08:44
|
|
parser: Rework shrinking of input buffers
Don't try to grow the input buffer in xmlParserShrink. This makes sure
that no memory allocations are made and the function always succeeds.
Remove unnecessary invocations of SHRINK. Invoke SHRINK at the end of
DTD parsing loops.
Shrink before growing.
|
|
|
b167c731
|
2023-03-14T14:42:36
|
|
parser: Fix short-lived regression causing infinite loops
Fix 3eb6bf03. We really have to halt the parser, so the input buffer
gets reset.
|
|
|
f8efa589
|
2023-03-14T13:55:06
|
|
malloc-fail: Handle malloc failures in xmlSchemaInitTypes
Note that this changes the return value of public function
xmlSchemaInitTypes from void to int. This shouldn't break the ABI on
most platforms.
Found when investigating #500.
|
|
|
d7daf9fd
|
2023-03-14T13:02:36
|
|
xmllint: Fix use-after-free with --maxmem
Fixes #498.
|
|
|
e7c3a4ca
|
2023-03-13T19:19:46
|
|
parser: Deprecate some parser input functions
|
|
|
2099441f
|
2023-03-13T17:51:13
|
|
parser: Stop calling xmlParserInputShrink
Introduce xmlParserShrink which takes a parser context to simplify error
handling.
|
|
|
48379394
|
2023-03-13T17:11:27
|
|
malloc-fail: Stop using XPath stack frames
There's too much code which assumes that if ctxt->value is non-null,
a value can be successfully popped off the stack. This assumption can
break with stack frames when malloc fails.
Instead of trying to fix all call sites, remove the stack frame logic.
It only offered very little protection against misbehaving extension
functions. We already check the stack size after a function call which
should be enough.
Found by OSS-Fuzz.
|
|
|
bd63d730
|
2023-03-12T17:40:55
|
|
html: Impose some length limits
Impose length limits on names, attribute values, PIs and comments,
similar to the XML parser.
|
|
|
3eb6bf03
|
2023-03-12T16:47:15
|
|
parser: Stop calling xmlParserInputGrow
Introduce xmlParserGrow which takes a parser context to simplify error
handling.
|
|
|
b51478dc
|
2023-02-24T16:21:17
|
|
Revert "malloc-fail: Avoid use-after-free after unsuccessful valuePush"
This reverts commit 6a12be77c6a94c374ab7476087edcee2ba41d9b4.
There's too much code reading ctxt->value directly and making the wrong
assumptions.
|
|
|
4f0a0fb7
|
2023-02-22T14:24:24
|
|
xinclude: Fix include guard
|
|
|
905386ec
|
2023-02-13T11:14:34
|
|
autotools: Fix make distcheck
- Add private/xinclude.h to EXTRA_DIST
- Add runsuite.log to CLEANFILES
Fixes #485.
|
|
|
6a12be77
|
2023-01-31T12:46:30
|
|
malloc-fail: Avoid use-after-free after unsuccessful valuePush
In xpath.c there's a lot of code like:
valuePush(ctxt, xmlCacheNewX());
...
valuePop(ctxt);
If xmlCacheNewX fails, no value will be pushed on the stack. If there's
no error check in between, valuePop will pop an unrelated value which
can lead to use-after-free errors.
Instead of trying to fix all call sites, we simply stop popping values
if an error was signaled. This requires to change the CHECK_TYPE macro
which is often used to determine whether a value can be safely popped.
Found with libFuzzer, see #344.
|
|
|
59b33661
|
2022-12-27T14:15:51
|
|
error: Limit number of parser errors
Reporting errors is expensive and some abusive test cases can generate
an error for each invalid input byte. This causes the parser to spend
most of the time with error handling. Limit the number of errors and
warnings to 100.
|
|
|
a41b09c7
|
2022-12-23T21:29:28
|
|
parser: Improve detection of entity loops
Set a flag to detect entity loops at once instead of processing until
the depth limit is exceeded.
|
|
|
b47ebf04
|
2022-12-21T00:02:47
|
|
parser: Deprecate xmlString*DecodeEntities
These are internal functions.
|
|
|
ce76ebfd
|
2022-12-19T20:56:23
|
|
entities: Stop counting entities
This was only used in the old version of xmlParserEntityCheck.
|
|
|
a3c8b180
|
2022-12-19T20:51:52
|
|
entities: Add entity flag for loop check
|
|
|
463bbeec
|
2022-12-19T18:39:45
|
|
entities: Rework entity amplification checks
This commit implements robust detection of entity amplification attacks,
better known as the "billion laughs" attack.
We now limit the size of the document after substitution of entities to
10 times the size before expansion. This guarantees linear behavior by
definition. There already was a similar check before, but the accounting
of "sizeentities" (size of external entities) and "sizeentcopy" (size of
all copies created by entity references) wasn't accurate.
We also need saturation arithmetic since we're historically limited to
"unsigned long" which is 32-bit on many platforms.
A maximum of 10 MB of substitutions is always allowed. This should make
use cases like DITA work which have caused problems in the past.
The old checks based on the number of entities were removed. This is
accounted for by adding a fixed cost to each entity reference.
Entity amplification checks are now enabled even if XML_PARSE_HUGE is
set. This option is mainly used to allow larger text nodes. Most users
were unaware that it also disabled entity expansion checks.
Some of the limits might be adjusted later. If this change turns out to
affect legitimate use cases, we can add a separate parser option to
disable the checks.
Fixes #294.
Fixes #345.
|
|
|
7e3f469b
|
2022-12-19T15:59:49
|
|
entities: Use flags to store '<' check results
Instead of abusing the LSB of the "checked" member, store the result
of testing for occurrence of '<' character in "flags".
Also use the flags in xmlParseStringEntityRef instead of rescanning
every time.
|
|
|
481d79d4
|
2022-12-19T15:26:46
|
|
entities: Add XML_ENT_PARSED flag
To check whether an entity was already parsed, the code previously
tested whether "checked" was non-zero or "children" was non-null. The
"children" check could be unreliable because an empty entity also
results in an empty (NULL) node list. Use a separate flag to make this
check more reliable.
|
|
|
f34f184f
|
2022-12-19T15:24:53
|
|
entities: Add "flags" member to struct xmlEntity
This will hold various flags and eventually replace the "checked"
member.
|
|
|
93a01c46
|
2022-12-08T03:58:41
|
|
libxml.h: Add comments and indentation
|
|
|
a6debffd
|
2022-12-08T03:37:24
|
|
xmlexports.h: Disable docs for internal macro XMLPUBLIC
|
|
|
3b6cc47a
|
2022-12-08T02:51:52
|
|
xmlexports.h: Remove LIBXML_FASTCALL optimization
This was an experimental and undocumented micro-optimization for
Windows which apparently required different calling conventions for
variable-argument functions, making it impossible to maintain without
domain knowledge.
|
|
|
ce9baf94
|
2022-12-08T02:48:27
|
|
Remove XMLCALL and XMLCDECL macros from public headers
|
|
|
ccb6d544
|
2022-11-27T02:09:27
|
|
Hide internal functions
These functions were never declared in public headers, so it should be
safe to hide them.
Fixes #139.
|
|
|
c16fd705
|
2022-11-25T14:52:37
|
|
xpath: Make init function private
|
|
|
53ab3840
|
2022-11-25T14:26:59
|
|
encoding: Make init function private
|
|
|
c73d464a
|
2022-11-24T15:00:03
|
|
threads: Deprecate some internal functions
|
|
|
65d381f3
|
2022-11-24T20:54:18
|
|
threads: Allocate mutexes statically
|
|
|
ed053c50
|
2022-11-25T12:27:14
|
|
dict: Make init/cleanup functions private
|
|
|
7010d877
|
2022-11-25T12:06:27
|
|
threads: Rework initialization
Make init/cleanup functions private. Merge xmlOnceInit into
xmlInitThreadsInternal.
|
|
|
9dbf1374
|
2022-11-24T20:52:57
|
|
parser: Make some module init/cleanup functions private
|
|
707ade22
|
2022-11-22T14:56:58
|
|
Visual Studio builds: Allow silencing deprecation warnings
Define XML_IGNORE_DEPRECATION_WARNINGS and the corresponding XML_POP_WARNINGS
for Visual Studio, and consequently define XML_IGNORE_FPTR_CAST_WARNINGS so
that we do not get a compiler warning on Visual Studio by doing a
__pragma(warning(pop)) without a corresponding __pragma(warning(push)).
Also correct the documentation a bit for XML_POP_WARNINGS.
|
|
b9590d5d
|
2022-11-18T11:23:23
|
|
Visual Studio: Define XML_DEPRECATED
We can mark APIs as deprecated using __declspec(deprecated) with Visual Studio
2005 and later, so add a definition of that so that we can help users avoid
using deprecated APIs when using Visual Studio as well.
For the existing GCC definition, check whether we are on GCC 3.1+ before
enabling the definition.
|
|
|
68a6518c
|
2022-11-15T18:23:33
|
|
parser: Rewrite push parser boundary checks
Remove inaccurate xmlParseCheckTransition check.
Remove non-incremental xmlParseGetLasts check.
Add functions that check for several boundary constructs more
accurately, keeping track of progress in ctxt->checkIndex.
Fixes #439.
|
|
|
2059df53
|
2022-11-14T22:27:58
|
|
buf: Deprecate static/immutable buffers
|
|
|
46cd7d22
|
2022-11-13T16:30:46
|
|
io: Remove xmlInputReadCallbackNop
In some cases, for example when using encoders, the read callback was
set to NULL, in other cases it was set to xmlInputReadCallbackNop.
xmlGROW only tested for xmlInputReadCallbackNop, resulting in errors
when parsing large encoded content from memory.
Always use a NULL callback for memory buffers to avoid ambiguities.
Fixes #262.
|
|
|
b693905f
|
2022-11-04T14:50:39
|
|
doc: Remove xmlDllMain from documentation and version script
This is a Windows-only symbol.
|
|
|
eef0a739
|
2022-10-30T12:21:20
|
|
xinclude: Implement "streaming" mode
When using xmlreader, XPointer expressions in XIncludes simply cannot
work. Expressions can reference nodes which weren't parsed yet or which
were already deleted.
After fixing nested XIncludes, we reference includes which were parsed
previously. When streaming, these nodes could have been deleted, leading
to use-after-free errors.
Disallow XPointer expressions and truncate the include table in
streaming mode.
|
|
|
2fc8d123
|
2022-10-22T19:08:43
|
|
xinclude: Make xmlXIncludeCopyNode non-recursive
Avoid call stack overflows.
Also switch to xmlStaticCopyNode which avoids duplicate namespace
definitions.
|
|
|
5bfaf230
|
2022-10-11T13:00:33
|
|
win32: Fix build with VS2013
Should fix #420.
|
|
|
a9669679
|
2022-09-09T01:44:00
|
|
error: Don't use initGenericErrorDefaultFunc
The code in xmlInitParser did only set the error handler if it was NULL
which should never happen.
|
|
|
30c8d9bb
|
2022-09-05T02:02:54
|
|
http: Simplify IPv6 checks
This should also enable IPv6 support on Windows. Untested and mostly
useless anyway, since we don't support HTTPS.
|
|
|
9e5a016e
|
2022-09-05T01:08:33
|
|
autotools: Fix network checks on Windows
|
|
|
0d901258
|
2022-09-04T16:41:43
|
|
Fix Windows compiler warnings in python/types.c
|
|
|
fe02289f
|
2022-09-04T03:19:01
|
|
Remove arg cast configure checks
We can simply cast to non-const char * unconditionally.
|
|
|
1e60c768
|
2022-09-04T01:49:41
|
|
Remove HAVE_WIN32_THREADS configuration flag
Check for LIBXML_THREAD_ENABLED and _WIN32 instead.
|
|
|
59f2f60e
|
2022-09-02T00:27:57
|
|
Remove "runtime debugging"
This doesn't seem useful as configuration option.
|
|
|
65dc8a63
|
2022-09-01T00:13:19
|
|
Make xmlNewSAXParserCtx take a const sax handler
Also improve documentation.
|
|
|
39dfb048
|
2022-08-26T02:41:15
|
|
Don't forget to install xmlversion.h
Fixes 7f7961df.
|
|
|
0f568c0b
|
2022-08-26T01:22:33
|
|
Consolidate private header files
Private functions were previously declared
- in header files in the root directory
- in public headers guarded with IN_LIBXML
- in libxml.h
- redundantly in source files that used them.
Consolidate all private header files in include/private.
|
|
|
48f84ea8
|
2022-08-25T21:31:08
|
|
Remove internal macros from parserInternals.h
Replace MOVETO_ENDTAG with code that updates line and column numbers.
|
|
|
58fc89e8
|
2022-08-25T20:57:30
|
|
Deprecate internal parser functions
|
|
|
a308c0cd
|
2022-08-25T20:18:16
|
|
Deprecate old HTML SAX API
|
|
|
51035c53
|
2022-08-25T19:53:04
|
|
Generate deprecation warnings for old SAX API
|
|
|
7f7961df
|
2022-08-25T13:47:16
|
|
Remove generated files from distribution
- libxml2.spec
- libxml-2.0.pc
- xml2-config
- include/libxml/xmlversion.h
- python/libxml2.py
- python/libxml2-export.c
- python/libxml2-py.c
- python/libxml2-py.h
- python/libxml2class.py
- python/libxml2class.txt
- python/setup.py
|
|
|
34a050cd
|
2022-08-24T16:35:58
|
|
Move some HTML functions to correct header file
|
|
|
9a82b94a
|
2022-08-24T04:21:58
|
|
Introduce xmlNewSAXParserCtxt and htmlNewSAXParserCtxt
Add API functions to create a parser context with a custom SAX handler
without having to mess with ctxt->sax manually.
|
|
|
92bb889b
|
2022-08-24T00:03:44
|
|
Don't index anything in DOC_DISABLE sections
Somewhat misleadingly, the DOC_DISABLE directive only disabled warnings.
Now we really stop the documentation generator from indexing.
This results in additional warnings for xmlThrDef* functions. This should
be fixed by documenting or deprecating them.
|
|
|
75b5bc2b
|
2022-08-23T20:57:49
|
|
Deprecate some global variables
Most of these should be completely unused.
|
|
|
5264fa11
|
2022-08-18T20:17:27
|
|
Fix warnings from apibuild.py
|
|
|
e986d09c
|
2022-07-15T14:02:26
|
|
Skip incorrectly opened HTML comments
Commit 4fd69f3e fixed handling of '<' characters not followed by an
ASCII letter. But a '<!' sequence followed by invalid characters should
be treated as bogus comment and skipped.
Fixes #380.
|
|
e9270ef0
|
2022-05-06T10:44:03
|
|
fix Schematron spelling
|
|
|
67070107
|
2022-04-20T23:17:14
|
|
Add configuration flag for XPointer locations support
Add a new configuration flag that controls whether the outdated support
for XPointer locations (ranges and points) is enabled.
--with-xptr-locs # Autotools
LIBXML2_WITH_XPTR_LOCS # CMake
The latest spec for what it essentially an XPath extension seems to be
this working draft from 2002:
https://www.w3.org/TR/xptr-xpointer/
The xpointer() scheme is listed as "being reviewed" in the XPointer
registry since at least 2006. libxml2 seems to be the only modern
software that tries to implement this spec, but the code has many bugs
and quality issues.
The flag defaults to "off" and support for this extensions has to be
requested explicitly. The relevant API functions are deprecated.
|
|
|
aab584dc
|
2022-03-06T23:23:43
|
|
Clean up encoding switching code
- Remove xmlSwitchToEncodingInt which was basically just a wrapper
around xmlSwitchInputEncodingInt.
- Simplify xmlSwitchEncoding.
- Improve error handling in xmlSwitchInputEncodingInt.
- Deprecate xmlSwitchInputEncoding.
|
|
|
2070ade6
|
2022-03-06T23:19:04
|
|
Undeprecate schema init functions
These functions aren't called from xmlInitParser, so it's better to keep
them public for now.
|
|
|
40483d0c
|
2022-03-06T13:55:48
|
|
Deprecate module init and cleanup functions
These functions shouldn't be part of the public API. Most init
functions are only thread-safe when called from xmlInitParser. Global
variables should only be cleaned up by calling xmlCleanupParser.
|
|
|
422176ad
|
2022-03-05T03:18:58
|
|
Don't set HAVE_WIN32_THREADS in win32config.h
This flag must be set on the command line.
|
|
|
4a8c71eb
|
2022-03-04T03:35:57
|
|
Remove DOCBparser
This code has been broken and deprecated since version 2.6.0, released
in 2003. Because of a bug in commit 961b535c, DOCBparser.c was never
compiled since 2012. I couldn't find a Debian package using any of its
symbols, so it seems safe to remove this module.
|
|
|
ebb17970
|
2022-03-04T02:31:59
|
|
Remove unneeded #includes
|
|
|
f025d6eb
|
2022-03-04T01:32:04
|
|
Use stdint.h with newer MSVC
stdint.h is available since Visual Studio 2010.
|
|
|
84085a26
|
2022-03-04T01:23:14
|
|
Remove cruft from win32config.h
|
|
|
21ddad52
|
2022-03-04T01:07:40
|
|
Remove ICONV_CONST test
We can simply cast the offending pointer to (void *).
|
|
|
ae3cf483
|
2022-03-03T23:57:59
|
|
Remove isinf/isnan emulation in win32config.h
There's already a better fallback in xpath.c.
|