kmx git

Commit	Date	Message
26b71d60	2020-02-07T14:36:10	tests: iterator: fix iterator expecting too few items The testcase iterator::workdir::filesystem_gunk sets up quite a lot of directories, which is why it only runs in case GITTEST_INVASIVE_SPEED is set in the environment. Because we do not run our default CI with this variable, we didn't notice commit 852c83ee4 (refs: refuse to delete HEAD, 2020-01-15) breaking the test as it introduced a new reference to the "testrepo" repository. Fix the oversight by increasing the number of expected iterator items.
46228d86	2020-02-06T11:10:27	transports: http: fix custom headers not being applied In commit b9c5b15a7 (http: use the new httpclient, 2019-12-22), the HTTP code got refactored to extract a generic HTTP client that operates independently of the Git protocol. Part of refactoring was the creation of a new `git_http_request` struct that encapsulates the generation of requests. Our Git-specific HTTP transport was converted to use that in `generate_request`, but during the process we forgot to set up custom headers for the `git_http_request` and as a result we do not send out these headers anymore. Fix the issue by correctly setting up the request's custom headers and add a test to verify we correctly send them.
93a9044f	2020-01-31T08:49:34	fetchhead: strip credentials from remote URL If fetching from an anonymous remote via its URL, then the URL gets written into the FETCH_HEAD reference. This is mainly done to give valuable context to some commands, like for example git-merge(1), which will put the URL into the generated MERGE_MSG. As a result, what gets written into FETCH_HEAD may become public in some cases. This is especially important considering that URLs may contain credentials, e.g. when cloning 'https://foo:bar@example.com/repo' we persist the complete URL into FETCH_HEAD and put it without any kind of sanitization into the MERGE_MSG. This is obviously bad, as your login data has now just leaked as soon as you do git-push(1). When writing the URL into FETCH_HEAD, upstream git does strip credentials first. Let's do the same by trying to parse the remote URL as a "real" URL, removing any credentials and then re-formatting the URL. In case this fails, e.g. when it's a file path or not a valid URL, we just fall back to using the URL as-is without any sanitization. Add tests to verify our behaviour.
3f54ba8b	2020-01-18T13:51:40	credential: change git_cred to git_credential We avoid abbreviations where possible; rename git_cred to git_credential. In addition, we have standardized on a trailing `_t` for enum types, instead of using "type" in the name. So `git_credtype_t` has become `git_credential_t` and its members have become `GIT_CREDENTIAL` instead of `GIT_CREDTYPE`. Finally, the source and header files have been renamed to `credential` instead of `cred`. Keep previous name and values as deprecated, and include the new header files from the previous ones.
7fd9b3f5	2020-01-01T20:48:15	ci: add NTLM tests Download poxygit, a debugging git server, and clone from it using NTLM, both IIS-style (with connection affinity) and Apache-style ("broken", requiring constant reauthentication).
bf55facf	2019-10-25T12:24:34	tests: allow users to use expect/continue
dcd3b815	2019-12-13T15:28:57	tests: support CLAR_TRACE_LEVEL The CLAR_TRACE_LEVEL environment variable was supported when building with GIT_TRACE. Now we always build with GIT_TRACE, but that variable is not provided to tests. Simply support clar tracing always.
fe4efe2e	2019-12-03T10:17:30	tests: test that clone returns 4321 This conditional was backwards. We should instead test that clone returns 4321, not that 4321 returns clone.
e995f74e	2019-12-06T15:39:08	net: introduce git_net_url_joinpath Provide a mechanism to add a path and query string to an existing url so that we can easily append `/info/refs?...` type url segments to a url given to us by a user.
471daeea	2019-12-01T14:00:49	net: refactor gitno redirect handling Move the redirect handling into `git_net_url` for consistency.
e5fb5fe5	2019-10-20T17:19:01	ci: perform SPNEGO tests Attempt to obtain a Kerberos ticket from LIBGIT2.ORG and then clone the Negotiate-protected site at test.libgit2.org with that ticket.
94beb3a3	2020-01-18T14:03:23	merge: update enum type name for consistency libgit2 does not use `type_t` suffixes as it's redundant; thus, rename `git_merge_diff_type_t` to `git_merge_diff_t` for consistency.
1908884d	2020-01-17T08:34:30	Merge pull request #5361 from csware/no-return-freed_object Do not return free'd git_repository object on error
470a05d0	2020-01-16T17:53:50	Do not return free'd git_repository object on error Regression introduced in commit dde6d9c706bf1ecab545da55ab874a016587af1f. This issue causes lots of crashes in TortoiseGit. Signed-off-by: Sven Strickroth <email@cs-ware.de>
852c83ee	2020-01-15T13:31:21	refs: refuse to delete HEAD This requires adding a new symbolic ref to the testrepo fixture. Some of the existing tests attempt to delete HEAD, expecting a different failure. Introduce and use a non-HEAD symbolic ref instead. Adjust a few other tests as needed. Fixes #5357
7d55bee6	2020-01-10T12:44:51	win32: fix relative symlinks pointing into dirs On Windows platforms, we need some logic to emulate symlink(3P) defined by POSIX. As unprivileged symlinks on Windows are a rather new feature, our current implementation is comparatively new and still has some rough edges in special cases. One such case is relative symlinks. While relative symlinks to files in the same directory work as expected, libgit2 currently fails to create reltaive symlinks pointing into other directories. This is due to the fact that we forgot to translate the Unix-style target path to Windows-style. Most importantly, we are currently not converting directory separators from "/" to "\". Fix the issue by calling `git_win32_path_canonicalize` on the target. Add a test that verifies our ability to create such relative links across directories.
7142964f	2019-12-13T10:56:19	netops: handle intact query parameters in service_suffix removal Some servers leave the query parameters intact in the Location header when responding with a redirect. The service_suffix removal check as written assumed that the server removed them. Handle both cases. Along with PR #5325, this fixes #5321. There are two new tests. The first already passed; the second previously failed.
2dc7b5ef	2019-12-14T12:53:04	tests: pack: add missing asserts around `git_packbuilder_write`
11e8ee1f	2020-01-06T15:41:18	tests: submodule: verify setup of relative URLs When setting up relative URLs for a submodule, then we resolve it to the actual location and write that into ".git/config" instead of writing the relative value. We do not yet have a test to nail down this behaviour, which is now being added by this commit.
2f6f10bb	2019-12-13T13:35:40	Merge pull request #5300 from tiennou/fix/branch-documentation branch: clarify documentation around branches
97b8491b	2019-12-08T15:25:52	refs: rename git_reference__set_name to git_reference__realloc As git_reference__name will reallocate storage to account for longer names (it's actually allocator-dependent), it will cause all existing pointers to the old object to become dangling, as they now point to freed memory. Fix the issue by renaming to a more descriptive name, and pass a pointer to the actual reference that can safely be invalidated if the realloc succeeds.
b3178587	2019-12-13T08:35:25	Merge pull request #5333 from lrm29/attr_binary_macro attr: Update definition of binary macro
cf286d5e	2019-12-12T10:58:56	attr: Update definition of binary macro
14ff3516	2019-12-03T23:15:47	path: support non-ascii drive letters on dos Windows/DOS only supports drive letters that are alpha characters A-Z. However, you can `subst` any one-character as a drive letter, including numbers or even emoji. Test that we can identify emoji as drive letters.
85d4ff77	2019-12-03T19:50:18	index: ensure that we respect core.protectNTFS=false Users may want to turn off core.protectNTFS, perhaps to import (and then repair) a broken tree. Ensure that core.protectNTFS=false is honored.
ba4c769b	2019-12-03T23:23:02	tree: ensure we protect NTFS paths everywhere
e4034dfa	2019-12-03T19:24:59	path: protect NTFS everywhere Enable core.protectNTFS by default everywhere and in every codepath, not just on checkout.
d9c0c9cf	2019-12-03T19:17:41	test: ensure we can't add a protected path Test that when we enable core.protectNTFS that we cannot add platform-specific invalid paths to the index.
72df1cd8	2019-12-03T19:01:00	test: improve badname verification test The name of the `add_invalid_filename` function suggests that we _want_ to add an invalid filename. Rename the function to show that we expect to _fail_ to add the invalid filename.
f3b28604	2019-12-03T18:57:16	test: ensure treebuilder validate new protection rules Ensure that the new protection around .git::$INDEX_ALLOCATION rules are enabled for using the treebuilder when core.protectNTFS is set.
336991db	2019-12-03T18:56:31	test: ensure index adds validate new protection rules Ensure that the new protection around .git::$INDEX_ALLOCATION rules are enabled for adding to the index when core.protectNTFS is set.
a3cbd204	2019-12-03T18:49:23	test: improve badname verification test The name of the `write_invalid_filename` function suggests that we _want_ to write an invalid filename. Rename the function to show that we expect to _fail_ to write the invalid filename.
e1832eb2	2019-09-18T16:33:18	path: also guard `.gitmodules` against NTFS Alternate Data Streams We just safe-guarded `.git` against NTFS Alternate Data Stream-related attack vectors, and now it is time to do the same for `.gitmodules`. Note: In the added regression test, we refrain from verifying all kinds of variations between short names and NTFS Alternate Data Streams: as the new code disallows _all_ Alternate Data Streams of `.gitmodules`, it is enough to test one in order to know that all of them are guarded against. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
3f7851ea	2019-09-18T14:32:05	Disallow NTFS Alternate Data Stream attacks, even on Linux/macOS A little-known feature of NTFS is that it offers to store metadata in so-called "Alternate Data Streams" (inspired by Apple's "resource forks") that are copied together with the file they are associated with. These Alternate Data Streams can be accessed via `<file name>:<stream name>:<stream type>`. Directories, too, have Alternate Data Streams, and they even have a default stream type `$INDEX_ALLOCATION`. Which means that `abc/` and `abc::$INDEX_ALLOCATION/` are actually equivalent. This is of course another attack vector on the Git directory that we definitely want to prevent. On Windows, we already do this incidentally, by disallowing colons in file/directory names. While it looks as if files'/directories' Alternate Data Streams are not accessible in the Windows Subsystem for Linux, and neither via CIFS/SMB-mounted network shares in Linux, it _is_ possible to access them on SMB-mounted network shares on macOS. Therefore, let's go the extra mile and prevent this particular attack _everywhere_. To keep things simple, let's just disallow any Alternate Data Stream of `.git`. This is libgit2's variant of CVE-2019-1352. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
64c612cc	2019-09-18T15:25:02	Protect against 8.3 "short name" attacks also on Linux/macOS The Windows Subsystem for Linux (WSL) is getting increasingly popular, in particular because it makes it _so_ easy to run Linux software on Windows' files, via the auto-mounted Windows drives (`C:\` is mapped to `/mnt/c/`, no need to set that up manually). Unfortunately, files/directories on the Windows drives can be accessed via their _short names_, if that feature is enabled (which it is on the `C:` drive by default). Which means that we have to safeguard even our Linux users against the short name attacks. Further, while the default options of CIFS/SMB-mounts seem to disallow accessing files on network shares via their short names on Linux/macOS, it _is_ possible to do so with the right options. So let's just safe-guard against short name attacks _everywhere_. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
d29d4de2	2019-09-18T15:08:56	cl_git_fail: do not report bogus error message When we expect a checkout operation to fail, but it succeeds, we actually do not want to see the error messages that were generated in the meantime for errors that were handled gracefully by the code (e.g. when an object could not be found in a pack: in this case, the next backend would have been given a chance to look up the object, and probably would have found it because the checkout succeeded, after all). Which means that in the specific case of `cl_git_fail()`, we actually want to clear the global error state _after_ evaluating the command: we know that any still-available error would be bogus, seeing as the command succeeded (unexpectedly). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
7f6fdb82	2019-12-01T14:11:22	Merge pull request #5312 from pks-t/pks/patch-base85-overflow patch_parse: fix out-of-bounds reads caused by integer underflow
7f20778b	2019-11-29T09:14:04	Merge pull request #5311 from pks-t/pks/clar-trace-warning tests: fix compiler warning if tracing is disabled
361ebbcb	2019-11-28T15:36:40	tests: config: only test parsing huge file with GITTEST_INVASIVE_SPEED The test in config::stress::huge_section_with_many_values takes quite a long time to execute. Hide it behind the GITTEST_INVASIVE_SPEED environment varibale to not needlessly blow up execution time of tests. As this environment variable is being set by the continuous integration, we will execute it regularly anyway.
33e6c402	2019-11-28T15:26:36	patch_parse: fix out-of-bounds reads caused by integer underflow The patch format for binary files is a simple Base85 encoding with a length byte as prefix that encodes the current line's length. For each line, we thus check whether the line's actual length matches its expected length in order to not faultily apply a truncated patch. This also acts as a check to verify that we're not reading outside of the line's string: if (encoded_len > ctx->parse_ctx.line_len - 1) { error = git_parse_err(...); goto done; } There is the possibility for an integer underflow, though. Given a line with a single prefix byte, only, `line_len` will be zero when reaching this check. As a result, subtracting one from that will result in an integer underflow, causing us to assume that there's a wealth of bytes available later on. Naturally, this may result in an out-of-bounds read. Fix the issue by checking both `encoded_len` and `line_len` for a non-zero value. The binary format doesn't make use of zero-length lines anyway, so we need to know that there are both encoded bytes and remaining characters available at all. This patch also adds a test that works based on the last error message. Checking error messages is usually too tightly coupled, but in fact parsing the patch failed even before the change. Thus the only possibility is to use e.g. Valgrind, but that'd result in us not catching issues when run without Valgrind. As a result, using the error message is considered a viable tradeoff as we know that we didn't start decoding Base85 in the first place.
1d470a71	2019-11-28T14:45:15	tests: fix compiler warning if tracing is disabled If building libgit2's test suite with tracing disabled, then the compiler will emit a warning due to the unused `message_prefix` function. Fix the issue by wrapping the whole file into ifdef's for `GIT_TRACE` and providing separate empty function implementations for both `cl_global_trace_register` and `cl_global_trace_disable`.
fb439c97	2019-11-28T14:41:58	Merge pull request #5306 from herrerog/patchid diff: complete support for git patchid
61176a9b	2019-11-28T14:31:16	Merge pull request #5243 from pks-t/pks/config-optimize-mem Memory optimizations for config entries
ece5bb5e	2019-11-07T14:10:00	diff: make patchid computation work with all types of commits. Current implementation of patchid is not computing a correct patchid when given a patch where, for example, a new file is added or removed. Some more corner cases need to be handled to have same behavior as git patch-id command. Add some more tests to cover those corner cases. Signed-off-by: Gregory Herrero <gregory.herrero@oracle.com>
b63ad958	2019-11-28T13:19:50	Merge pull request #5309 from libgit2/ethomson/trace Improve trace support in tests
0e5243b7	2019-11-28T12:42:36	Merge pull request #5123 from libgit2/ethomson/off_t Move `git_off_t` to `git_object_size_t`
b7f70bc2	2019-11-27T12:36:17	tests: optionally show test execution tracing Only show test trace execution when the CLAR_TRACE_TESTS environment variable is set. This reduces the noise during tracing.
85efe896	2019-11-27T12:34:10	tests: display trace level with prefix in tests
6460e8ab	2019-06-23T18:13:29	internal: use off64_t instead of git_off_t Prefer `off64_t` internally.
0005c77a	2019-11-24T15:49:49	test: add an azure repos test We currently talk to Azure Repos for executing an online test (online::clone::path_whitespace). Add a simpler test to talk to Azure Repos to make it obvious that strange test failures are not likely the whitespace in the path, but actually a function of talking to Azure Repos itself.
048e94ad	2019-11-07T14:13:14	patch_parse: correct parsing of patch containing not shown binary data. When not shown binary data is added or removed in a patch, patch parser is currently returning 'error -1 - corrupt git binary header at line 4'. Fix it by correctly handling case where binary data is added/removed. Signed-off-by: Gregory Herrero <gregory.herrero@oracle.com>
de7659cc	2019-11-10T18:44:56	patch_parse: use paths from "---"/"+++" lines for binary patches For some patches, it is not possible to derive the old and new file paths from the patch header's first line, most importantly when they contain spaces. In such a case, we derive both paths from the "---" and "+++" lines, which allow for non-ambiguous parsing. We fail to use these paths when parsing binary patches without data, though, as we always expect the header paths to be filled in. Fix this by using the "---"/"+++" paths by default and only fall back to header paths if they aren't set. If neither of those paths are set, we just return an error. Add two tests to verify this behaviour, one of which would have previously caused a segfault.
01ea911b	2019-11-06T06:04:55	Merge pull request #5299 from pks-t/pks/config-mem-snapshots config_mem: implement support for snapshots
146e5bf7	2019-11-06T07:27:35	config_mem: implement support for snapshots Similar as in commit dadbb33b6 (Fix crash if snapshotting a config_snapshot, 2019-11-01), let's implement snapshots for in-memory configuration entries. As this deletes more code than it adds, it doesn't make any sense to not allow for this and allows users to treat config backends mostly the same.
de543e29	2019-11-05T22:44:27	patch_parse: fix segfault when header path contains whitespace only When parsing header paths from a patch, we reject any patches with empty paths as malformed patches. We perform the check whether a path is empty before sanitizing it, though, which may lead to a path becoming empty after the check, e.g. if we have trimmed whitespace. This may lead to a segfault later when any part of our patching logic actually references such a path, which may then be a `NULL` pointer. Fix the issue by performing the check after sanitizing. Add tests to catch the issue as they would have produced a segfault previosuly.
b7dcea04	2019-09-26T15:06:12	config_entries: micro-optimize storage of multivars Multivars are configuration entries that have many values for the same name; we can thus micro-optimize this case by just retaining the name of the first configuration entry and freeing all the others, letting them point to the string of the first entry. The attached test case is an extreme example that demonstrates this. It contains a section name that is approximately 500kB in size with 20.000 entries "a=b". Without the optimization, this would require at least 20000500kB bytes, which is around 10GB. With this patch, it only requires 500kB+200001B=20500kB. The obvious culprit here is the section header, which we repeatedly include in each of the configuration entry's names. This makes it very easier for an adversary to provide a small configuration file that disproportionally blows up in memory during processing and is thus a feasible way for a denial-of-service attack. Unfortunately, we cannot fix the root cause by e.g. having a separate "section" field that may easily be deduplicated due to the `git_config_entry` structure being part of our public API. So this micro-optimization is the best we can do for now.
82d7a114	2019-11-05T11:18:14	Merge pull request #5293 from csware/config_snapshot-snapshot Fix crash if snapshotting a config_snapshot
bf2911d7	2019-11-02T07:30:32	Merge pull request #5275 from pks-t/pks/reflogs-with-newlines reflogs: fix behaviour around reflogs with newlines
dadbb33b	2019-11-01T18:55:54	Fix crash if snapshotting a config_snapshot Signed-off-by: Sven Strickroth <email@cs-ware.de>
0974e02f	2019-10-30T20:35:48	commit: add failing tests for object checking for git_commit_with_signature There can be a significant difference between the system where we created the buffer (if at all) and when the caller provides us with the contents of a commit. Provide some test cases (we have to adapt the existing ones because they refer to trees and commits which do not exist).
2a7d6de3	2019-10-29T07:52:31	Merge pull request #5276 from pks-t/pks/patch-fuzzing-fixes patch_parse: fixes for fuzzing errors
a31f4c4b	2019-10-24T13:16:03	Merge pull request #5227 from ddevault/check apply: add GIT_APPLY_CHECK
c405f231	2019-10-24T10:26:43	Merge pull request #5264 from henkesn/refs-unlock-on-commit refs: unlock unmodified refs on transaction commit
02af1fcb	2019-09-14T14:03:36	apply: add GIT_APPLY_CHECK This adds an option which will check if a diff is applicable without actually applying it; equivalent to git apply --check.
37141ff7	2019-10-21T18:56:59	patch_parse: detect overflow when calculating old/new line position When the patch contains lines close to INT_MAX, then it may happen that we end up with an integer overflow when calculating the line of the current diff hunk. Reject such patches as unreasonable to avoid the integer overflow. As the calculation is performed on integers, we introduce two new helpers `git__add_int_overflow` and `git__sub_int_overflow` that perform the integer overflow check in a generic way.
468e3ddc	2019-10-19T16:48:11	patch_parse: fix out-of-bounds read with No-NL lines We've got two locations where we copy lines into the patch. The first one is when copying normal " ", "-" or "+" lines, while the second location gets executed when we copy "\ No newline at end of file" lines. While the first one correctly uses `git__strndup` to copy only until the newline, the other one doesn't. Thus, if the line occurs at the end of the patch and if there is no terminating NUL character, then it may result in an out-of-bounds read. Fix the issue by using `git__strndup`, as was already done in the other location. Furthermore, add allocation checks to both locations to detect out-of-memory situations.
6c6c15e9	2019-10-19T15:52:35	patch_parse: reject empty path names When parsing patch headers, we currently accept empty path names just fine, e.g. a line "--- \n" would be parsed as the empty filename. This is not a valid patch format and may cause `NULL` pointer accesses at a later place as `git_buf_detach` will return `NULL` in that case. Reject such patches as malformed with a nice error message.
223e7e43	2019-10-19T15:42:54	patch_parse: reject patches with multiple old/new paths It's currently possible to have patches with multiple old path name headers. As we didn't check for this case, this resulted in a memory leak when overwriting the old old path with the new old path because we simply discarded the old pointer. Instead of fixing this by free'ing the old pointer, we should reject such patches altogether. It doesn't make any sense for the "---" or "+++" markers to occur multiple times within a patch n the first place. This also implicitly fixes the memory leak.
7968e90f	2019-10-18T12:33:07	refdb_fs: properly parse corrupted reflogs In previous versions, libgit2 could be coerced into writing reflog messages with embedded newlines into the reflog by using `git_stash_save` with a message containing newlines. While the root cause is fixed now, it was noticed that upstream git is in fact able to read such corrupted reflog messages just fine. Make the reflog parser more lenient in order to just skip over malformatted reflog lines to bring us in line with git. This requires us to change an existing test that verified that we do indeed _fail_ to parse such logs.
d8233feb	2019-10-18T09:24:14	reflog: allow adding entries with newlines in their message Currently, the reflog disallows any entries that have a message with newlines, as that would effectively break the reflog format, which may contain a single line per entry, only. Upstream git behaves a bit differently, though, especially when considering stashes: instead of rejecting any reflog entry with newlines, git will simply replace newlines with spaces. E.g. executing 'git stash push -m "foo\nbar"' will create a reflog entry with "foo bar" as entry message. This commit adjusts our own logic to stop rejecting commit messages with newlines. Previously, this logic was part of `git_reflog_append`, only. There is a second place though where we add reflog entries, which is the serialization code in the filesystem refdb. As it didn't contain any sanity checks whatsoever, the refdb would have been perfectly happy to write malformatted reflog entries to the disk. This is being fixed with the same logic as for the reflog itself.
c9464bf7	2019-10-17T18:33:12	Merge pull request #5273 from dlax/parse-diff-without-extended-headers patch_parse: handle patches without extended headers
73e9535d	2019-10-17T13:43:26	tests: submodule: test cloning edge cases Add two more tests that verify our behaviour in some edge cases, notably when cloning into a non-empty directory and when cloning the same submodule twice.
de412fc2	2019-10-17T13:36:22	tests: submodule: make use of sandboxes to clean repos The test submodule::add::submodule_clone doesn't use a sandbox, and thus the created repo will not get deleted after the test has finished. Convert the test to use the empty standard repo sandbox instead to fix this.
09b1ac11	2019-10-17T13:32:22	tests: submodule: fix tests for cloning submodules The test submodule::add::homemade_clone unfortunately doesn't test what's expected, but does instead clone the submodule to a directory that is outside of the parent repository. Fixing this by cloning to the correct location isn't possible, though, as `git_submodule_add_setup` will have pre-created a ".git" file already, which will cause `git_clone` to error out. As it's not possible to perform the clone without fiddling around with the repo's layout, let's just remove this test as that is in fact what the new `git_submodule_clone` function is for.
47531f47	2019-10-11T12:44:09	refs: unlock unmodified refs on transaction commit Refs which are locked in a transaction without an altered target, still should to be unlocked on `git_transaction_commit`. `git_transaction_free` also unlocks refs but the moment of calling of `git_transaction_free` cannot be controlled in all situations. Some binding libs call `git_transaction_free` on garbage collection or not at all if the application exits before and don't provide public access to `git_transaction_free`. It is better to release locks as soon as possible.
3c5d78bd	2019-05-01T16:16:26	submodule: provide a wrapper for simple submodule clone steps
11de594f	2019-10-16T22:11:33	patch_parse: handle patches without extended headers Extended header lines (especially the "index <hash>..<hash> <mode>") are not required by "git apply" so it import patches. So we allow the from-file/to-file lines (--- a/file\n+++ b/file) to directly follow the git diff header. This fixes #5267.
ef5a3851	2019-10-11T07:47:17	Merge pull request #5257 from henkesn/master Fix file locking on POSIX OS
1f9b4970	2019-10-11T07:44:10	Merge pull request #5260 from pks-t/pks/cmake3 cmake: update minimum CMake version to v3.5.1
3335a034	2019-10-10T15:28:46	refs: fix locks getting forcibly removed The flag GIT_FILEBUF_FORCE currently does two things: 1. It will cause the filebuf to create non-existing leading directories for the file that is about to be written. 2. It will forcibly remove any pre-existing locks. While most call sites actually do want (1), they do not want to remove pre-existing locks, as that renders the locking mechanisms effectively useless. Introduce a new flag `GIT_FILEBUF_CREATE_LEADING_DIRS` to separate both behaviours cleanly from each other and convert callers to use it instead of `GIT_FILEBUF_FORCE` to have them honor locked files correctly. As this conversion removes all current users of `GIT_FILEBUF_FORCE`, this commit removes the flag altogether.
6716e2f3	2019-10-10T12:34:45	Merge pull request #5248 from dlax/parse-patch-empty-files patch_parse: handle patches with new empty files
ebabb88f	2019-10-10T09:25:32	cmake: update minimum CMake version to v3.5.1 Back in commit cf9f34521 (cmake: bump minimum version to 2.8.11, 2017-09-06), we have bumped the minimum CMake version to require at least v2.8.11. The main hold-backs back then were distributions like RHEL/CentOS as well as Ubuntu Trusty, which caused us to not target a more modern version. Nowadays, Ubuntu Trusty has been EOL'd and CentOS 6 has CMake v3.6.1 available via the EPEL6 repository, and thus it seems fair to upgrade to a more recent version. Going through repology [1], one can see that all supported mainstream distributions do in fact have CMake 3 available. Going through the list, the minimum version that is supported by all mainstream distros is in fact v3.5.1: - CentOS 6 via EPEL6: 3.6.1 - Debian Oldstable: 3.7.2 - Fedora 26: 3.8.2 - OpenMandriva 3.x: 3.5.1 - Slackware 14.2: 3.5.2 - Ubuntu 16.04: 3.5.1 Consequentally, let's upgrade CMake to the minimum version of 3.5.1 and remove all the version CMake checks that aren't required anymore. [1]: https://repology.org/project/cmake/versions
63307cba	2019-09-28T17:32:18	Merge pull request #5226 from pks-t/pks/regexp-api regexp: implement a new regular expression API
b61810bf	2019-09-28T15:52:25	patch_parse: handle patches with new empty files Patches containing additions of empty files will not contain diff data but will end with the index header line followed by the terminating sequence "-- ". We follow the same logic as in cc4c44a and allow "-- " to immediately follow the index header.
f585b129	2019-09-12T14:29:28	posix: remove superseded POSIX regex wrappers The old POSIX regex wrappers have been superseded by our own regexp API that provides a higher-level abstraction. Remove the POSIX wrappers in favor of the new one.
d77378eb	2019-09-13T08:54:26	regexp: implement new regular expression API We currently support a set of different regular expression backends with PCRE, PCRE2, regcomp(3P) and regcomp_l(3). The current implementation of this is done via a simple POSIX wrapper that either directly uses supplied functions or that is a very small wrapper. To support PCRE and PCRE2, we use their provided <pcreposix.h> and <pcre2posix.h> wrappers. These wrappers are implemented in such a way that the accompanying libraries pcre-posix and pcre2-posix provide the same symbols as the libc ones, namely regcomp(3P) et al. This works out on some systems just fine, most importantly on glibc-based ones, where the regular expression functions are implemented as weak aliases and thus get overridden by linking in the pcre{,2}-posix library. On other systems we depend on the linking order of libc and pcre library, and as libc always comes first we will end up with the functions of the libc implementation. As a result, we may use the structures `regex_t` and `regmatch_t` declared by <pcre{,2}posix.h>, but use functions defined by the libc, leading to segfaults. The issue is not easily solvable. Somed distributions like Debian have resolved this by patching PCRE and PCRE2 to carry custom prefixes to all the POSIX function wrappers. But this is not supported by upstream and thus inherently unportable between distributions. We could instead try to modify linking order, but this starts becoming fragile and will not work e.g. when libgit2 is loaded via dlopen(3P) or similar ways. In the end, this means that we simply cannot use the POSIX wrappers provided by the PCRE libraries at all. Thus, this commit introduces a new regular expression API. The new API is on a tad higher level than the previous POSIX abstraction layer, as it tries to abstract away any non-portable flags like e.g. REG_EXTENDED, which has no equivalents in all of our supported backends. As there are no users of POSIX regular expressions that do _not_ reguest REG_EXTENDED this is fine to be abstracted away, though. Due to the API being higher-level than before, it should generally be a tad easier to use than the previous one. Note: ideally, the new API would've been called `git_regex_foobar` with a file "regex.h" and "regex.c". Unfortunately, this is currently impossible to implement due to naming clashes between the then-existing "regex.h" and <regex.h> provided by the libc. As we add the source directory of libgit2 to the header search path, an include of <regex.h> would always find our own "regex.h". Thus, we have to take the bitter pill of adding one more character to all the functions to disambiguate the includes. To improve guarantees around cross-backend compatibility, this commit also brings along an improved regular expression test suite core::regexp.
174b7a32	2019-09-19T12:24:06	buffer: fix printing into out-of-memory buffer Before printing into a `git_buf` structure, we always call `ENSURE_SIZE` first. This macro will reallocate the buffer as-needed depending on whether the current amount of allocated bytes is sufficient or not. If `asize` is big enough, then it will just do nothing, otherwise it will call out to `git_buf_try_grow`. But in fact, it is insufficient to only check `asize`. When we fail to allocate any more bytes e.g. via `git_buf_try_grow`, then we set the buffer's pointer to `git_buf__oom`. Note that we touch neither `asize` nor `size`. So if we just check `asize > targetsize`, then we will happily let the caller of `ENSURE_SIZE` proceed with an out-of-memory buffer. As a result, we will print all bytes into the out-of-memory buffer instead, resulting in an out-of-bounds write. Fix the issue by having `ENSURE_SIZE` verify that the buffer is not marked as OOM. Add a test to verify that we're not writing into the OOM buffer.
208f1d7a	2019-09-19T12:46:37	buffer: fix infinite loop when growing buffers When growing buffers, we repeatedly multiply the currently allocated number of bytes by 1.5 until it exceeds the requested number of bytes. This has two major problems: 1. If the current number of bytes is tiny and one wishes to resize to a comparatively huge number of bytes, then we may need to loop thousands of times. 2. If resizing to a value close to `SIZE_MAX` (which would fail anyway), then we probably hit an infinite loop as multiplying the current amount of bytes will repeatedly result in integer overflows. When reallocating buffers, one typically chooses values close to 1.5 to enable re-use of resulting memory holes in later reallocations. But because of this, it really only makes sense to use a factor of 1.5 _once_, but not looping until we finally are able to fit it. Thus, we can completely avoid the loop and just opt for the much simpler algorithm of multiplying with 1.5 once and, if the result doesn't fit, just use the target size. This avoids both problems of looping extensively and hitting overflows. This commit also adds a test that would've previously resulted in an infinite loop.
8bf0f7eb	2019-09-09T13:00:27	cred: separate public interface from low-level details
5d8a4659	2019-09-13T10:31:49	Merge pull request #5195 from tiennou/fix/commitish-smart-push smart: use push_glob instead of manual filtering
b545be3d	2019-09-10T11:14:36	open:fix memory leak when passing NULL to git_repository_open_ext
c3a7892f	2019-09-09T13:10:24	Merge pull request #5209 from mkostyuk/apply-wrong-patch apply: Fix a patch corruption related to EOFNL handling
17d6cd45	2019-09-09T13:06:22	Merge pull request #5210 from buddyspike/master ignore: correct handling of nested rules overriding wild card unignore
585fbd74	2019-08-28T23:18:31	apply: Test for EOFNL mishandling when several hunks are processed Introduce an unit test to validate that git_apply__patch() properly handles EOFNL changes in case of patches with several hunks.
a7b4b639	2019-08-24T12:14:31	ignore: correct handling of nested rules overriding wild card unignore problem: filesystem_iterator loads .gitignore files in top-down order. subsequently, ignore module evaluates them in the order they are loaded. this creates a problem if we have unignored a rule (using a wild card) in a sub dir and ignored it again in a level further below (see the test included in this patch). solution: process ignores in reverse order. closes #4963
5fc27aac	2019-08-27T13:38:08	Merge pull request #5208 from mkostyuk/apply-removed-new-file apply: git_apply_to_tree fails to apply patches that add new files
5498c318	2019-08-27T13:10:53	apply: free test data
6de48085	2019-08-27T11:29:24	Merge pull request #5189 from libgit2/ethomson/attrs_from_head Optionally read `.gitattributes` from HEAD
aaa48d06	2019-08-27T11:26:50	Merge pull request #5196 from pks-t/pks/config-include-onbranch config: implement "onbranch" conditional
8cbef12d	2019-08-08T11:52:54	util: do not perform allocations in insertsort Our hand-rolled fallback sorting function `git__insertsort_r` does an in-place sort of the given array. As elements may not necessarily be pointers, it needs a way of swapping two values of arbitrary size, which is currently implemented by allocating a temporary buffer of the element's size. This is problematic, though, as the emulated `qsort` interface doesn't provide any return values and thus cannot signal an error if allocation of that temporary buffer has failed. Convert the function to swap via a temporary buffer allocated on the stack. Like this, it can `memcpy` contents of both elements in small batches without requiring a heap allocation. The buffer size has been chosen such that in most cases, a single iteration of copying will suffice. Most importantly, it can fully contain `git_oid` structures and pointers. Add a bunch of tests for the `git__qsort_r` interface to verify nothing breaks. Furthermore, this removes the declaration of `git__insertsort_r` and makes it static as it is not used anywhere else.

26b71d60

2020-02-07T14:36:10

tests: iterator: fix iterator expecting too few items The testcase iterator::workdir::filesystem_gunk sets up quite a lot of directories, which is why it only runs in case GITTEST_INVASIVE_SPEED is set in the environment. Because we do not run our default CI with this variable, we didn't notice commit 852c83ee4 (refs: refuse to delete HEAD, 2020-01-15) breaking the test as it introduced a new reference to the "testrepo" repository. Fix the oversight by increasing the number of expected iterator items.

46228d86

2020-02-06T11:10:27

transports: http: fix custom headers not being applied In commit b9c5b15a7 (http: use the new httpclient, 2019-12-22), the HTTP code got refactored to extract a generic HTTP client that operates independently of the Git protocol. Part of refactoring was the creation of a new `git_http_request` struct that encapsulates the generation of requests. Our Git-specific HTTP transport was converted to use that in `generate_request`, but during the process we forgot to set up custom headers for the `git_http_request` and as a result we do not send out these headers anymore. Fix the issue by correctly setting up the request's custom headers and add a test to verify we correctly send them.

93a9044f

2020-01-31T08:49:34

fetchhead: strip credentials from remote URL If fetching from an anonymous remote via its URL, then the URL gets written into the FETCH_HEAD reference. This is mainly done to give valuable context to some commands, like for example git-merge(1), which will put the URL into the generated MERGE_MSG. As a result, what gets written into FETCH_HEAD may become public in some cases. This is especially important considering that URLs may contain credentials, e.g. when cloning 'https://foo:bar@example.com/repo' we persist the complete URL into FETCH_HEAD and put it without any kind of sanitization into the MERGE_MSG. This is obviously bad, as your login data has now just leaked as soon as you do git-push(1). When writing the URL into FETCH_HEAD, upstream git does strip credentials first. Let's do the same by trying to parse the remote URL as a "real" URL, removing any credentials and then re-formatting the URL. In case this fails, e.g. when it's a file path or not a valid URL, we just fall back to using the URL as-is without any sanitization. Add tests to verify our behaviour.

3f54ba8b

2020-01-18T13:51:40

credential: change git_cred to git_credential We avoid abbreviations where possible; rename git_cred to git_credential. In addition, we have standardized on a trailing `_t` for enum types, instead of using "type" in the name. So `git_credtype_t` has become `git_credential_t` and its members have become `GIT_CREDENTIAL` instead of `GIT_CREDTYPE`. Finally, the source and header files have been renamed to `credential` instead of `cred`. Keep previous name and values as deprecated, and include the new header files from the previous ones.

7fd9b3f5

2020-01-01T20:48:15

ci: add NTLM tests Download poxygit, a debugging git server, and clone from it using NTLM, both IIS-style (with connection affinity) and Apache-style ("broken", requiring constant reauthentication).

bf55facf

2019-10-25T12:24:34

tests: allow users to use expect/continue

dcd3b815

2019-12-13T15:28:57

tests: support CLAR_TRACE_LEVEL The CLAR_TRACE_LEVEL environment variable was supported when building with GIT_TRACE. Now we always build with GIT_TRACE, but that variable is not provided to tests. Simply support clar tracing always.

fe4efe2e

2019-12-03T10:17:30

tests: test that clone returns 4321 This conditional was backwards. We should instead test that clone returns 4321, not that 4321 returns clone.

e995f74e

2019-12-06T15:39:08

net: introduce git_net_url_joinpath Provide a mechanism to add a path and query string to an existing url so that we can easily append `/info/refs?...` type url segments to a url given to us by a user.

471daeea

2019-12-01T14:00:49

net: refactor gitno redirect handling Move the redirect handling into `git_net_url` for consistency.

e5fb5fe5

2019-10-20T17:19:01

ci: perform SPNEGO tests Attempt to obtain a Kerberos ticket from LIBGIT2.ORG and then clone the Negotiate-protected site at test.libgit2.org with that ticket.

94beb3a3

2020-01-18T14:03:23

merge: update enum type name for consistency libgit2 does not use `type_t` suffixes as it's redundant; thus, rename `git_merge_diff_type_t` to `git_merge_diff_t` for consistency.

1908884d

2020-01-17T08:34:30

Merge pull request #5361 from csware/no-return-freed_object Do not return free'd git_repository object on error

470a05d0

2020-01-16T17:53:50

Do not return free'd git_repository object on error Regression introduced in commit dde6d9c706bf1ecab545da55ab874a016587af1f. This issue causes lots of crashes in TortoiseGit. Signed-off-by: Sven Strickroth <email@cs-ware.de>

852c83ee

2020-01-15T13:31:21

refs: refuse to delete HEAD This requires adding a new symbolic ref to the testrepo fixture. Some of the existing tests attempt to delete HEAD, expecting a different failure. Introduce and use a non-HEAD symbolic ref instead. Adjust a few other tests as needed. Fixes #5357

7d55bee6

2020-01-10T12:44:51

win32: fix relative symlinks pointing into dirs On Windows platforms, we need some logic to emulate symlink(3P) defined by POSIX. As unprivileged symlinks on Windows are a rather new feature, our current implementation is comparatively new and still has some rough edges in special cases. One such case is relative symlinks. While relative symlinks to files in the same directory work as expected, libgit2 currently fails to create reltaive symlinks pointing into other directories. This is due to the fact that we forgot to translate the Unix-style target path to Windows-style. Most importantly, we are currently not converting directory separators from "/" to "\". Fix the issue by calling `git_win32_path_canonicalize` on the target. Add a test that verifies our ability to create such relative links across directories.

7142964f

2019-12-13T10:56:19

netops: handle intact query parameters in service_suffix removal Some servers leave the query parameters intact in the Location header when responding with a redirect. The service_suffix removal check as written assumed that the server removed them. Handle both cases. Along with PR #5325, this fixes #5321. There are two new tests. The first already passed; the second previously failed.

2dc7b5ef

2019-12-14T12:53:04

tests: pack: add missing asserts around `git_packbuilder_write`

11e8ee1f

2020-01-06T15:41:18

tests: submodule: verify setup of relative URLs When setting up relative URLs for a submodule, then we resolve it to the actual location and write that into ".git/config" instead of writing the relative value. We do not yet have a test to nail down this behaviour, which is now being added by this commit.

2f6f10bb

2019-12-13T13:35:40

Merge pull request #5300 from tiennou/fix/branch-documentation branch: clarify documentation around branches

97b8491b

2019-12-08T15:25:52

refs: rename git_reference__set_name to git_reference__realloc As git_reference__name will reallocate storage to account for longer names (it's actually allocator-dependent), it will cause all existing pointers to the old object to become dangling, as they now point to freed memory. Fix the issue by renaming to a more descriptive name, and pass a pointer to the actual reference that can safely be invalidated if the realloc succeeds.

b3178587

2019-12-13T08:35:25

Merge pull request #5333 from lrm29/attr_binary_macro attr: Update definition of binary macro

cf286d5e

2019-12-12T10:58:56

attr: Update definition of binary macro

14ff3516

2019-12-03T23:15:47

path: support non-ascii drive letters on dos Windows/DOS only supports drive letters that are alpha characters A-Z. However, you can `subst` any one-character as a drive letter, including numbers or even emoji. Test that we can identify emoji as drive letters.

85d4ff77

2019-12-03T19:50:18

index: ensure that we respect core.protectNTFS=false Users may want to turn off core.protectNTFS, perhaps to import (and then repair) a broken tree. Ensure that core.protectNTFS=false is honored.

ba4c769b

2019-12-03T23:23:02

tree: ensure we protect NTFS paths everywhere

e4034dfa

2019-12-03T19:24:59

path: protect NTFS everywhere Enable core.protectNTFS by default everywhere and in every codepath, not just on checkout.

d9c0c9cf

2019-12-03T19:17:41

test: ensure we can't add a protected path Test that when we enable core.protectNTFS that we cannot add platform-specific invalid paths to the index.

72df1cd8

2019-12-03T19:01:00

test: improve badname verification test The name of the `add_invalid_filename` function suggests that we _want_ to add an invalid filename. Rename the function to show that we expect to _fail_ to add the invalid filename.

f3b28604

2019-12-03T18:57:16

test: ensure treebuilder validate new protection rules Ensure that the new protection around .git::$INDEX_ALLOCATION rules are enabled for using the treebuilder when core.protectNTFS is set.

336991db

2019-12-03T18:56:31

test: ensure index adds validate new protection rules Ensure that the new protection around .git::$INDEX_ALLOCATION rules are enabled for adding to the index when core.protectNTFS is set.

a3cbd204

2019-12-03T18:49:23

test: improve badname verification test The name of the `write_invalid_filename` function suggests that we _want_ to write an invalid filename. Rename the function to show that we expect to _fail_ to write the invalid filename.

e1832eb2

2019-09-18T16:33:18

path: also guard `.gitmodules` against NTFS Alternate Data Streams We just safe-guarded `.git` against NTFS Alternate Data Stream-related attack vectors, and now it is time to do the same for `.gitmodules`. Note: In the added regression test, we refrain from verifying all kinds of variations between short names and NTFS Alternate Data Streams: as the new code disallows _all_ Alternate Data Streams of `.gitmodules`, it is enough to test one in order to know that all of them are guarded against. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

3f7851ea

2019-09-18T14:32:05

Disallow NTFS Alternate Data Stream attacks, even on Linux/macOS A little-known feature of NTFS is that it offers to store metadata in so-called "Alternate Data Streams" (inspired by Apple's "resource forks") that are copied together with the file they are associated with. These Alternate Data Streams can be accessed via `<file name>:<stream name>:<stream type>`. Directories, too, have Alternate Data Streams, and they even have a default stream type `$INDEX_ALLOCATION`. Which means that `abc/` and `abc::$INDEX_ALLOCATION/` are actually equivalent. This is of course another attack vector on the Git directory that we definitely want to prevent. On Windows, we already do this incidentally, by disallowing colons in file/directory names. While it looks as if files'/directories' Alternate Data Streams are not accessible in the Windows Subsystem for Linux, and neither via CIFS/SMB-mounted network shares in Linux, it _is_ possible to access them on SMB-mounted network shares on macOS. Therefore, let's go the extra mile and prevent this particular attack _everywhere_. To keep things simple, let's just disallow *any* Alternate Data Stream of `.git`. This is libgit2's variant of CVE-2019-1352. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

64c612cc

2019-09-18T15:25:02

Protect against 8.3 "short name" attacks also on Linux/macOS The Windows Subsystem for Linux (WSL) is getting increasingly popular, in particular because it makes it _so_ easy to run Linux software on Windows' files, via the auto-mounted Windows drives (`C:\` is mapped to `/mnt/c/`, no need to set that up manually). Unfortunately, files/directories on the Windows drives can be accessed via their _short names_, if that feature is enabled (which it is on the `C:` drive by default). Which means that we have to safeguard even our Linux users against the short name attacks. Further, while the default options of CIFS/SMB-mounts seem to disallow accessing files on network shares via their short names on Linux/macOS, it _is_ possible to do so with the right options. So let's just safe-guard against short name attacks _everywhere_. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

d29d4de2

2019-09-18T15:08:56

cl_git_fail: do not report bogus error message When we expect a checkout operation to fail, but it succeeds, we actually do not want to see the error messages that were generated in the meantime for errors that were handled gracefully by the code (e.g. when an object could not be found in a pack: in this case, the next backend would have been given a chance to look up the object, and probably would have found it because the checkout succeeded, after all). Which means that in the specific case of `cl_git_fail()`, we actually want to clear the global error state _after_ evaluating the command: we know that any still-available error would be bogus, seeing as the command succeeded (unexpectedly). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

7f6fdb82

2019-12-01T14:11:22

Merge pull request #5312 from pks-t/pks/patch-base85-overflow patch_parse: fix out-of-bounds reads caused by integer underflow

7f20778b

2019-11-29T09:14:04

Merge pull request #5311 from pks-t/pks/clar-trace-warning tests: fix compiler warning if tracing is disabled

361ebbcb

2019-11-28T15:36:40

tests: config: only test parsing huge file with GITTEST_INVASIVE_SPEED The test in config::stress::huge_section_with_many_values takes quite a long time to execute. Hide it behind the GITTEST_INVASIVE_SPEED environment varibale to not needlessly blow up execution time of tests. As this environment variable is being set by the continuous integration, we will execute it regularly anyway.

33e6c402

2019-11-28T15:26:36

patch_parse: fix out-of-bounds reads caused by integer underflow The patch format for binary files is a simple Base85 encoding with a length byte as prefix that encodes the current line's length. For each line, we thus check whether the line's actual length matches its expected length in order to not faultily apply a truncated patch. This also acts as a check to verify that we're not reading outside of the line's string: if (encoded_len > ctx->parse_ctx.line_len - 1) { error = git_parse_err(...); goto done; } There is the possibility for an integer underflow, though. Given a line with a single prefix byte, only, `line_len` will be zero when reaching this check. As a result, subtracting one from that will result in an integer underflow, causing us to assume that there's a wealth of bytes available later on. Naturally, this may result in an out-of-bounds read. Fix the issue by checking both `encoded_len` and `line_len` for a non-zero value. The binary format doesn't make use of zero-length lines anyway, so we need to know that there are both encoded bytes and remaining characters available at all. This patch also adds a test that works based on the last error message. Checking error messages is usually too tightly coupled, but in fact parsing the patch failed even before the change. Thus the only possibility is to use e.g. Valgrind, but that'd result in us not catching issues when run without Valgrind. As a result, using the error message is considered a viable tradeoff as we know that we didn't start decoding Base85 in the first place.

1d470a71

2019-11-28T14:45:15

tests: fix compiler warning if tracing is disabled If building libgit2's test suite with tracing disabled, then the compiler will emit a warning due to the unused `message_prefix` function. Fix the issue by wrapping the whole file into ifdef's for `GIT_TRACE` and providing separate empty function implementations for both `cl_global_trace_register` and `cl_global_trace_disable`.

fb439c97

2019-11-28T14:41:58

Merge pull request #5306 from herrerog/patchid diff: complete support for git patchid

61176a9b

2019-11-28T14:31:16

Merge pull request #5243 from pks-t/pks/config-optimize-mem Memory optimizations for config entries

ece5bb5e

2019-11-07T14:10:00

diff: make patchid computation work with all types of commits. Current implementation of patchid is not computing a correct patchid when given a patch where, for example, a new file is added or removed. Some more corner cases need to be handled to have same behavior as git patch-id command. Add some more tests to cover those corner cases. Signed-off-by: Gregory Herrero <gregory.herrero@oracle.com>

b63ad958

2019-11-28T13:19:50

Merge pull request #5309 from libgit2/ethomson/trace Improve trace support in tests

0e5243b7

2019-11-28T12:42:36

Merge pull request #5123 from libgit2/ethomson/off_t Move `git_off_t` to `git_object_size_t`

b7f70bc2

2019-11-27T12:36:17

tests: optionally show test execution tracing Only show test trace execution when the CLAR_TRACE_TESTS environment variable is set. This reduces the noise during tracing.

85efe896

2019-11-27T12:34:10

tests: display trace level with prefix in tests

6460e8ab

2019-06-23T18:13:29

internal: use off64_t instead of git_off_t Prefer `off64_t` internally.

0005c77a

2019-11-24T15:49:49

test: add an azure repos test We currently talk to Azure Repos for executing an online test (online::clone::path_whitespace). Add a simpler test to talk to Azure Repos to make it obvious that strange test failures are not likely the whitespace in the path, but actually a function of talking to Azure Repos itself.

048e94ad

2019-11-07T14:13:14

patch_parse: correct parsing of patch containing not shown binary data. When not shown binary data is added or removed in a patch, patch parser is currently returning 'error -1 - corrupt git binary header at line 4'. Fix it by correctly handling case where binary data is added/removed. Signed-off-by: Gregory Herrero <gregory.herrero@oracle.com>

de7659cc

2019-11-10T18:44:56

patch_parse: use paths from "---"/"+++" lines for binary patches For some patches, it is not possible to derive the old and new file paths from the patch header's first line, most importantly when they contain spaces. In such a case, we derive both paths from the "---" and "+++" lines, which allow for non-ambiguous parsing. We fail to use these paths when parsing binary patches without data, though, as we always expect the header paths to be filled in. Fix this by using the "---"/"+++" paths by default and only fall back to header paths if they aren't set. If neither of those paths are set, we just return an error. Add two tests to verify this behaviour, one of which would have previously caused a segfault.

01ea911b

2019-11-06T06:04:55

Merge pull request #5299 from pks-t/pks/config-mem-snapshots config_mem: implement support for snapshots

146e5bf7

2019-11-06T07:27:35

config_mem: implement support for snapshots Similar as in commit dadbb33b6 (Fix crash if snapshotting a config_snapshot, 2019-11-01), let's implement snapshots for in-memory configuration entries. As this deletes more code than it adds, it doesn't make any sense to not allow for this and allows users to treat config backends mostly the same.

de543e29

2019-11-05T22:44:27

patch_parse: fix segfault when header path contains whitespace only When parsing header paths from a patch, we reject any patches with empty paths as malformed patches. We perform the check whether a path is empty before sanitizing it, though, which may lead to a path becoming empty after the check, e.g. if we have trimmed whitespace. This may lead to a segfault later when any part of our patching logic actually references such a path, which may then be a `NULL` pointer. Fix the issue by performing the check after sanitizing. Add tests to catch the issue as they would have produced a segfault previosuly.

b7dcea04

2019-09-26T15:06:12

config_entries: micro-optimize storage of multivars Multivars are configuration entries that have many values for the same name; we can thus micro-optimize this case by just retaining the name of the first configuration entry and freeing all the others, letting them point to the string of the first entry. The attached test case is an extreme example that demonstrates this. It contains a section name that is approximately 500kB in size with 20.000 entries "a=b". Without the optimization, this would require at least 20000*500kB bytes, which is around 10GB. With this patch, it only requires 500kB+20000*1B=20500kB. The obvious culprit here is the section header, which we repeatedly include in each of the configuration entry's names. This makes it very easier for an adversary to provide a small configuration file that disproportionally blows up in memory during processing and is thus a feasible way for a denial-of-service attack. Unfortunately, we cannot fix the root cause by e.g. having a separate "section" field that may easily be deduplicated due to the `git_config_entry` structure being part of our public API. So this micro-optimization is the best we can do for now.

82d7a114

2019-11-05T11:18:14

Merge pull request #5293 from csware/config_snapshot-snapshot Fix crash if snapshotting a config_snapshot

bf2911d7

2019-11-02T07:30:32

Merge pull request #5275 from pks-t/pks/reflogs-with-newlines reflogs: fix behaviour around reflogs with newlines

dadbb33b

2019-11-01T18:55:54

Fix crash if snapshotting a config_snapshot Signed-off-by: Sven Strickroth <email@cs-ware.de>

0974e02f

2019-10-30T20:35:48

commit: add failing tests for object checking for git_commit_with_signature There can be a significant difference between the system where we created the buffer (if at all) and when the caller provides us with the contents of a commit. Provide some test cases (we have to adapt the existing ones because they refer to trees and commits which do not exist).

2a7d6de3

2019-10-29T07:52:31

Merge pull request #5276 from pks-t/pks/patch-fuzzing-fixes patch_parse: fixes for fuzzing errors

a31f4c4b

2019-10-24T13:16:03

Merge pull request #5227 from ddevault/check apply: add GIT_APPLY_CHECK

c405f231

2019-10-24T10:26:43

Merge pull request #5264 from henkesn/refs-unlock-on-commit refs: unlock unmodified refs on transaction commit

02af1fcb

2019-09-14T14:03:36

apply: add GIT_APPLY_CHECK This adds an option which will check if a diff is applicable without actually applying it; equivalent to git apply --check.

37141ff7

2019-10-21T18:56:59

patch_parse: detect overflow when calculating old/new line position When the patch contains lines close to INT_MAX, then it may happen that we end up with an integer overflow when calculating the line of the current diff hunk. Reject such patches as unreasonable to avoid the integer overflow. As the calculation is performed on integers, we introduce two new helpers `git__add_int_overflow` and `git__sub_int_overflow` that perform the integer overflow check in a generic way.

468e3ddc

2019-10-19T16:48:11

patch_parse: fix out-of-bounds read with No-NL lines We've got two locations where we copy lines into the patch. The first one is when copying normal " ", "-" or "+" lines, while the second location gets executed when we copy "\ No newline at end of file" lines. While the first one correctly uses `git__strndup` to copy only until the newline, the other one doesn't. Thus, if the line occurs at the end of the patch and if there is no terminating NUL character, then it may result in an out-of-bounds read. Fix the issue by using `git__strndup`, as was already done in the other location. Furthermore, add allocation checks to both locations to detect out-of-memory situations.

6c6c15e9

2019-10-19T15:52:35

patch_parse: reject empty path names When parsing patch headers, we currently accept empty path names just fine, e.g. a line "--- \n" would be parsed as the empty filename. This is not a valid patch format and may cause `NULL` pointer accesses at a later place as `git_buf_detach` will return `NULL` in that case. Reject such patches as malformed with a nice error message.

223e7e43

2019-10-19T15:42:54

patch_parse: reject patches with multiple old/new paths It's currently possible to have patches with multiple old path name headers. As we didn't check for this case, this resulted in a memory leak when overwriting the old old path with the new old path because we simply discarded the old pointer. Instead of fixing this by free'ing the old pointer, we should reject such patches altogether. It doesn't make any sense for the "---" or "+++" markers to occur multiple times within a patch n the first place. This also implicitly fixes the memory leak.

7968e90f

2019-10-18T12:33:07

refdb_fs: properly parse corrupted reflogs In previous versions, libgit2 could be coerced into writing reflog messages with embedded newlines into the reflog by using `git_stash_save` with a message containing newlines. While the root cause is fixed now, it was noticed that upstream git is in fact able to read such corrupted reflog messages just fine. Make the reflog parser more lenient in order to just skip over malformatted reflog lines to bring us in line with git. This requires us to change an existing test that verified that we do indeed _fail_ to parse such logs.

d8233feb

2019-10-18T09:24:14

reflog: allow adding entries with newlines in their message Currently, the reflog disallows any entries that have a message with newlines, as that would effectively break the reflog format, which may contain a single line per entry, only. Upstream git behaves a bit differently, though, especially when considering stashes: instead of rejecting any reflog entry with newlines, git will simply replace newlines with spaces. E.g. executing 'git stash push -m "foo\nbar"' will create a reflog entry with "foo bar" as entry message. This commit adjusts our own logic to stop rejecting commit messages with newlines. Previously, this logic was part of `git_reflog_append`, only. There is a second place though where we add reflog entries, which is the serialization code in the filesystem refdb. As it didn't contain any sanity checks whatsoever, the refdb would have been perfectly happy to write malformatted reflog entries to the disk. This is being fixed with the same logic as for the reflog itself.

c9464bf7

2019-10-17T18:33:12

Merge pull request #5273 from dlax/parse-diff-without-extended-headers patch_parse: handle patches without extended headers

73e9535d

2019-10-17T13:43:26

tests: submodule: test cloning edge cases Add two more tests that verify our behaviour in some edge cases, notably when cloning into a non-empty directory and when cloning the same submodule twice.

de412fc2

2019-10-17T13:36:22

tests: submodule: make use of sandboxes to clean repos The test submodule::add::submodule_clone doesn't use a sandbox, and thus the created repo will not get deleted after the test has finished. Convert the test to use the empty standard repo sandbox instead to fix this.

09b1ac11

2019-10-17T13:32:22

tests: submodule: fix tests for cloning submodules The test submodule::add::homemade_clone unfortunately doesn't test what's expected, but does instead clone the submodule to a directory that is outside of the parent repository. Fixing this by cloning to the correct location isn't possible, though, as `git_submodule_add_setup` will have pre-created a ".git" file already, which will cause `git_clone` to error out. As it's not possible to perform the clone without fiddling around with the repo's layout, let's just remove this test as that is in fact what the new `git_submodule_clone` function is for.

47531f47

2019-10-11T12:44:09

refs: unlock unmodified refs on transaction commit Refs which are locked in a transaction without an altered target, still should to be unlocked on `git_transaction_commit`. `git_transaction_free` also unlocks refs but the moment of calling of `git_transaction_free` cannot be controlled in all situations. Some binding libs call `git_transaction_free` on garbage collection or not at all if the application exits before and don't provide public access to `git_transaction_free`. It is better to release locks as soon as possible.

3c5d78bd

2019-05-01T16:16:26

submodule: provide a wrapper for simple submodule clone steps

11de594f

2019-10-16T22:11:33

patch_parse: handle patches without extended headers Extended header lines (especially the "index <hash>..<hash> <mode>") are not required by "git apply" so it import patches. So we allow the from-file/to-file lines (--- a/file\n+++ b/file) to directly follow the git diff header. This fixes #5267.

ef5a3851

2019-10-11T07:47:17

Merge pull request #5257 from henkesn/master Fix file locking on POSIX OS

1f9b4970

2019-10-11T07:44:10

Merge pull request #5260 from pks-t/pks/cmake3 cmake: update minimum CMake version to v3.5.1

3335a034

2019-10-10T15:28:46

refs: fix locks getting forcibly removed The flag GIT_FILEBUF_FORCE currently does two things: 1. It will cause the filebuf to create non-existing leading directories for the file that is about to be written. 2. It will forcibly remove any pre-existing locks. While most call sites actually do want (1), they do not want to remove pre-existing locks, as that renders the locking mechanisms effectively useless. Introduce a new flag `GIT_FILEBUF_CREATE_LEADING_DIRS` to separate both behaviours cleanly from each other and convert callers to use it instead of `GIT_FILEBUF_FORCE` to have them honor locked files correctly. As this conversion removes all current users of `GIT_FILEBUF_FORCE`, this commit removes the flag altogether.

6716e2f3

2019-10-10T12:34:45

Merge pull request #5248 from dlax/parse-patch-empty-files patch_parse: handle patches with new empty files

ebabb88f

2019-10-10T09:25:32

cmake: update minimum CMake version to v3.5.1 Back in commit cf9f34521 (cmake: bump minimum version to 2.8.11, 2017-09-06), we have bumped the minimum CMake version to require at least v2.8.11. The main hold-backs back then were distributions like RHEL/CentOS as well as Ubuntu Trusty, which caused us to not target a more modern version. Nowadays, Ubuntu Trusty has been EOL'd and CentOS 6 has CMake v3.6.1 available via the EPEL6 repository, and thus it seems fair to upgrade to a more recent version. Going through repology [1], one can see that all supported mainstream distributions do in fact have CMake 3 available. Going through the list, the minimum version that is supported by all mainstream distros is in fact v3.5.1: - CentOS 6 via EPEL6: 3.6.1 - Debian Oldstable: 3.7.2 - Fedora 26: 3.8.2 - OpenMandriva 3.x: 3.5.1 - Slackware 14.2: 3.5.2 - Ubuntu 16.04: 3.5.1 Consequentally, let's upgrade CMake to the minimum version of 3.5.1 and remove all the version CMake checks that aren't required anymore. [1]: https://repology.org/project/cmake/versions

63307cba

2019-09-28T17:32:18

Merge pull request #5226 from pks-t/pks/regexp-api regexp: implement a new regular expression API

b61810bf

2019-09-28T15:52:25

patch_parse: handle patches with new empty files Patches containing additions of empty files will not contain diff data but will end with the index header line followed by the terminating sequence "-- ". We follow the same logic as in cc4c44a and allow "-- " to immediately follow the index header.

f585b129

2019-09-12T14:29:28

posix: remove superseded POSIX regex wrappers The old POSIX regex wrappers have been superseded by our own regexp API that provides a higher-level abstraction. Remove the POSIX wrappers in favor of the new one.

d77378eb

2019-09-13T08:54:26

regexp: implement new regular expression API We currently support a set of different regular expression backends with PCRE, PCRE2, regcomp(3P) and regcomp_l(3). The current implementation of this is done via a simple POSIX wrapper that either directly uses supplied functions or that is a very small wrapper. To support PCRE and PCRE2, we use their provided <pcreposix.h> and <pcre2posix.h> wrappers. These wrappers are implemented in such a way that the accompanying libraries pcre-posix and pcre2-posix provide the same symbols as the libc ones, namely regcomp(3P) et al. This works out on some systems just fine, most importantly on glibc-based ones, where the regular expression functions are implemented as weak aliases and thus get overridden by linking in the pcre{,2}-posix library. On other systems we depend on the linking order of libc and pcre library, and as libc always comes first we will end up with the functions of the libc implementation. As a result, we may use the structures `regex_t` and `regmatch_t` declared by <pcre{,2}posix.h>, but use functions defined by the libc, leading to segfaults. The issue is not easily solvable. Somed distributions like Debian have resolved this by patching PCRE and PCRE2 to carry custom prefixes to all the POSIX function wrappers. But this is not supported by upstream and thus inherently unportable between distributions. We could instead try to modify linking order, but this starts becoming fragile and will not work e.g. when libgit2 is loaded via dlopen(3P) or similar ways. In the end, this means that we simply cannot use the POSIX wrappers provided by the PCRE libraries at all. Thus, this commit introduces a new regular expression API. The new API is on a tad higher level than the previous POSIX abstraction layer, as it tries to abstract away any non-portable flags like e.g. REG_EXTENDED, which has no equivalents in all of our supported backends. As there are no users of POSIX regular expressions that do _not_ reguest REG_EXTENDED this is fine to be abstracted away, though. Due to the API being higher-level than before, it should generally be a tad easier to use than the previous one. Note: ideally, the new API would've been called `git_regex_foobar` with a file "regex.h" and "regex.c". Unfortunately, this is currently impossible to implement due to naming clashes between the then-existing "regex.h" and <regex.h> provided by the libc. As we add the source directory of libgit2 to the header search path, an include of <regex.h> would always find our own "regex.h". Thus, we have to take the bitter pill of adding one more character to all the functions to disambiguate the includes. To improve guarantees around cross-backend compatibility, this commit also brings along an improved regular expression test suite core::regexp.

174b7a32

2019-09-19T12:24:06

buffer: fix printing into out-of-memory buffer Before printing into a `git_buf` structure, we always call `ENSURE_SIZE` first. This macro will reallocate the buffer as-needed depending on whether the current amount of allocated bytes is sufficient or not. If `asize` is big enough, then it will just do nothing, otherwise it will call out to `git_buf_try_grow`. But in fact, it is insufficient to only check `asize`. When we fail to allocate any more bytes e.g. via `git_buf_try_grow`, then we set the buffer's pointer to `git_buf__oom`. Note that we touch neither `asize` nor `size`. So if we just check `asize > targetsize`, then we will happily let the caller of `ENSURE_SIZE` proceed with an out-of-memory buffer. As a result, we will print all bytes into the out-of-memory buffer instead, resulting in an out-of-bounds write. Fix the issue by having `ENSURE_SIZE` verify that the buffer is not marked as OOM. Add a test to verify that we're not writing into the OOM buffer.

208f1d7a

2019-09-19T12:46:37

buffer: fix infinite loop when growing buffers When growing buffers, we repeatedly multiply the currently allocated number of bytes by 1.5 until it exceeds the requested number of bytes. This has two major problems: 1. If the current number of bytes is tiny and one wishes to resize to a comparatively huge number of bytes, then we may need to loop thousands of times. 2. If resizing to a value close to `SIZE_MAX` (which would fail anyway), then we probably hit an infinite loop as multiplying the current amount of bytes will repeatedly result in integer overflows. When reallocating buffers, one typically chooses values close to 1.5 to enable re-use of resulting memory holes in later reallocations. But because of this, it really only makes sense to use a factor of 1.5 _once_, but not looping until we finally are able to fit it. Thus, we can completely avoid the loop and just opt for the much simpler algorithm of multiplying with 1.5 once and, if the result doesn't fit, just use the target size. This avoids both problems of looping extensively and hitting overflows. This commit also adds a test that would've previously resulted in an infinite loop.

8bf0f7eb

2019-09-09T13:00:27

cred: separate public interface from low-level details

5d8a4659

2019-09-13T10:31:49

Merge pull request #5195 from tiennou/fix/commitish-smart-push smart: use push_glob instead of manual filtering

b545be3d

2019-09-10T11:14:36

open:fix memory leak when passing NULL to git_repository_open_ext

c3a7892f

2019-09-09T13:10:24

Merge pull request #5209 from mkostyuk/apply-wrong-patch apply: Fix a patch corruption related to EOFNL handling

17d6cd45

2019-09-09T13:06:22

Merge pull request #5210 from buddyspike/master ignore: correct handling of nested rules overriding wild card unignore

585fbd74

2019-08-28T23:18:31

apply: Test for EOFNL mishandling when several hunks are processed Introduce an unit test to validate that git_apply__patch() properly handles EOFNL changes in case of patches with several hunks.

a7b4b639

2019-08-24T12:14:31

ignore: correct handling of nested rules overriding wild card unignore problem: filesystem_iterator loads .gitignore files in top-down order. subsequently, ignore module evaluates them in the order they are loaded. this creates a problem if we have unignored a rule (using a wild card) in a sub dir and ignored it again in a level further below (see the test included in this patch). solution: process ignores in reverse order. closes #4963

5fc27aac

2019-08-27T13:38:08

Merge pull request #5208 from mkostyuk/apply-removed-new-file apply: git_apply_to_tree fails to apply patches that add new files

5498c318

2019-08-27T13:10:53

apply: free test data

6de48085

2019-08-27T11:29:24

Merge pull request #5189 from libgit2/ethomson/attrs_from_head Optionally read `.gitattributes` from HEAD

aaa48d06

2019-08-27T11:26:50

Merge pull request #5196 from pks-t/pks/config-include-onbranch config: implement "onbranch" conditional

8cbef12d

2019-08-08T11:52:54

util: do not perform allocations in insertsort Our hand-rolled fallback sorting function `git__insertsort_r` does an in-place sort of the given array. As elements may not necessarily be pointers, it needs a way of swapping two values of arbitrary size, which is currently implemented by allocating a temporary buffer of the element's size. This is problematic, though, as the emulated `qsort` interface doesn't provide any return values and thus cannot signal an error if allocation of that temporary buffer has failed. Convert the function to swap via a temporary buffer allocated on the stack. Like this, it can `memcpy` contents of both elements in small batches without requiring a heap allocation. The buffer size has been chosen such that in most cases, a single iteration of copying will suffice. Most importantly, it can fully contain `git_oid` structures and pointers. Add a bunch of tests for the `git__qsort_r` interface to verify nothing breaks. Furthermore, this removes the declaration of `git__insertsort_r` and makes it static as it is not used anywhere else.

thodg/libgit2/tests

tests

Log