src


Log

Author Commit Date CI Message
Patrick Steinhardt b3178587 2019-12-13T08:35:25 Merge pull request #5333 from lrm29/attr_binary_macro attr: Update definition of binary macro
Laurence McGlashan cf286d5e 2019-12-12T10:58:56 attr: Update definition of binary macro
Edward Thomson e4034dfa 2019-12-03T19:24:59 path: protect NTFS everywhere Enable core.protectNTFS by default everywhere and in every codepath, not just on checkout.
Edward Thomson 14ff3516 2019-12-03T23:15:47 path: support non-ascii drive letters on dos Windows/DOS only supports drive letters that are alpha characters A-Z. However, you can `subst` any one-character as a drive letter, including numbers or even emoji. Test that we can identify emoji as drive letters.
Johannes Schindelin e1832eb2 2019-09-18T16:33:18 path: also guard `.gitmodules` against NTFS Alternate Data Streams We just safe-guarded `.git` against NTFS Alternate Data Stream-related attack vectors, and now it is time to do the same for `.gitmodules`. Note: In the added regression test, we refrain from verifying all kinds of variations between short names and NTFS Alternate Data Streams: as the new code disallows _all_ Alternate Data Streams of `.gitmodules`, it is enough to test one in order to know that all of them are guarded against. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Johannes Schindelin 3f7851ea 2019-09-18T14:32:05 Disallow NTFS Alternate Data Stream attacks, even on Linux/macOS A little-known feature of NTFS is that it offers to store metadata in so-called "Alternate Data Streams" (inspired by Apple's "resource forks") that are copied together with the file they are associated with. These Alternate Data Streams can be accessed via `<file name>:<stream name>:<stream type>`. Directories, too, have Alternate Data Streams, and they even have a default stream type `$INDEX_ALLOCATION`. Which means that `abc/` and `abc::$INDEX_ALLOCATION/` are actually equivalent. This is of course another attack vector on the Git directory that we definitely want to prevent. On Windows, we already do this incidentally, by disallowing colons in file/directory names. While it looks as if files'/directories' Alternate Data Streams are not accessible in the Windows Subsystem for Linux, and neither via CIFS/SMB-mounted network shares in Linux, it _is_ possible to access them on SMB-mounted network shares on macOS. Therefore, let's go the extra mile and prevent this particular attack _everywhere_. To keep things simple, let's just disallow *any* Alternate Data Stream of `.git`. This is libgit2's variant of CVE-2019-1352. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Johannes Schindelin 64c612cc 2019-09-18T15:25:02 Protect against 8.3 "short name" attacks also on Linux/macOS The Windows Subsystem for Linux (WSL) is getting increasingly popular, in particular because it makes it _so_ easy to run Linux software on Windows' files, via the auto-mounted Windows drives (`C:\` is mapped to `/mnt/c/`, no need to set that up manually). Unfortunately, files/directories on the Windows drives can be accessed via their _short names_, if that feature is enabled (which it is on the `C:` drive by default). Which means that we have to safeguard even our Linux users against the short name attacks. Further, while the default options of CIFS/SMB-mounts seem to disallow accessing files on network shares via their short names on Linux/macOS, it _is_ possible to do so with the right options. So let's just safe-guard against short name attacks _everywhere_. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Edward Thomson b8464342 2019-12-03T17:47:31 path: rename function that detects end of filename The function `only_spaces_and_dots` used to detect the end of the filename on win32. Now we look at spaces and dots _before_ the end of the string _or_ a `:` character, which would signify a win32 alternate data stream. Thus, rename the function `ntfs_end_of_filename` to indicate that it detects the (virtual) end of a filename, that any further characters would be elided to the given path.
Sven Strickroth bdf96512 2019-12-03T21:17:30 MSVC: Fix warning C4133 on x64: "function": Incompatible types - from "unsigned long *" to "size_t *" Signed-off-by: Sven Strickroth <email@cs-ware.de>
Edward Thomson 36bfc27a 2019-12-01T14:38:54 Merge pull request #5314 from pks-t/pks/dll-main-removal global: convert to fiber-local storage to fix exit races
Patrick Steinhardt 5c6180b5 2019-11-29T11:06:11 global: convert to fiber-local storage to fix exit races On Windows platforms, we automatically clean up the thread-local storage upon detaching a thread via `DllMain()`. The thing is that this happens for every thread of applications that link against the libgit2 DLL, even those that don't have anything to do with libgit2 itself. As a result, we cannot assume that these unsuspecting threads make use of our `git_libgit2_init()` and `git_libgit2_shutdow()` reference counting, which may lead to racy situations: Thread 1 Thread 2 git_libgit2_shutdown() DllMain(DETACH_THREAD) git__free_tls_data() git_atomic_dec() == 0 git__free_tls_data() TlsFree(_tls_index) TlsGetValue(_tls_index) Due to the second thread never having executed `git_libgit2_init()`, the first thread will clean up TLS data and as a result also free the `_tls_index` variable. When detaching the second thread, we unconditionally access the now-free'd `_tls_index` variable, which is obviously not going to work out well. Fix the issue by converting the code to use fiber-local storage instead of thread-local storage. While FLS will behave the exact same as TLS if no fibers are in use, it does allow us to specify a destructor similar to the one that is accepted by pthread_key_create(3P). Like this, we do not have to manually free indices anymore, but will let the FLS handle calling the destructor. This allows us to get rid of `DllMain()` completely, as we only used it to keep track of when threads were exiting and results in an overall simplification of TLS cleanup.
Patrick Steinhardt 33e6c402 2019-11-28T15:26:36 patch_parse: fix out-of-bounds reads caused by integer underflow The patch format for binary files is a simple Base85 encoding with a length byte as prefix that encodes the current line's length. For each line, we thus check whether the line's actual length matches its expected length in order to not faultily apply a truncated patch. This also acts as a check to verify that we're not reading outside of the line's string: if (encoded_len > ctx->parse_ctx.line_len - 1) { error = git_parse_err(...); goto done; } There is the possibility for an integer underflow, though. Given a line with a single prefix byte, only, `line_len` will be zero when reaching this check. As a result, subtracting one from that will result in an integer underflow, causing us to assume that there's a wealth of bytes available later on. Naturally, this may result in an out-of-bounds read. Fix the issue by checking both `encoded_len` and `line_len` for a non-zero value. The binary format doesn't make use of zero-length lines anyway, so we need to know that there are both encoded bytes and remaining characters available at all. This patch also adds a test that works based on the last error message. Checking error messages is usually too tightly coupled, but in fact parsing the patch failed even before the change. Thus the only possibility is to use e.g. Valgrind, but that'd result in us not catching issues when run without Valgrind. As a result, using the error message is considered a viable tradeoff as we know that we didn't start decoding Base85 in the first place.
Patrick Steinhardt fb439c97 2019-11-28T14:41:58 Merge pull request #5306 from herrerog/patchid diff: complete support for git patchid
Patrick Steinhardt 61176a9b 2019-11-28T14:31:16 Merge pull request #5243 from pks-t/pks/config-optimize-mem Memory optimizations for config entries
Gregory Herrero ece5bb5e 2019-11-07T14:10:00 diff: make patchid computation work with all types of commits. Current implementation of patchid is not computing a correct patchid when given a patch where, for example, a new file is added or removed. Some more corner cases need to be handled to have same behavior as git patch-id command. Add some more tests to cover those corner cases. Signed-off-by: Gregory Herrero <gregory.herrero@oracle.com>
Patrick Steinhardt 0b5540b9 2019-11-28T13:56:54 Merge pull request #5307 from palmin/hash_sha256 ssh: include sha256 host key hash when supported
Patrick Steinhardt dfea0713 2019-11-28T13:51:40 Merge pull request #5272 from tiennou/examples/cli-ification Various examples shape-ups
Patrick Steinhardt 0e5243b7 2019-11-28T12:42:36 Merge pull request #5123 from libgit2/ethomson/off_t Move `git_off_t` to `git_object_size_t`
Edward Thomson 6460e8ab 2019-06-23T18:13:29 internal: use off64_t instead of git_off_t Prefer `off64_t` internally.
Edward Thomson 05237ee5 2019-06-23T17:20:17 integer: use int64_t's for checks Use int64_t internally for type visibility.
Edward Thomson ee0c8618 2019-06-23T17:19:31 offmap: store off64_t's instead of git_off_t's Prefer `off64_t` to `git_off_t` internally for visibility.
Edward Thomson 8be12026 2019-06-23T17:09:22 mmap: use a 64-bit signed type `off64_t` for mmap Prefer `off64_t` to `git_off_t` for internal visibility.
Edward Thomson 7e1cc296 2019-11-25T13:17:42 mmap: remove unnecessary assertion 64 bit types are always 64 bit.
Edward Thomson cb77423f 2019-11-24T16:22:31 valgrind: add valgrind hints in OpenSSL Provide usage hints to valgrind. We trust the data coming back from OpenSSL to have been properly initialized. (And if it has not, it's an OpenSSL bug, not a libgit2 bug.) We previously took the `VALGRIND` option to CMake as a hint to disable mmap. Remove that; it's broken. Now use it to pass on the `VALGRIND` definition so that sources can provide valgrind hints.
Edward Thomson 2ad3eb3e 2019-11-24T15:59:26 valgrind: add suppressions for undefined use valgrind will warn that OpenSSL will use undefined data in connect/read when talking to certain other TLS stacks. Thankfully, this only seems to occur when gcc is the compiler, so hopefully valgrind is just misunderstanding an optimization. Regardless, suppress this warning.
Edward Thomson 4dffa295 2019-06-23T18:09:00 blame: use a size_t for the buffer
Edward Thomson 6c13cf6d 2019-11-22T15:18:54 filestamp: use `uint64_t` for object size Instead of using a signed type (`off_t`) use an unsigned `uint64_t` for the size of the files.
Edward Thomson fefefd1d 2019-06-23T16:42:14 odb: use `git_object_size_t` for object size Instead of using a signed type (`off_t`) use a new `git_object_size_t` for the sizes of objects.
Edward Thomson fb2198db 2019-06-23T16:23:59 futils_filesize: use `uint64_t` for object size Instead of using a signed type (`off_t`) use `uint64_t` for the maximum size of files.
Edward Thomson 4334b177 2019-06-23T15:43:38 blob: use `git_object_size_t` for object size Instead of using a signed type (`off_t`) use a new `git_object_size_t` for the sizes of objects.
Edward Thomson bed9fc6b 2019-06-23T15:16:47 odb: use `git_object_size_t` for object size Instead of using a signed type (`off_t`) use a new `git_object_size_t` for the sizes of objects.
Anders Borum 48c3f7e1 2019-11-20T11:21:14 ssh: include sha256 host key hash when supported
Gregory Herrero 048e94ad 2019-11-07T14:13:14 patch_parse: correct parsing of patch containing not shown binary data. When not shown binary data is added or removed in a patch, patch parser is currently returning 'error -1 - corrupt git binary header at line 4'. Fix it by correctly handling case where binary data is added/removed. Signed-off-by: Gregory Herrero <gregory.herrero@oracle.com>
Gregory Herrero b921964b 2019-11-07T13:08:51 diff_print: add support for GIT_DIFF_FORMAT_PATCH_ID. Git is generating patch-id using a stripped down version of a patch where hunk header and index information are not present. Signed-off-by: Gregory Herrero <gregory.herrero@oracle.com>
Gregory Herrero accd7848 2019-11-07T13:02:38 diff_print: add a new 'print_index' flag when printing diff. Add a new 'print_index' flag to let the caller decide whether or not 'index <oid>..<oid>' should be printed. Since patch id needs not to have index when hashing a patch, it will be useful soon. Signed-off-by: Gregory Herrero <gregory.herrero@oracle.com>
Edward Thomson 47dd665a 2019-11-16T15:21:56 Merge pull request #5303 from pks-t/pks/patch-path-in-body-only patch_parse: use paths from "---"/"+++" lines for binary patches
Edward Thomson cb6bc6f2 2019-11-16T15:17:54 Merge pull request #5285 from pcpthm/winhttp-308 Follow 308 redirect in WinHTTP transport
Edward Thomson 541b8fc5 2019-11-16T15:12:52 Merge pull request #5302 from tiennou/fix/p_lstat-errno fileops: correct error return on p_lstat failures when mkdir
Patrick Steinhardt de7659cc 2019-11-10T18:44:56 patch_parse: use paths from "---"/"+++" lines for binary patches For some patches, it is not possible to derive the old and new file paths from the patch header's first line, most importantly when they contain spaces. In such a case, we derive both paths from the "---" and "+++" lines, which allow for non-ambiguous parsing. We fail to use these paths when parsing binary patches without data, though, as we always expect the header paths to be filled in. Fix this by using the "---"/"+++" paths by default and only fall back to header paths if they aren't set. If neither of those paths are set, we just return an error. Add two tests to verify this behaviour, one of which would have previously caused a segfault.
Etienne Samson 0c2b0206 2019-11-09T09:41:13 fileops: correct error return on p_lstat failures when mkdir IIRC I got a strange return once from lstat, which translated in a weird error class/message being reported. As a safety measure, enforce a -1 return in that case.
Edward Thomson 01ea911b 2019-11-06T06:04:55 Merge pull request #5299 from pks-t/pks/config-mem-snapshots config_mem: implement support for snapshots
Etienne Samson cd5e33fb 2019-11-06T11:08:23 global: DRY includes of assert.h
Patrick Steinhardt 146e5bf7 2019-11-06T07:27:35 config_mem: implement support for snapshots Similar as in commit dadbb33b6 (Fix crash if snapshotting a config_snapshot, 2019-11-01), let's implement snapshots for in-memory configuration entries. As this deletes more code than it adds, it doesn't make any sense to not allow for this and allows users to treat config backends mostly the same.
Patrick Steinhardt de543e29 2019-11-05T22:44:27 patch_parse: fix segfault when header path contains whitespace only When parsing header paths from a patch, we reject any patches with empty paths as malformed patches. We perform the check whether a path is empty before sanitizing it, though, which may lead to a path becoming empty after the check, e.g. if we have trimmed whitespace. This may lead to a segfault later when any part of our patching logic actually references such a path, which may then be a `NULL` pointer. Fix the issue by performing the check after sanitizing. Add tests to catch the issue as they would have produced a segfault previosuly.
Patrick Steinhardt b7dcea04 2019-09-26T15:06:12 config_entries: micro-optimize storage of multivars Multivars are configuration entries that have many values for the same name; we can thus micro-optimize this case by just retaining the name of the first configuration entry and freeing all the others, letting them point to the string of the first entry. The attached test case is an extreme example that demonstrates this. It contains a section name that is approximately 500kB in size with 20.000 entries "a=b". Without the optimization, this would require at least 20000*500kB bytes, which is around 10GB. With this patch, it only requires 500kB+20000*1B=20500kB. The obvious culprit here is the section header, which we repeatedly include in each of the configuration entry's names. This makes it very easier for an adversary to provide a small configuration file that disproportionally blows up in memory during processing and is thus a feasible way for a denial-of-service attack. Unfortunately, we cannot fix the root cause by e.g. having a separate "section" field that may easily be deduplicated due to the `git_config_entry` structure being part of our public API. So this micro-optimization is the best we can do for now.
Patrick Steinhardt 62320860 2019-09-26T14:43:19 config_entries: only keep track of a single entry list Whenever adding a configuration entry to the config entries structure, we allocate two list heads: - The first list head is added to the global list of config entries in order to be able to iterate over configuration entries in the order they were originally added. - The second list head is added to the map of entries in order to efficiently look up an entry by its name. If no entry with the same name exists in the map, then we add the new entry to the map directly. Otherwise, we append the new entry's list head to the pre-existing entry's list in order to keep track of multivars. While the former usecase is perfectly sound, the second usecase can be optimized. The only reason why we keep track of multivar entries in another separate list is to be able to determine whether an entry is unique or not by seeing whether its `next` pointer is set. So we keep track of a complete list of multivar entries just to have a single bit of information of whether it has other multivar entries with the same entry name. We can completely get rid of this secondary list by just adding a `first` field to the list structure itself. When executing `git_config_entries_append`, we will then simply check whether the configuration map already has an entry with the same name -- if so, we will set the `first` to zero to indicate that it is not the initial entry anymore. Instead of a second list head in the map, we can thus now directly store the list head of the first global list inside of the map and just refer to that bit. Note that the more obvious solution would be to store a `unique` field instead of a `first` field. But as we will only ever inspect the `first` field of the _last_ entry that has been moved into the map, these are semantically equivalent in that case. Having a `first` field also allows for a minor optimization: for multivar values, we can free the `name` field of all entries that are _not_ first and have them point to the name of the first entry instead.
Patrick Steinhardt 8a88701e 2019-09-26T13:37:18 config_entries: mark local functions as static Some functions which are only used in "config_entries.c" are not marked as static, which is being fixed by this very commit.
Patrick Steinhardt 56b203a5 2019-10-24T12:20:27 config_file: keep reference to config entries when creating iterator When creating a configuration file iterator, then we first refresh the backend and then afterwards duplicate all refreshed configuration entries into the iterator in order to avoid seeing any concurrent modifications of the entries while iterating. The duplication of entries is not guarded, though, as we do not increase the refcount of the entries that we duplicate right now. This opens us up for a race, as another thread may concurrently refresh the repository configuration and thus swap out the current set of entries. As we didn't increase the refcount, this may lead to the entries being free'd while we iterate over them in the first thread. Fix the issue by properly handling the lifecycle of the backend's entries via `config_file_entries_take` and `git_config_entries_free`, respectively.
Patrick Steinhardt 0927156a 2019-10-24T12:32:11 config_file: refactor taking entries ref to return an error code The function to take a reference to the config file's config entries currently returns the reference via return value. Due to this, it's harder than necessary to integrate into our typical coding style, as one needs to make sure that a proper error code is set before erroring out from the caller. This bites us in `config_file_delete`, where we call `goto out` directly when `config_file_entries_take` returns `NULL`, but we actually forget to set up the error code and thus return success. Fix the issue by refactoring the function to return an error code and pass the reference via an out-pointer.
Patrick Steinhardt db301087 2019-10-24T12:17:02 config_file: remove unused includes
Patrick Steinhardt c2749849 2019-10-24T12:00:11 config_file: rename function names As with the predecessing commit, this commit renames backend functions of the configuration file backend. This helps to clearly separate functionality and also to be able to see from backtraces which backend is currently in use.
Patrick Steinhardt b30b04a9 2019-11-05T12:34:14 config_snapshot: rename function names The configuration snapshot backend has been extracted from the old files backend back in 2bff84ba4 (config_file: separate out read-only backend, 2019-07-26). To keep code churn manageable, the local functions weren't renamed yet and thus still have references to the old diskfile backend. Rename them accordingly to make them easier to understand.
Patrick Steinhardt 82d7a114 2019-11-05T11:18:14 Merge pull request #5293 from csware/config_snapshot-snapshot Fix crash if snapshotting a config_snapshot
romkatv 1886478d 2019-11-05T07:45:11 fix a bug introduced in 8a23597b
Edward Thomson bf2911d7 2019-11-02T07:30:32 Merge pull request #5275 from pks-t/pks/reflogs-with-newlines reflogs: fix behaviour around reflogs with newlines
Sven Strickroth dadbb33b 2019-11-01T18:55:54 Fix crash if snapshotting a config_snapshot Signed-off-by: Sven Strickroth <email@cs-ware.de>
Carlos Martín Nieto 718f24ad 2019-10-30T20:39:03 commit: verify objects exist in git_commit_with_signature There can be a significant difference between the system where we created the buffer (if at all) and when the caller provides us with the contents of a commit. Verify that the commit we are being asked to create references objects which do exist in the target repository.
Patrick Steinhardt 2a7d6de3 2019-10-29T07:52:31 Merge pull request #5276 from pks-t/pks/patch-fuzzing-fixes patch_parse: fixes for fuzzing errors
pcpthm 3f998aee 2019-10-26T17:21:29 Follow 308 redirect in WinHTTP transport
Patrick Steinhardt a31f4c4b 2019-10-24T13:16:03 Merge pull request #5227 from ddevault/check apply: add GIT_APPLY_CHECK
Patrick Steinhardt c405f231 2019-10-24T10:26:43 Merge pull request #5264 from henkesn/refs-unlock-on-commit refs: unlock unmodified refs on transaction commit
Drew DeVault 02af1fcb 2019-09-14T14:03:36 apply: add GIT_APPLY_CHECK This adds an option which will check if a diff is applicable without actually applying it; equivalent to git apply --check.
Patrick Steinhardt 37141ff7 2019-10-21T18:56:59 patch_parse: detect overflow when calculating old/new line position When the patch contains lines close to INT_MAX, then it may happen that we end up with an integer overflow when calculating the line of the current diff hunk. Reject such patches as unreasonable to avoid the integer overflow. As the calculation is performed on integers, we introduce two new helpers `git__add_int_overflow` and `git__sub_int_overflow` that perform the integer overflow check in a generic way.
Patrick Steinhardt 468e3ddc 2019-10-19T16:48:11 patch_parse: fix out-of-bounds read with No-NL lines We've got two locations where we copy lines into the patch. The first one is when copying normal " ", "-" or "+" lines, while the second location gets executed when we copy "\ No newline at end of file" lines. While the first one correctly uses `git__strndup` to copy only until the newline, the other one doesn't. Thus, if the line occurs at the end of the patch and if there is no terminating NUL character, then it may result in an out-of-bounds read. Fix the issue by using `git__strndup`, as was already done in the other location. Furthermore, add allocation checks to both locations to detect out-of-memory situations.
Patrick Steinhardt 6c6c15e9 2019-10-19T15:52:35 patch_parse: reject empty path names When parsing patch headers, we currently accept empty path names just fine, e.g. a line "--- \n" would be parsed as the empty filename. This is not a valid patch format and may cause `NULL` pointer accesses at a later place as `git_buf_detach` will return `NULL` in that case. Reject such patches as malformed with a nice error message.
Patrick Steinhardt 223e7e43 2019-10-19T15:42:54 patch_parse: reject patches with multiple old/new paths It's currently possible to have patches with multiple old path name headers. As we didn't check for this case, this resulted in a memory leak when overwriting the old old path with the new old path because we simply discarded the old pointer. Instead of fixing this by free'ing the old pointer, we should reject such patches altogether. It doesn't make any sense for the "---" or "+++" markers to occur multiple times within a patch n the first place. This also implicitly fixes the memory leak.
Patrick Steinhardt 7968e90f 2019-10-18T12:33:07 refdb_fs: properly parse corrupted reflogs In previous versions, libgit2 could be coerced into writing reflog messages with embedded newlines into the reflog by using `git_stash_save` with a message containing newlines. While the root cause is fixed now, it was noticed that upstream git is in fact able to read such corrupted reflog messages just fine. Make the reflog parser more lenient in order to just skip over malformatted reflog lines to bring us in line with git. This requires us to change an existing test that verified that we do indeed _fail_ to parse such logs.
Patrick Steinhardt 8532ed11 2019-10-18T12:14:19 refdb_fs: convert reflog parsing to use parser The refdb_fs code to parse the reflog currently uses a hand-rolled parser. Convert it to use our `git_parse_ctx` structure instead.
Patrick Steinhardt d8233feb 2019-10-18T09:24:14 reflog: allow adding entries with newlines in their message Currently, the reflog disallows any entries that have a message with newlines, as that would effectively break the reflog format, which may contain a single line per entry, only. Upstream git behaves a bit differently, though, especially when considering stashes: instead of rejecting any reflog entry with newlines, git will simply replace newlines with spaces. E.g. executing 'git stash push -m "foo\nbar"' will create a reflog entry with "foo bar" as entry message. This commit adjusts our own logic to stop rejecting commit messages with newlines. Previously, this logic was part of `git_reflog_append`, only. There is a second place though where we add reflog entries, which is the serialization code in the filesystem refdb. As it didn't contain any sanity checks whatsoever, the refdb would have been perfectly happy to write malformatted reflog entries to the disk. This is being fixed with the same logic as for the reflog itself.
Patrick Steinhardt 28481609 2019-10-18T09:41:20 stash: refactor code that prepares commit messages
Patrick Steinhardt ca2d34a8 2019-10-18T09:06:48 stash: modernize code style of `git_stash_save` The code style of `git_stash_save` doesn't really match our current coding style. Update it to match our current policies more closely.
Patrick Steinhardt c9464bf7 2019-10-17T18:33:12 Merge pull request #5273 from dlax/parse-diff-without-extended-headers patch_parse: handle patches without extended headers
Sebastian Henke 47531f47 2019-10-11T12:44:09 refs: unlock unmodified refs on transaction commit Refs which are locked in a transaction without an altered target, still should to be unlocked on `git_transaction_commit`. `git_transaction_free` also unlocks refs but the moment of calling of `git_transaction_free` cannot be controlled in all situations. Some binding libs call `git_transaction_free` on garbage collection or not at all if the application exits before and don't provide public access to `git_transaction_free`. It is better to release locks as soon as possible.
Etienne Samson 3c5d78bd 2019-05-01T16:16:26 submodule: provide a wrapper for simple submodule clone steps
Denis Laxalde 11de594f 2019-10-16T22:11:33 patch_parse: handle patches without extended headers Extended header lines (especially the "index <hash>..<hash> <mode>") are not required by "git apply" so it import patches. So we allow the from-file/to-file lines (--- a/file\n+++ b/file) to directly follow the git diff header. This fixes #5267.
Etienne Samson dbc17a7e 2019-09-21T08:46:08 negotiate: use GSS.framework on macOS
Etienne Samson 0eecb660 2019-10-13T13:53:18 cmake: remove extra GIT_NTLM define
Etienne Samson aa234ac0 2019-09-21T08:47:01 util: hide helper qsort code to silence unused functions warning
Patrick Steinhardt ef5a3851 2019-10-11T07:47:17 Merge pull request #5257 from henkesn/master Fix file locking on POSIX OS
Patrick Steinhardt 1f9b4970 2019-10-11T07:44:10 Merge pull request #5260 from pks-t/pks/cmake3 cmake: update minimum CMake version to v3.5.1
Sebastian Henke 3335a034 2019-10-10T15:28:46 refs: fix locks getting forcibly removed The flag GIT_FILEBUF_FORCE currently does two things: 1. It will cause the filebuf to create non-existing leading directories for the file that is about to be written. 2. It will forcibly remove any pre-existing locks. While most call sites actually do want (1), they do not want to remove pre-existing locks, as that renders the locking mechanisms effectively useless. Introduce a new flag `GIT_FILEBUF_CREATE_LEADING_DIRS` to separate both behaviours cleanly from each other and convert callers to use it instead of `GIT_FILEBUF_FORCE` to have them honor locked files correctly. As this conversion removes all current users of `GIT_FILEBUF_FORCE`, this commit removes the flag altogether.
Patrick Steinhardt 6716e2f3 2019-10-10T12:34:45 Merge pull request #5248 from dlax/parse-patch-empty-files patch_parse: handle patches with new empty files
Patrick Steinhardt ebabb88f 2019-10-10T09:25:32 cmake: update minimum CMake version to v3.5.1 Back in commit cf9f34521 (cmake: bump minimum version to 2.8.11, 2017-09-06), we have bumped the minimum CMake version to require at least v2.8.11. The main hold-backs back then were distributions like RHEL/CentOS as well as Ubuntu Trusty, which caused us to not target a more modern version. Nowadays, Ubuntu Trusty has been EOL'd and CentOS 6 has CMake v3.6.1 available via the EPEL6 repository, and thus it seems fair to upgrade to a more recent version. Going through repology [1], one can see that all supported mainstream distributions do in fact have CMake 3 available. Going through the list, the minimum version that is supported by all mainstream distros is in fact v3.5.1: - CentOS 6 via EPEL6: 3.6.1 - Debian Oldstable: 3.7.2 - Fedora 26: 3.8.2 - OpenMandriva 3.x: 3.5.1 - Slackware 14.2: 3.5.2 - Ubuntu 16.04: 3.5.1 Consequentally, let's upgrade CMake to the minimum version of 3.5.1 and remove all the version CMake checks that aren't required anymore. [1]: https://repology.org/project/cmake/versions
Patrick Steinhardt f04a58b0 2019-10-03T12:55:48 Merge pull request #4445 from tiennou/shallow/dry-commit-parsing DRY commit parsing
Patrick Steinhardt 5cf17e0f 2019-10-03T09:39:42 commit_list: store in/out-degrees as uint16_t The commit list's in- and out-degrees are currently stored as `unsigned short`. When assigning it the value of `git_array_size`, which returns an `size_t`, this generates a warning on some Win32 platforms due to loosing precision. We could just cast the returned value of `git_array_size`, which would work fine for 99.99% of all cases as commits typically have less than 2^16 parents. For crafted commits though we might end up with a wrong value, and thus we should definitely check whether the array size actually fits into the field. To ease the check, let's convert the fields to store the degrees as `uint16_t`. We shouldn't rely on such unspecific types anyway, as it may lead to different behaviour across platforms. Furthermore, this commit introduces a new `git__is_uint16` function to check whether it actually fits -- if not, we return an error.
Etienne Samson 5988cf34 2017-12-15T18:11:51 commit_list: unify commit information parsing
Etienne Samson 1c847a6a 2018-10-25T19:40:19 commit: generic parse mechanism This allows us to pick which data from a commit we're interested in. This will be used by the revwalk code, which is only interested in parents' and committer data.
Edward Thomson 63307cba 2019-09-28T17:32:18 Merge pull request #5226 from pks-t/pks/regexp-api regexp: implement a new regular expression API
Denis Laxalde b61810bf 2019-09-28T15:52:25 patch_parse: handle patches with new empty files Patches containing additions of empty files will not contain diff data but will end with the index header line followed by the terminating sequence "-- ". We follow the same logic as in cc4c44a and allow "-- " to immediately follow the index header.
Patrick Steinhardt 70325370 2019-09-27T11:16:02 Merge pull request #5106 from tiennou/fix/ref-api-fixes git_refdb API fixes
Sven Strickroth 452b7f8f 2019-09-25T20:29:21 Don't use enum for flags Using an `enum` causes trouble when used with C++ as bitwise operations are not possible w/o casting (e.g., `opts.flags &= ~GIT_BLOB_FILTER_CHECK_FOR_BINARY;` is invalid as there is no `&=` operator for `enum`). Signed-off-by: Sven Strickroth <email@cs-ware.de>
Edward Thomson 3c1aa232 2019-09-21T16:09:00 Merge pull request #5232 from pks-t/pks/buffer-ensure-size-oom buffer: fix writes into out-of-memory buffers
Patrick Steinhardt f585b129 2019-09-12T14:29:28 posix: remove superseded POSIX regex wrappers The old POSIX regex wrappers have been superseded by our own regexp API that provides a higher-level abstraction. Remove the POSIX wrappers in favor of the new one.
Patrick Steinhardt 7aacf027 2019-09-13T08:55:33 global: convert all users of POSIX regex to use our new regexp API The old POSIX regex API has been superseded by our new regexp API. Convert all users to make use of the new one.
Patrick Steinhardt d77378eb 2019-09-13T08:54:26 regexp: implement new regular expression API We currently support a set of different regular expression backends with PCRE, PCRE2, regcomp(3P) and regcomp_l(3). The current implementation of this is done via a simple POSIX wrapper that either directly uses supplied functions or that is a very small wrapper. To support PCRE and PCRE2, we use their provided <pcreposix.h> and <pcre2posix.h> wrappers. These wrappers are implemented in such a way that the accompanying libraries pcre-posix and pcre2-posix provide the same symbols as the libc ones, namely regcomp(3P) et al. This works out on some systems just fine, most importantly on glibc-based ones, where the regular expression functions are implemented as weak aliases and thus get overridden by linking in the pcre{,2}-posix library. On other systems we depend on the linking order of libc and pcre library, and as libc always comes first we will end up with the functions of the libc implementation. As a result, we may use the structures `regex_t` and `regmatch_t` declared by <pcre{,2}posix.h>, but use functions defined by the libc, leading to segfaults. The issue is not easily solvable. Somed distributions like Debian have resolved this by patching PCRE and PCRE2 to carry custom prefixes to all the POSIX function wrappers. But this is not supported by upstream and thus inherently unportable between distributions. We could instead try to modify linking order, but this starts becoming fragile and will not work e.g. when libgit2 is loaded via dlopen(3P) or similar ways. In the end, this means that we simply cannot use the POSIX wrappers provided by the PCRE libraries at all. Thus, this commit introduces a new regular expression API. The new API is on a tad higher level than the previous POSIX abstraction layer, as it tries to abstract away any non-portable flags like e.g. REG_EXTENDED, which has no equivalents in all of our supported backends. As there are no users of POSIX regular expressions that do _not_ reguest REG_EXTENDED this is fine to be abstracted away, though. Due to the API being higher-level than before, it should generally be a tad easier to use than the previous one. Note: ideally, the new API would've been called `git_regex_foobar` with a file "regex.h" and "regex.c". Unfortunately, this is currently impossible to implement due to naming clashes between the then-existing "regex.h" and <regex.h> provided by the libc. As we add the source directory of libgit2 to the header search path, an include of <regex.h> would always find our own "regex.h". Thus, we have to take the bitter pill of adding one more character to all the functions to disambiguate the includes. To improve guarantees around cross-backend compatibility, this commit also brings along an improved regular expression test suite core::regexp.
Patrick Steinhardt 174b7a32 2019-09-19T12:24:06 buffer: fix printing into out-of-memory buffer Before printing into a `git_buf` structure, we always call `ENSURE_SIZE` first. This macro will reallocate the buffer as-needed depending on whether the current amount of allocated bytes is sufficient or not. If `asize` is big enough, then it will just do nothing, otherwise it will call out to `git_buf_try_grow`. But in fact, it is insufficient to only check `asize`. When we fail to allocate any more bytes e.g. via `git_buf_try_grow`, then we set the buffer's pointer to `git_buf__oom`. Note that we touch neither `asize` nor `size`. So if we just check `asize > targetsize`, then we will happily let the caller of `ENSURE_SIZE` proceed with an out-of-memory buffer. As a result, we will print all bytes into the out-of-memory buffer instead, resulting in an out-of-bounds write. Fix the issue by having `ENSURE_SIZE` verify that the buffer is not marked as OOM. Add a test to verify that we're not writing into the OOM buffer.
Patrick Steinhardt 208f1d7a 2019-09-19T12:46:37 buffer: fix infinite loop when growing buffers When growing buffers, we repeatedly multiply the currently allocated number of bytes by 1.5 until it exceeds the requested number of bytes. This has two major problems: 1. If the current number of bytes is tiny and one wishes to resize to a comparatively huge number of bytes, then we may need to loop thousands of times. 2. If resizing to a value close to `SIZE_MAX` (which would fail anyway), then we probably hit an infinite loop as multiplying the current amount of bytes will repeatedly result in integer overflows. When reallocating buffers, one typically chooses values close to 1.5 to enable re-use of resulting memory holes in later reallocations. But because of this, it really only makes sense to use a factor of 1.5 _once_, but not looping until we finally are able to fit it. Thus, we can completely avoid the loop and just opt for the much simpler algorithm of multiplying with 1.5 once and, if the result doesn't fit, just use the target size. This avoids both problems of looping extensively and hitting overflows. This commit also adds a test that would've previously resulted in an infinite loop.
Patrick Steinhardt 3e8a17b0 2019-09-21T15:18:42 buffer: fix memory leak if unable to grow buffer If growing a buffer fails, we set its pointer to the static `git_buf__oom` structure. While we correctly free the old pointer if `git__malloc` returned an error, we do not free it if there was an integer overflow while calculating the new allocation size. Fix this issue by freeing the pointer to plug the memory leak.
Etienne Samson 49a3289e 2019-09-21T08:25:23 cred: add missing private header in GSSAPI block Should have been part of 8bf0f7eb26c65b2b937b1f40a384b9b269b0b76d
Patrick Steinhardt aa407ca3 2019-09-19T13:23:59 Merge pull request #5206 from tiennou/cmake/pkgconfig-building CMake pkg-config modulification