Log

Author Commit Date CI Message
Patrick Steinhardt 6777db8e 2019-12-10T20:40:17 Merge pull request #5331 from pks-t/security-fixes Security fixes for master
Patrick Steinhardt b8b8eee3 2019-12-10T13:44:27 changelog: document security fixes
Edward Thomson 85d4ff77 2019-12-03T19:50:18 index: ensure that we respect core.protectNTFS=false Users may want to turn off core.protectNTFS, perhaps to import (and then repair) a broken tree. Ensure that core.protectNTFS=false is honored.
Edward Thomson ba4c769b 2019-12-03T23:23:02 tree: ensure we protect NTFS paths everywhere
Edward Thomson 14ff3516 2019-12-03T23:15:47 path: support non-ascii drive letters on dos Windows/DOS only supports drive letters that are alpha characters A-Z. However, you can `subst` any one-character as a drive letter, including numbers or even emoji. Test that we can identify emoji as drive letters.
Edward Thomson e4034dfa 2019-12-03T19:24:59 path: protect NTFS everywhere Enable core.protectNTFS by default everywhere and in every codepath, not just on checkout.
Edward Thomson d9c0c9cf 2019-12-03T19:17:41 test: ensure we can't add a protected path Test that when we enable core.protectNTFS that we cannot add platform-specific invalid paths to the index.
Edward Thomson 72df1cd8 2019-12-03T19:01:00 test: improve badname verification test The name of the `add_invalid_filename` function suggests that we _want_ to add an invalid filename. Rename the function to show that we expect to _fail_ to add the invalid filename.
Edward Thomson f3b28604 2019-12-03T18:57:16 test: ensure treebuilder validate new protection rules Ensure that the new protection around .git::$INDEX_ALLOCATION rules are enabled for using the treebuilder when core.protectNTFS is set.
Edward Thomson 336991db 2019-12-03T18:56:31 test: ensure index adds validate new protection rules Ensure that the new protection around .git::$INDEX_ALLOCATION rules are enabled for adding to the index when core.protectNTFS is set.
Edward Thomson a3cbd204 2019-12-03T18:49:23 test: improve badname verification test The name of the `write_invalid_filename` function suggests that we _want_ to write an invalid filename. Rename the function to show that we expect to _fail_ to write the invalid filename.
Johannes Schindelin d29d4de2 2019-09-18T15:08:56 cl_git_fail: do not report bogus error message When we expect a checkout operation to fail, but it succeeds, we actually do not want to see the error messages that were generated in the meantime for errors that were handled gracefully by the code (e.g. when an object could not be found in a pack: in this case, the next backend would have been given a chance to look up the object, and probably would have found it because the checkout succeeded, after all). Which means that in the specific case of `cl_git_fail()`, we actually want to clear the global error state _after_ evaluating the command: we know that any still-available error would be bogus, seeing as the command succeeded (unexpectedly). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Edward Thomson b8464342 2019-12-03T17:47:31 path: rename function that detects end of filename The function `only_spaces_and_dots` used to detect the end of the filename on win32. Now we look at spaces and dots _before_ the end of the string _or_ a `:` character, which would signify a win32 alternate data stream. Thus, rename the function `ntfs_end_of_filename` to indicate that it detects the (virtual) end of a filename, that any further characters would be elided to the given path.
Johannes Schindelin e1832eb2 2019-09-18T16:33:18 path: also guard `.gitmodules` against NTFS Alternate Data Streams We just safe-guarded `.git` against NTFS Alternate Data Stream-related attack vectors, and now it is time to do the same for `.gitmodules`. Note: In the added regression test, we refrain from verifying all kinds of variations between short names and NTFS Alternate Data Streams: as the new code disallows _all_ Alternate Data Streams of `.gitmodules`, it is enough to test one in order to know that all of them are guarded against. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Johannes Schindelin 3f7851ea 2019-09-18T14:32:05 Disallow NTFS Alternate Data Stream attacks, even on Linux/macOS A little-known feature of NTFS is that it offers to store metadata in so-called "Alternate Data Streams" (inspired by Apple's "resource forks") that are copied together with the file they are associated with. These Alternate Data Streams can be accessed via `<file name>:<stream name>:<stream type>`. Directories, too, have Alternate Data Streams, and they even have a default stream type `$INDEX_ALLOCATION`. Which means that `abc/` and `abc::$INDEX_ALLOCATION/` are actually equivalent. This is of course another attack vector on the Git directory that we definitely want to prevent. On Windows, we already do this incidentally, by disallowing colons in file/directory names. While it looks as if files'/directories' Alternate Data Streams are not accessible in the Windows Subsystem for Linux, and neither via CIFS/SMB-mounted network shares in Linux, it _is_ possible to access them on SMB-mounted network shares on macOS. Therefore, let's go the extra mile and prevent this particular attack _everywhere_. To keep things simple, let's just disallow *any* Alternate Data Stream of `.git`. This is libgit2's variant of CVE-2019-1352. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Johannes Schindelin 64c612cc 2019-09-18T15:25:02 Protect against 8.3 "short name" attacks also on Linux/macOS The Windows Subsystem for Linux (WSL) is getting increasingly popular, in particular because it makes it _so_ easy to run Linux software on Windows' files, via the auto-mounted Windows drives (`C:\` is mapped to `/mnt/c/`, no need to set that up manually). Unfortunately, files/directories on the Windows drives can be accessed via their _short names_, if that feature is enabled (which it is on the `C:` drive by default). Which means that we have to safeguard even our Linux users against the short name attacks. Further, while the default options of CIFS/SMB-mounts seem to disallow accessing files on network shares via their short names on Linux/macOS, it _is_ possible to do so with the right options. So let's just safe-guard against short name attacks _everywhere_. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Edward Thomson 6bd37c34 2019-12-04T14:33:14 Merge pull request #5318 from libgit2/ethomson/release_docs release.md: note that we do two security releases
Edward Thomson ea702a0b 2019-12-04T14:25:36 release.md: note that we do two security releases Note that for security releases, we update the two most recent major release branches.
Edward Thomson ad1548e4 2019-12-04T13:51:03 Merge pull request #5317 from csware/size_t MSVC: Fix warning C4133 on x64: "function": Incompatible types - from "unsigned long *" to "size_t *"
Sven Strickroth bdf96512 2019-12-03T21:17:30 MSVC: Fix warning C4133 on x64: "function": Incompatible types - from "unsigned long *" to "size_t *" Signed-off-by: Sven Strickroth <email@cs-ware.de>
Edward Thomson 4fe52df6 2019-12-03T18:26:24 Merge pull request #5316 from libgit2/ethomson/publish_docs_on_master ci: only push docs from the libgit2/libgit2 repo
Edward Thomson 9c30fbed 2019-12-03T18:09:03 ci: only push docs from the libgit2/libgit2 repo Users may fork libgit2 and run libgit2's CI on that, which is delightful! However, if they do, we'll fail the documentation publish phase, which is correct (because we don't allow them to publish _their_ version of the docs) but regrettable (since it should not fail). Only run the documentation publish phase when we merge branches into the libgit2/libgit2 repo.
Edward Thomson 36bfc27a 2019-12-01T14:38:54 Merge pull request #5314 from pks-t/pks/dll-main-removal global: convert to fiber-local storage to fix exit races
Edward Thomson d298f9b2 2019-12-01T14:11:56 Merge pull request #5315 from kastiglione/dl/fix-copypaste-in-git_cherrypick_commit-docstring Fix copy&paste in git_cherrypick_commit docstring
Edward Thomson 7f6fdb82 2019-12-01T14:11:22 Merge pull request #5312 from pks-t/pks/patch-base85-overflow patch_parse: fix out-of-bounds reads caused by integer underflow
Dave Lee b7cf4b9e 2019-11-29T14:16:04 Fix copy&paste in git_cherrypick_commit docstring
Patrick Steinhardt 5c6180b5 2019-11-29T11:06:11 global: convert to fiber-local storage to fix exit races On Windows platforms, we automatically clean up the thread-local storage upon detaching a thread via `DllMain()`. The thing is that this happens for every thread of applications that link against the libgit2 DLL, even those that don't have anything to do with libgit2 itself. As a result, we cannot assume that these unsuspecting threads make use of our `git_libgit2_init()` and `git_libgit2_shutdow()` reference counting, which may lead to racy situations: Thread 1 Thread 2 git_libgit2_shutdown() DllMain(DETACH_THREAD) git__free_tls_data() git_atomic_dec() == 0 git__free_tls_data() TlsFree(_tls_index) TlsGetValue(_tls_index) Due to the second thread never having executed `git_libgit2_init()`, the first thread will clean up TLS data and as a result also free the `_tls_index` variable. When detaching the second thread, we unconditionally access the now-free'd `_tls_index` variable, which is obviously not going to work out well. Fix the issue by converting the code to use fiber-local storage instead of thread-local storage. While FLS will behave the exact same as TLS if no fibers are in use, it does allow us to specify a destructor similar to the one that is accepted by pthread_key_create(3P). Like this, we do not have to manually free indices anymore, but will let the FLS handle calling the destructor. This allows us to get rid of `DllMain()` completely, as we only used it to keep track of when threads were exiting and results in an overall simplification of TLS cleanup.
Edward Thomson 7f20778b 2019-11-29T09:14:04 Merge pull request #5311 from pks-t/pks/clar-trace-warning tests: fix compiler warning if tracing is disabled
Edward Thomson 61038425 2019-11-29T09:13:33 Merge pull request #5313 from pks-t/pks/config-invasive tests: config: only test parsing huge file with GITTEST_INVASIVE_SPEED
Patrick Steinhardt 361ebbcb 2019-11-28T15:36:40 tests: config: only test parsing huge file with GITTEST_INVASIVE_SPEED The test in config::stress::huge_section_with_many_values takes quite a long time to execute. Hide it behind the GITTEST_INVASIVE_SPEED environment varibale to not needlessly blow up execution time of tests. As this environment variable is being set by the continuous integration, we will execute it regularly anyway.
Patrick Steinhardt 33e6c402 2019-11-28T15:26:36 patch_parse: fix out-of-bounds reads caused by integer underflow The patch format for binary files is a simple Base85 encoding with a length byte as prefix that encodes the current line's length. For each line, we thus check whether the line's actual length matches its expected length in order to not faultily apply a truncated patch. This also acts as a check to verify that we're not reading outside of the line's string: if (encoded_len > ctx->parse_ctx.line_len - 1) { error = git_parse_err(...); goto done; } There is the possibility for an integer underflow, though. Given a line with a single prefix byte, only, `line_len` will be zero when reaching this check. As a result, subtracting one from that will result in an integer underflow, causing us to assume that there's a wealth of bytes available later on. Naturally, this may result in an out-of-bounds read. Fix the issue by checking both `encoded_len` and `line_len` for a non-zero value. The binary format doesn't make use of zero-length lines anyway, so we need to know that there are both encoded bytes and remaining characters available at all. This patch also adds a test that works based on the last error message. Checking error messages is usually too tightly coupled, but in fact parsing the patch failed even before the change. Thus the only possibility is to use e.g. Valgrind, but that'd result in us not catching issues when run without Valgrind. As a result, using the error message is considered a viable tradeoff as we know that we didn't start decoding Base85 in the first place.
Patrick Steinhardt 1d470a71 2019-11-28T14:45:15 tests: fix compiler warning if tracing is disabled If building libgit2's test suite with tracing disabled, then the compiler will emit a warning due to the unused `message_prefix` function. Fix the issue by wrapping the whole file into ifdef's for `GIT_TRACE` and providing separate empty function implementations for both `cl_global_trace_register` and `cl_global_trace_disable`.
Patrick Steinhardt fb439c97 2019-11-28T14:41:58 Merge pull request #5306 from herrerog/patchid diff: complete support for git patchid
Patrick Steinhardt 61176a9b 2019-11-28T14:31:16 Merge pull request #5243 from pks-t/pks/config-optimize-mem Memory optimizations for config entries
Gregory Herrero ece5bb5e 2019-11-07T14:10:00 diff: make patchid computation work with all types of commits. Current implementation of patchid is not computing a correct patchid when given a patch where, for example, a new file is added or removed. Some more corner cases need to be handled to have same behavior as git patch-id command. Add some more tests to cover those corner cases. Signed-off-by: Gregory Herrero <gregory.herrero@oracle.com>
Patrick Steinhardt 0b5540b9 2019-11-28T13:56:54 Merge pull request #5307 from palmin/hash_sha256 ssh: include sha256 host key hash when supported
Patrick Steinhardt dfea0713 2019-11-28T13:51:40 Merge pull request #5272 from tiennou/examples/cli-ification Various examples shape-ups
Patrick Steinhardt b63ad958 2019-11-28T13:19:50 Merge pull request #5309 from libgit2/ethomson/trace Improve trace support in tests
Patrick Steinhardt 0e5243b7 2019-11-28T12:42:36 Merge pull request #5123 from libgit2/ethomson/off_t Move `git_off_t` to `git_object_size_t`
Edward Thomson 7198d345 2019-11-28T15:12:05 Merge pull request #5310 from lberk/compat-git-attr-t Add compat typdef for git_attr_t
Lukas Berk 5ace4ccf 2019-11-27T22:40:17 Move deprecated git_attr_t typedef to previous attribute section
Lukas Berk 3739a15c 2019-11-27T21:31:25 Add attr.h include
Lukas Berk aea049b6 2019-11-27T20:05:32 Add compat typdef for git_attr_t Some libraries haven't updated to git_attr_value_t and break. Adding the comapt typedef as suggested.
Edward Thomson b7f70bc2 2019-11-27T12:36:17 tests: optionally show test execution tracing Only show test trace execution when the CLAR_TRACE_TESTS environment variable is set. This reduces the noise during tracing.
Edward Thomson 85efe896 2019-11-27T12:34:10 tests: display trace level with prefix in tests
Edward Thomson 625a3a49 2019-11-27T12:29:34 trace: enable tracing by default Tracing is meant to be extremely low-impact when not enabled. We currently ship no tracing calls in libgit2, but if / when we do, the tracing infrastructure is created to skip tracing as quickly as possible. It should compile to a simple test when tracing is off. Thus, there's on reason to not enable it by default.
Edward Thomson 7805122b 2019-11-27T14:22:27 Merge pull request #5308 from libgit2/ethomson/cifix CI Build Updates
Edward Thomson 6460e8ab 2019-06-23T18:13:29 internal: use off64_t instead of git_off_t Prefer `off64_t` internally.
Edward Thomson 05237ee5 2019-06-23T17:20:17 integer: use int64_t's for checks Use int64_t internally for type visibility.
Edward Thomson ee0c8618 2019-06-23T17:19:31 offmap: store off64_t's instead of git_off_t's Prefer `off64_t` to `git_off_t` internally for visibility.
Edward Thomson 8be12026 2019-06-23T17:09:22 mmap: use a 64-bit signed type `off64_t` for mmap Prefer `off64_t` to `git_off_t` for internal visibility.
Edward Thomson 7e1cc296 2019-11-25T13:17:42 mmap: remove unnecessary assertion 64 bit types are always 64 bit.
Edward Thomson c863b3c8 2019-11-24T16:49:23 ci: enable the VALGRIND flag on builds
Edward Thomson cb77423f 2019-11-24T16:22:31 valgrind: add valgrind hints in OpenSSL Provide usage hints to valgrind. We trust the data coming back from OpenSSL to have been properly initialized. (And if it has not, it's an OpenSSL bug, not a libgit2 bug.) We previously took the `VALGRIND` option to CMake as a hint to disable mmap. Remove that; it's broken. Now use it to pass on the `VALGRIND` definition so that sources can provide valgrind hints.
Edward Thomson 2ad3eb3e 2019-11-24T15:59:26 valgrind: add suppressions for undefined use valgrind will warn that OpenSSL will use undefined data in connect/read when talking to certain other TLS stacks. Thankfully, this only seems to occur when gcc is the compiler, so hopefully valgrind is just misunderstanding an optimization. Regardless, suppress this warning.
Edward Thomson 0005c77a 2019-11-24T15:49:49 test: add an azure repos test We currently talk to Azure Repos for executing an online test (online::clone::path_whitespace). Add a simpler test to talk to Azure Repos to make it obvious that strange test failures are not likely the whitespace in the path, but actually a function of talking to Azure Repos itself.
Edward Thomson f592c737 2019-11-23T11:55:50 ci: don't install libssh2 since we build it
Edward Thomson b8e00b98 2019-11-23T21:17:15 ci: cache docker layers Our docker builds are getting expensive, let's cache some of this.
Edward Thomson 6df3ec4a 2019-11-23T21:14:32 valgrind: suppress libssh2_rsa_sha1_sign leaks
Edward Thomson c64b7aaa 2019-11-23T20:38:30 ci: build our own valgrind The valgrind in the PPA is broken and ignores `--exit-errorcode`. Build and install our own.
Edward Thomson 7adc32d5 2019-11-23T13:02:29 valgrind: suppress kexinit leaks
Edward Thomson fd831275 2019-11-23T12:40:46 ci: build shared libssh2
Edward Thomson 84807884 2019-11-23T12:40:02 ci: break dockerfile into stages Use a multi-stage docker build so that we can cache early stages and not need to download the apt-provided dependencies during every build (when only later stages change).
Edward Thomson 7a3d04dc 2019-11-23T12:14:23 ci: don't delete the apt cache Deleting the apt cache can be helpful for reducing the size of a container, but since we don't push it anywhere, it only hinders our ability to debug problems while working on the container. Keep it.
Edward Thomson 5dc1be8d 2019-11-23T11:25:56 valgrind: suppress uninitialized reads in libcrypto libcrypto will read uninitialized memory as entropy. Suppress warnings from this behavior.
Edward Thomson 767990e9 2019-11-23T11:25:38 ci: show distribution information The lsb-release command is missing on our images; just show the information from the file instead of relying on it.
Edward Thomson 91ba65af 2019-11-23T10:58:38 ci: provide a default for xcode generator Provide a sane default for `CMAKE_GENERATOR` in the build script so that it can be invoked without having to set that in the environment.
Edward Thomson f94c9276 2019-10-27T22:20:38 example: use `git_object_size_t` for object size
Edward Thomson 4dffa295 2019-06-23T18:09:00 blame: use a size_t for the buffer
Edward Thomson 6c13cf6d 2019-11-22T15:18:54 filestamp: use `uint64_t` for object size Instead of using a signed type (`off_t`) use an unsigned `uint64_t` for the size of the files.
Edward Thomson fefefd1d 2019-06-23T16:42:14 odb: use `git_object_size_t` for object size Instead of using a signed type (`off_t`) use a new `git_object_size_t` for the sizes of objects.
Edward Thomson fb2198db 2019-06-23T16:23:59 futils_filesize: use `uint64_t` for object size Instead of using a signed type (`off_t`) use `uint64_t` for the maximum size of files.
Edward Thomson 4334b177 2019-06-23T15:43:38 blob: use `git_object_size_t` for object size Instead of using a signed type (`off_t`) use a new `git_object_size_t` for the sizes of objects.
Edward Thomson bed9fc6b 2019-06-23T15:16:47 odb: use `git_object_size_t` for object size Instead of using a signed type (`off_t`) use a new `git_object_size_t` for the sizes of objects.
Edward Thomson 9b04d0be 2019-11-22T15:04:09 types: introduce `git_object_size_t` Introduce `git_object_size_t`, an unsigned type that we can use for the maximum size of git objects.
Anders Borum 48c3f7e1 2019-11-20T11:21:14 ssh: include sha256 host key hash when supported
Gregory Herrero 048e94ad 2019-11-07T14:13:14 patch_parse: correct parsing of patch containing not shown binary data. When not shown binary data is added or removed in a patch, patch parser is currently returning 'error -1 - corrupt git binary header at line 4'. Fix it by correctly handling case where binary data is added/removed. Signed-off-by: Gregory Herrero <gregory.herrero@oracle.com>
Gregory Herrero b921964b 2019-11-07T13:08:51 diff_print: add support for GIT_DIFF_FORMAT_PATCH_ID. Git is generating patch-id using a stripped down version of a patch where hunk header and index information are not present. Signed-off-by: Gregory Herrero <gregory.herrero@oracle.com>
Gregory Herrero accd7848 2019-11-07T13:02:38 diff_print: add a new 'print_index' flag when printing diff. Add a new 'print_index' flag to let the caller decide whether or not 'index <oid>..<oid>' should be printed. Since patch id needs not to have index when hashing a patch, it will be useful soon. Signed-off-by: Gregory Herrero <gregory.herrero@oracle.com>
Edward Thomson 47dd665a 2019-11-16T15:21:56 Merge pull request #5303 from pks-t/pks/patch-path-in-body-only patch_parse: use paths from "---"/"+++" lines for binary patches
Edward Thomson cb6bc6f2 2019-11-16T15:17:54 Merge pull request #5285 from pcpthm/winhttp-308 Follow 308 redirect in WinHTTP transport
Edward Thomson 541b8fc5 2019-11-16T15:12:52 Merge pull request #5302 from tiennou/fix/p_lstat-errno fileops: correct error return on p_lstat failures when mkdir
Patrick Steinhardt de7659cc 2019-11-10T18:44:56 patch_parse: use paths from "---"/"+++" lines for binary patches For some patches, it is not possible to derive the old and new file paths from the patch header's first line, most importantly when they contain spaces. In such a case, we derive both paths from the "---" and "+++" lines, which allow for non-ambiguous parsing. We fail to use these paths when parsing binary patches without data, though, as we always expect the header paths to be filled in. Fix this by using the "---"/"+++" paths by default and only fall back to header paths if they aren't set. If neither of those paths are set, we just return an error. Add two tests to verify this behaviour, one of which would have previously caused a segfault.
Etienne Samson 0c2b0206 2019-11-09T09:41:13 fileops: correct error return on p_lstat failures when mkdir IIRC I got a strange return once from lstat, which translated in a weird error class/message being reported. As a safety measure, enforce a -1 return in that case.
Edward Thomson 01ea911b 2019-11-06T06:04:55 Merge pull request #5299 from pks-t/pks/config-mem-snapshots config_mem: implement support for snapshots
Edward Thomson a3d8a437 2019-11-06T06:04:37 Merge pull request #5298 from pks-t/pks/patch-whitespace-only-paths patch_parse: fix segfault when header path contains whitespace only
Etienne Samson fe42557a 2019-11-06T11:08:52 examples: buff up rev-list by adding OID support This allows the example to be used as a quick revwalk test harness.
Etienne Samson 313908f9 2019-11-06T11:08:49 examples: normalize decls and usage of options structs
Etienne Samson 4a4ad2bc 2019-11-06T11:08:45 examples: add comments to add.c
Etienne Samson d4a593ef 2019-11-06T11:17:52 examples: modernize add code
Etienne Samson c9a09b91 2019-11-06T11:16:47 examples: extract argument conversion helper
Etienne Samson 204a464f 2019-11-06T11:08:39 examples: fixup for-each-ref style
Etienne Samson c924f36a 2019-11-06T11:08:35 examples: keep track of whether we processed a "--" arg
Etienne Samson 025a9357 2019-11-06T11:08:30 examples: move "args" to its own header
Etienne Samson 745ccc8a 2019-11-06T11:08:26 examples: remove duplicate includes from common.c
Etienne Samson cd5e33fb 2019-11-06T11:08:23 global: DRY includes of assert.h
Etienne Samson 882220bf 2019-11-06T11:08:19 examples: add missing include barriers
Etienne Samson 4867523e 2019-11-06T11:08:15 examples: add *.h files to IDEs
Patrick Steinhardt 146e5bf7 2019-11-06T07:27:35 config_mem: implement support for snapshots Similar as in commit dadbb33b6 (Fix crash if snapshotting a config_snapshot, 2019-11-01), let's implement snapshots for in-memory configuration entries. As this deletes more code than it adds, it doesn't make any sense to not allow for this and allows users to treat config backends mostly the same.
Patrick Steinhardt de543e29 2019-11-05T22:44:27 patch_parse: fix segfault when header path contains whitespace only When parsing header paths from a patch, we reject any patches with empty paths as malformed patches. We perform the check whether a path is empty before sanitizing it, though, which may lead to a path becoming empty after the check, e.g. if we have trimmed whitespace. This may lead to a segfault later when any part of our patching logic actually references such a path, which may then be a `NULL` pointer. Fix the issue by performing the check after sanitizing. Add tests to catch the issue as they would have produced a segfault previosuly.