src/streams


Log

Author Commit Date CI Message
Patrick Steinhardt 0ceac0d0 2019-01-23T14:45:19 mbedtls: fix potential size overflow when reading or writing data The mbedtls library uses a callback mechanism to allow downstream users to plug in their own receive and send functions. We implement `bio_read` and `bio_write` functions, which simply wrap the `git_stream_read` and `git_stream_write` functions, respectively. The problem arises due to the return value of the callback functions: mbedtls expects us to return an `int` containing the actual number of bytes that were read or written. But this is in fact completely misdesigned, as callers are allowed to pass in a buffer with length `SIZE_MAX`. We thus may be unable to represent the number of bytes written via the return value. Fix this by only ever reading or writing at most `INT_MAX` bytes.
Patrick Steinhardt 75918aba 2019-01-23T14:43:54 mbedtls: make global variables static The mbedtls stream implementation makes use of some global variables which are not marked as `static`, even though they're only used in this compilation unit. Fix this and remove a duplicate declaration.
Patrick Steinhardt 657197e6 2019-01-23T15:54:05 openssl: fix potential size overflow when writing data Our `openssl_write` function calls `SSL_write` by passing in both `data` and `len` arguments directly. Thing is, our `len` parameter is of type `size_t` and theirs is of type `int`. We thus need to clamp our length to be at most `INT_MAX`.
Patrick Steinhardt 7613086d 2019-01-23T15:49:28 streams: handle short writes only in generic stream Now that the function `git_stream__write_full` exists and callers of `git_stream_write` have been adjusted, we can lift logic for short writes out of the stream implementations. Instead, this is now handled either by `git_stream__write_full` or by callers of `git_stream_write` directly.
Patrick Steinhardt 5265b31c 2019-01-23T15:00:20 streams: fix callers potentially only writing partial data Similar to the write(3) function, implementations of `git_stream_write` do not guarantee that all bytes are written. Instead, they return the number of bytes that actually have been written, which may be smaller than the total number of bytes. Furthermore, due to an interface design issue, we cannot ever write more than `SSIZE_MAX` bytes at once, as otherwise we cannot represent the number of bytes written to the caller. Unfortunately, no caller of `git_stream_write` ever checks the return value, except to verify that no error occurred. Due to this, they are susceptible to the case where only partial data has been written. Fix this by introducing a new function `git_stream__write_full`. In contrast to `git_stream_write`, it will always return either success or failure, without returning the number of bytes written. Thus, it is able to write all `SIZE_MAX` bytes and loop around `git_stream_write` until all data has been written. Adjust all callers except the BIO callbacks in our mbedtls and OpenSSL streams, which already do the right thing and require the amount of bytes written.
Patrick Steinhardt 193e7ce9 2019-01-23T15:42:07 streams: make file-local functions static The callback functions that implement the `git_stream` structure are only used inside of their respective implementation files, but they are not marked as `static`. Fix this.
Edward Thomson f1986a23 2019-01-21T09:56:23 streams: don't write more than SSIZE_MAX Our streams implementation takes a `size_t` that indicates the length of the data buffer to be written, and returns an `ssize_t` that indicates the length that _was_ written. Clearly no such implementation can write more than `SSIZE_MAX` bytes. Ensure that each TLS stream implementation does not try to write more than `SSIZE_MAX` bytes (or smaller; if the given implementation takes a smaller size).
Edward Thomson e09f0c10 2019-01-23T10:21:42 deprecation: don't use deprecated stream cb Avoid the deprecated `git_stream_cb` typedef since we want to compile the library without deprecated functions or types. Instead, we can unroll the alias to its actual type.
Sven Strickroth bff7aed2 2019-01-24T16:44:04 Don't use deprecated constants Follow up for PR #4917. Signed-off-by: Sven Strickroth <email@cs-ware.de>
Edward Thomson f673e232 2018-12-27T13:47:34 git_error: use new names in internal APIs and usage Move to the `git_error` name in the internal API for error-related functions.
lhchavez 7b453e7e 2019-01-05T22:12:48 Fix a bunch of warnings This change fixes a bunch of warnings that were discovered by compiling with `clang -target=i386-pc-linux-gnu`. It turned out that the intrinsics were not necessarily being used in all platforms! Especially in GCC, since it does not support __has_builtin. Some more warnings were gleaned from the Windows build, but I stopped when I saw that some third-party dependencies (e.g. zlib) have warnings of their own, so we might never be able to enable -Werror there.
Edward Thomson 02bb39f4 2018-11-22T08:49:09 stream registration: take an enum type Accept an enum (`git_stream_t`) during custom stream registration that indicates whether the registration structure should be used for standard (non-TLS) streams or TLS streams.
Edward Thomson df2cc108 2018-11-18T10:29:07 stream: provide generic registration API Update the new stream registration API to be `git_stream_register` which takes a registration structure and a TLS boolean. This allows callers to register non-TLS streams as well as TLS streams. Provide `git_stream_register_tls` that takes just the init callback for backward compatibliity.
Edward Thomson 21142c5a 2018-10-29T10:04:48 http: remove cURL We previously used cURL to support HTTP proxies. Now that we've added this support natively, we can remove the curl dependency.
Edward Thomson 2878ad08 2018-10-29T08:59:33 streams: remove unused tls functions The implementations of git_openssl_stream_new and git_mbedtls_stream_new have callers protected by #ifdefs and are never called unless compiled in. There's no need for a dummy implementation. Remove them.
Edward Thomson 43b592ac 2018-10-25T08:49:01 tls: introduce a wrap function Introduce `git_tls_stream_wrap` which will take an existing `stream` with an already connected socket and begin speaking TLS on top of it. This is useful if you've built a connection to a proxy server and you wish to begin CONNECT over it to tunnel a TLS connection. Also update the pluggable TLS stream layer so that it can accept a registration structure that provides an `init` and `wrap` function, instead of a single initialization function.
Etienne Samson 03994912 2018-08-29T01:57:24 openssl: only say we're connected if the connection succeeded ssl_close uses this boolean to know if SSL_shutdown should be called. It turns out OpenSSL auto-shutdowns on failure, so if the call to SSL_connect fails, it will complain about "shutdown while in init", trampling the original error.
Etienne Samson caee0a66 2018-08-29T01:57:22 openssl: set the error class to GITERR_SSL
Patrick Steinhardt 2613fbb2 2018-10-18T11:58:14 global: replace remaining use of `git__strtol32` Replace remaining uses of the `git__strtol32` function. While these uses are all safe as the strings were either sanitized or from a trusted source, we want to remove `git__strtol32` altogether to avoid future misuse.
Etienne Samson 21496c30 2018-03-30T00:20:23 stransport: fix a warning on iOS "warning: values of type 'OSStatus' should not be used as format arguments; add an explicit cast to 'int' instead [-Wformat]"
Patrick Steinhardt d4198d4d 2018-07-26T12:11:34 mbedtls: remove unused variable "cacert" In commit 382ed1e87 (mbedtls: load default CA certificates, 2018-03-29), the function `git_mbedtls_stream_global_init` was refactored to call out to `git_mbedtls__set_cert_location` instead of setting up the certificates itself. The conversion forgot to remove the now-unused "cacert" variable, which is now only getting declared to be free'd at the end of the function. Remove it.
Edward Thomson 2fabb622 2018-07-21T01:36:46 mbedtls: free stream on shutdown
Edward Thomson 9e002cd5 2018-07-21T01:11:58 mbedtls: make ciphers_list a static array Instead of allocating the ciphers_list, make it a static array. This prevents us from leaking it or having to manage its memory.
Edward Thomson 4e62d26f 2018-07-21T00:45:24 mbedtls: free ciphers_list
Edward Thomson defa9709 2018-07-21T00:41:38 mbedtls: check allocations
Patrick Steinhardt d19381e2 2018-06-25T14:57:07 mbedtls: fix `inline` being used in mbedtls headers The mbedtls headers make direct use of the `inline` attribute to instruct the compiler to inline functions. As this function is not C90 compliant, this can cause the compiler to error as soon as any of these files is included and the `-std=c90` flag is being added. The mbedtls headers declaring functions as inline always have a prelude which define `inline` as a macro in case it is not yet defined. Thus, we can easily replace their define with our own define, which simply copies the logic of our own `GIT_INLINE` macro.
Patrick Steinhardt 75395c87 2018-06-29T13:35:14 streams: report OpenSSL errors if global init fails In case when the global initialization of the OpenSSL stream fails, the user is left without any hint as to what went wrong as we do not provide any error message at all. This commit refactors the init function to have a common error path, which now also sets an error message including the error string provided by OpenSSL.
Nikita Leshenko 05d89e72 2018-06-25T17:30:04 streams: openssl: Handle error in SSL_CTX_new SIGSEGV otherwise...
Quentin Minster b1cab70b 2018-05-30T02:15:09 streams: openssl: add missing check on OPENSSL_LEGACY_API The `CRYPTO_THREADID` type is no longer available in OpenSSL ≥ 1.1.0 with deprecated features disabled, and causes build failures. Since the `threadid_cb()` function is only ever called by `git_openssl_set_locking()` when `defined(OPENSSL_LEGACY_API)`, only define it then.
Patrick Steinhardt ba5e39ac 2018-05-04T15:25:11 streams: openssl: fix bogus warning on unused parameter Our provided callback function `threadid_cb(CRYPTO_THREADID *threadid)` sets up a unique thread ID by asking pthread for the current thread ID. Since openssl version 1.1, `CRYPTO_THREADID_set_numeric` is simply a no-op macro, leaving the `threadid` argument unused after the preprocessor has processed the macro. GCC does not account for that situation and will thus complain about `threadid` being unused. Silence this warning by using `GIT_UNUSED(threadid)`.
Carlos Martín Nieto 26a09a93 2018-04-30T21:34:36 Merge pull request #4608 from pks-t/pks/openssl-api-cleanup OpenSSL legacy API cleanups
Etienne Samson 173a0375 2018-02-08T23:50:14 openssl: remove leftover #ifdef This is the "OpenSSL available" global init function after all
Etienne Samson b3e0280d 2018-03-29T22:14:11 mbedtls: display error codes as hex for consistency with mbedTLS docs Remaining parts of https://github.com/JuliaLang/julia/blob/8d47a314537779c8fb86642c54925613628a91b0/deps/patches/libgit2-mbedtls-fixup.patch
Etienne Samson 382ed1e8 2018-03-29T22:14:09 mbedtls: load default CA certificates
Etienne Samson 1edde0bc 2018-03-29T22:14:08 mbedtls: use mbedTLS certificate verification Taken from https://github.com/JuliaLang/julia/blob/8d47a314537779c8fb86642c54925613628a91b0/deps/patches/libgit2-mbedtls-verify.patch, with some modifications.
Etienne Samson 4165bb7f 2018-03-29T22:14:06 mbedtls: use our own certificate validation Otherwise REQUIRED means that `git_stream_certificate` will always error. We're doing the mbedtls check in verify_server_cert though.
Etienne Samson ec79b0fd 2018-03-29T22:14:04 mbedtls: fix libgit2 hanging due to incomplete writes
Etienne Samson 2419cccd 2018-03-29T22:14:02 mbedtls: default cipher list support
Etienne Samson 60e1ad92 2018-03-29T22:14:01 mbedtls: add global initialization
Etienne Samson 1a1875f3 2018-03-29T22:13:58 mbedtls: proper certificate verification
Etienne Samson ca3b2234 2018-03-29T22:13:56 mbedtls: initial support
Patrick Steinhardt dd9de88a 2018-04-03T11:57:45 streams: openssl: provide `OPENSSL_init_ssl` for legacy API In order to further avoid using ifdef's in our code flow, provide the function `OPENSSL_init_ssl` in case we are using the legacy OpenSSL API.
Patrick Steinhardt ede63b99 2018-04-03T11:45:00 streams: openssl: unify version checks into single define By now, we have several locations where we are checking the version of OpenSSL to determine whether we can use the new "modern" API or need to use the pre-1.1 legacy API. As we have multiple implementations of OpenSSL with the rather recent libressl implementation, these checks need to honor versions of both implementations, which is rather tedious. Instead, we can just check once for the correct versions and define `OPENSSL_LEGACY_API` in case we cannot use the modern API.
Patrick Steinhardt 2505cbfc 2018-04-03T11:40:39 streams: openssl: move OpenSSL compat layer into implementation OpenSSL version 1.1 has broken its API in quite a few ways. To avoid having to use ifdef's everywhere, we have implemented the BIO functions added in version 1.1 ourselves in case we are using the legacy API. We were implementing them in the header file, though, which doesn't make a lot of sense, since these functions are only ever being used the the openssl stream implementation. Move these functions to the implementation file and mark them static.
Bernard Spil 7490d449 2018-04-02T20:00:07 Fix build with LibreSSL 2.7 LibreSSL 2.7 adds OpenSSL 1.1 API Signed-off-by: Bernard Spil <brnrd@FreeBSD.org>
Patrick Steinhardt 2022b004 2018-02-28T12:06:59 curl: explicitly initialize and cleanup global curl state Our curl-based streams make use of the easy curl interface. This interface automatically initializes and de-initializes the global curl state by calling out to `curl_global_init` and `curl_global_cleanup`. Thus, all global state will be repeatedly re-initialized when creating multiple curl streams in succession. Despite being inefficient, this is not thread-safe due to `curl_global_init` being not thread-safe itself. Thus a multi-threaded programing handling multiple curl streams at the same time is inherently racy. Fix the issue by globally initializing and cleaning up curl's state.
Patrick Steinhardt 84f03b3a 2018-02-16T10:48:55 streams: openssl: fix use of uninitialized variable When verifying the server certificate, we do try to make sure that the hostname actually matches the certificate alternative names. In cases where the host is either an IPv4 or IPv6 address, we have to compare the binary representations of the hostname with the declared IP address of the certificate. We only do that comparison in case we were successfully able to parse the hostname as an IP, which would always result in the memory region being initialized. Still, GCC 6.4.0 was complaining about usage of non-initialized memory. Fix the issue by simply asserting that `addr` needs to be initialized. This shuts up the GCC warning.
Edward Thomson a223bae5 2018-01-03T14:57:25 Merge pull request #4437 from pks-t/pks/openssl-hash-errors hash: openssl: check return values of SHA1_* functions
Patrick Steinhardt ba56f781 2018-01-03T12:54:42 streams: openssl: fix thread-safety for OpenSSL error messages The function `ERR_error_string` can be invoked without providing a buffer, in which case OpenSSL will simply return a string printed into a static buffer. Obviously and as documented in ERR_error_string(3), this is not thread-safe at all. As libgit2 is a library, though, it is easily possible that other threads may be using OpenSSL at the same time, which might lead to clobbered error strings. Fix the issue by instead using a stack-allocated buffer. According to the documentation, the caller has to provide a buffer of at least 256 bytes of size. While we do so, make sure that the buffer will never get overflown by switching to `ERR_error_string_n` to specify the buffer's size.
Etienne Samson 8be2a790 2017-12-05T23:21:05 openssl: free the peer certificate Per SSL_get_peer_certificate docs: ``` The reference count of the X509 object is incremented by one, so that it will not be destroyed when the session containing the peer certificate is freed. The X509 object must be explicitly freed using X509_free(). ```
Etienne Samson 2518eb81 2017-11-24T14:04:10 openssl: merge all the exit paths of verify_server_cert This makes it easier to cleanup allocated resources on exit.
Etienne Samson 1b2e83a9 2017-12-13T00:19:41 stransport: provide error message on trust failures Fixes #4440
Patrick Steinhardt 2d2e70f8 2017-11-30T18:10:28 openssl: fix thread-safety on non-glibc POSIX systems While the OpenSSL library provides all means to work safely in a multi-threaded application, we fail to do so correctly. Quoting from crypto_lock(3): OpenSSL can safely be used in multi-threaded applications provided that at least two callback functions are set, locking_function and threadid_func. We do in fact provide the means to set up the locking function via `git_openssl_set_locking()`, where we initialize a set of locks by using the POSIX threads API and set the correct callback function to lock and unlock them. But what we do not do is setting the `threadid_func` callback. This function is being used to correctly locate thread-local data of the OpenSSL library and should thus return per-thread identifiers. Digging deeper into OpenSSL's documentation, the library does provide a fallback in case that locking function is not provided by the user. On Windows and BeOS we should be safe, as it simply "uses the system's default thread identifying API". On other platforms though OpenSSL will fall back to using the address of `errno`, assuming it is thread-local. While this assumption holds true for glibc-based systems, POSIX in fact does not specify whether it is thread-local or not. Quoting from errno(3p): It is unspecified whether errno is a macro or an identifier declared with external linkage. And in fact, with musl there is at least one libc implementation which simply declares `errno` as a simple `int` without being thread-local. On those systems, the fallback threadid function of OpenSSL will not be thread-safe. Fix this by setting up our own callback for this setting. As users of libgit2 may want to set it themselves, we obviously cannot always set that function on initialization. But as we already set up primitives for threading in `git_openssl_set_locking()`, this function becomes the obvious choice where to implement the additional setup.
Etienne Samson 22317057 2017-03-21T00:36:32 https: Prevent OpenSSL from namespace-leaking
Etienne Samson e9369856 2017-03-21T00:25:15 stream: Gather streams to src/streams