src/global.c


Log

Author Commit Date CI Message
Patrick Steinhardt 5c6180b5 2019-11-29T11:06:11 global: convert to fiber-local storage to fix exit races On Windows platforms, we automatically clean up the thread-local storage upon detaching a thread via `DllMain()`. The thing is that this happens for every thread of applications that link against the libgit2 DLL, even those that don't have anything to do with libgit2 itself. As a result, we cannot assume that these unsuspecting threads make use of our `git_libgit2_init()` and `git_libgit2_shutdow()` reference counting, which may lead to racy situations: Thread 1 Thread 2 git_libgit2_shutdown() DllMain(DETACH_THREAD) git__free_tls_data() git_atomic_dec() == 0 git__free_tls_data() TlsFree(_tls_index) TlsGetValue(_tls_index) Due to the second thread never having executed `git_libgit2_init()`, the first thread will clean up TLS data and as a result also free the `_tls_index` variable. When detaching the second thread, we unconditionally access the now-free'd `_tls_index` variable, which is obviously not going to work out well. Fix the issue by converting the code to use fiber-local storage instead of thread-local storage. While FLS will behave the exact same as TLS if no fibers are in use, it does allow us to specify a destructor similar to the one that is accepted by pthread_key_create(3P). Like this, we do not have to manually free indices anymore, but will let the FLS handle calling the destructor. This allows us to get rid of `DllMain()` completely, as we only used it to keep track of when threads were exiting and results in an overall simplification of TLS cleanup.
Patrick Steinhardt b46c3594 2019-01-02T09:33:55 global: move init callbacks into an array We currently have an explicit callchain of all the initialization callbacks in our `init_common` function. This is perfectly fine, but requires us to manually keep track of how many shutdown callbacks there may be installed: to avoid allocations before libgit2 is fully initialized, we assume that every initializer may register at most one shutdown function. These shutdown functions are stored in a static array of size `MAX_SHUTDOWN_CB`, which then needs to be updated manually whenever a new initializer function is being added. The situation can be easily fixed: convert the callchain of init functions into an array and iterate over it to initialize all subsystems. This allows us to define the `git__shutdown_callbacks` array with the same size as the initializer array and rids us of the need to always update `MAX_SHUTDOWN_CB`.
Edward Thomson df2cc108 2018-11-18T10:29:07 stream: provide generic registration API Update the new stream registration API to be `git_stream_register` which takes a registration structure and a TLS boolean. This allows callers to register non-TLS streams as well as TLS streams. Provide `git_stream_register_tls` that takes just the init callback for backward compatibliity.
Edward Thomson 21142c5a 2018-10-29T10:04:48 http: remove cURL We previously used cURL to support HTTP proxies. Now that we've added this support natively, we can remove the curl dependency.
Edward Thomson 43b592ac 2018-10-25T08:49:01 tls: introduce a wrap function Introduce `git_tls_stream_wrap` which will take an existing `stream` with an already connected socket and begin speaking TLS on top of it. This is useful if you've built a connection to a proxy server and you wish to begin CONNECT over it to tunnel a TLS connection. Also update the pluggable TLS stream layer so that it can accept a registration structure that provides an `init` and `wrap` function, instead of a single initialization function.
Patrick Steinhardt 9865cd16 2018-03-20T14:23:49 alloc: make memory allocators use function pointers Currently, our memory allocators are being redirected to the correct implementation at compile time by simply using macros. In order to make them swappable at runtime, this commit reshuffles that by instead making use of a global "git_allocator" structure, whose pointers are set up to reference the allocator functions. Like this, it becomes easy to swap out allocators by simply setting these function pointers. In order to initialize a "git_allocator", our provided allocators "stdalloc" and "crtdbg" both provide an init function. This is being called to initialize a passed in allocator struct and set up its members correctly. No support is yet included to enable users of libgit2 to switch out the memory allocator at a global level.
Patrick Steinhardt 81c9894f 2018-05-09T14:06:57 Merge pull request #4645 from pks-t/pks/racy-init-deinit global: adjust init count under lock
Patrick Steinhardt 0933fdc5 2018-05-04T13:40:54 global: adjust init count under lock Our global initialization functions `git_libgit2_init()` and `git_libgit2_shutdown()` both adjust a global init counter to determine whether we are the first respectively last user of libgit2. On Unix-systems do not do so under lock, though, which opens the possibility of a race between these two functions: Thread 1 Thread 2 git__n_inits = 0; git_libgit2_init(); git_atomic_inc(&git__n_inits); /* git__n_inits == 1 */ git_libgit2_shutdown(); if (git_atomic_dec(&git__n_inits) != 0) /* git__n_inits == 0, no early exit here */ pthread_mutex_lock(&_init_mutex); shutdown_common(); pthread_mutex_unlock(&_init_mutex); pthread_mutex_lock(&_init_mutex); init_once(); pthread_mutex_unlock(&_init_mutex); So we can end up in a situation where we try to shutdown shared data structures before they have been initialized. Fix the race by always locking `_init_mutex` before incrementing or decrementing `git__n_inits`.
Etienne Samson 60e1ad92 2018-03-29T22:14:01 mbedtls: add global initialization
Patrick Steinhardt 2022b004 2018-02-28T12:06:59 curl: explicitly initialize and cleanup global curl state Our curl-based streams make use of the easy curl interface. This interface automatically initializes and de-initializes the global curl state by calling out to `curl_global_init` and `curl_global_cleanup`. Thus, all global state will be repeatedly re-initialized when creating multiple curl streams in succession. Despite being inefficient, this is not thread-safe due to `curl_global_init` being not thread-safe itself. Thus a multi-threaded programing handling multiple curl streams at the same time is inherently racy. Fix the issue by globally initializing and cleaning up curl's state.
Etienne Samson e9369856 2017-03-21T00:25:15 stream: Gather streams to src/streams
Patrick Steinhardt 0c7f49dd 2017-06-30T13:39:01 Make sure to always include "common.h" first Next to including several files, our "common.h" header also declares various macros which are then used throughout the project. As such, we have to make sure to always include this file first in all implementation files. Otherwise, we might encounter problems or even silent behavioural differences due to macros or defines not being defined as they should be. So in fact, our header and implementation files should make sure to always include "common.h" first. This commit does so by establishing a common include pattern. Header files inside of "src" will now always include "common.h" as its first other file, separated by a newline from all the other includes to make it stand out as special. There are two cases for the implementation files. If they do have a matching header file, they will always include this one first, leading to "common.h" being transitively included as first file. If they do not have a matching header file, they instead include "common.h" as first file themselves. This fixes the outlined problems and will become our standard practice for header and source files inside of the "src/" from now on.
Patrick Steinhardt 2ce2a48f 2017-05-02T13:37:15 transports: ssh: clean up after libssh2 on exit After calling `libssh2_init`, we need to clean up after the library by executing `libssh2_exit` as soon as we exit. Register a shutdown handler to do so which simply calls `libssh2_exit`. This fixes several memory leaks.
Patrick Steinhardt 038f0e1b 2016-11-02T08:49:24 global: reset global state on shutdown without threading When threading is not enabled for libgit2, we keep global state in a simple static variable. When libgit2 is shut down, we clean up the global state by freeing the global state's dynamically allocated memory. When libgit2 is built with threading, we additionally free the thread-local storage and thus completely remove the global state. In a non-threaded build, though, we simply leave the global state as-is, which may result in an error upon reinitializing libgit2. Fix the issue by zeroing out the variable on a shutdown, thus returning it to its initial state.
Patrick Steinhardt 59c6c286 2016-10-27T12:31:17 global: synchronize initialization and shutdown with pthreads When trying to initialize and tear down global data structures from different threads at once with `git_libgit2_init` and `git_libgit2_shutdown`, we race around initializing data. While we use `pthread_once` to assert that we only initilize data a single time, we actually reset the `pthread_once_t` on the last call to `git_libgit2_shutdown`. As resetting this variable is not synchronized with other threads trying to access it, this is actually racy when one thread tries to do a complete shutdown of libgit2 while another thread tries to initialize it. Fix the issue by creating a mutex which synchronizes `init_once` and the library shutdown.
Edward Thomson 2381d9e4 2016-08-03T17:01:48 mwindow: init mwindow files in git_libgit2_init
Patrick Steinhardt aab266c9 2016-06-20T20:07:33 threads: add platform-independent thread initialization function
Patrick Steinhardt 432af52b 2016-06-07T12:55:17 global: clean up crt only after freeing tls data The thread local storage is used to hold some global state that is dynamically allocated and should be freed upon exit. On Windows, we clean up the C run-time right after execution of registered shutdown callbacks and before cleaning up the TLS. When we clean up the CRT, we also cause it to analyze for memory leaks. As we did not free the TLS yet this will lead to false positives. Fix the issue by first freeing the TLS and cleaning up the CRT only afterwards.
Edward Thomson 14cf05da 2016-05-26T12:52:29 win32: clean up unused warnings in DllMain
Carlos Martin Nieto a177756b 2016-03-18T13:00:27 win32: free thread-local data on thread exit
Edward Thomson ec5a43b6 2016-03-18T06:37:41 Merge pull request #3699 from libgit2/cmn/win32-free-tls win32: free thread-local data on thread exit
Edward Thomson 967e073d 2016-02-27T16:42:02 merge driver: correct global initialization
Dirkjan Bussink fa72d6da 2016-03-14T12:02:00 Setup better defaults for OpenSSL ciphers This ensures that when using OpenSSL a safe default set of ciphers is selected. This is done so that the client communicates securely and we don't accidentally enable unsafe ciphers like RC4, or even worse some old export ciphers. Implements the first part of https://github.com/libgit2/libgit2/issues/3682
Carlos Martín Nieto 22f3d3aa 2016-03-03T22:26:31 ssh: initialize libssh2 We should have been doing this, but it initializes itself upon first use, which works as long as nobody's doing concurrent network operations. Initialize it on our init to make sure it's not getting initialized concurrently.
Carlos Martín Nieto bf127eec 2016-02-19T13:24:41 global: remove an unused variable
Edward Thomson 8a6d6677 2016-02-08T16:14:03 global: make openssl registration like the rest
Edward Thomson bad2702c 2016-02-06T11:25:47 global: refactor setup and cleanup Move the common initialization and cleanup methods to reduce unnecessary duplication.
Carlos Martín Nieto de870533 2015-10-02T03:43:11 settings: add a setter for a custom user-agent
Carlos Martín Nieto f85fc367 2015-07-26T21:12:00 error: store the error messages in a reusable buffer Instead of allocating a brand new buffer for each error string we want to store, we can use a per-thread buffer to store the error string and re-use the underlying storage. We already use the buffer to format the string, so this mostly makes that more direct.
joshaber 9830fbba 2015-07-22T11:33:18 Merge branch 'master' into fix-init-ordering
joshaber cf198fdf 2015-07-22T10:51:38 Increment `git__n_inits` before doing `init_once`. Fixes #3318.
Jeff Hostetler 93b42728 2015-06-09T14:38:30 Include stacktrace summary in memory leak output.
Tim Hentenaar 20f8edb7 2015-06-04T02:22:10 global: Ensure we free our SSL context.
Carlos Martín Nieto 24e53d2f 2015-03-19T09:55:20 Rename GIT_SSL to GIT_OPENSSL This is what it's meant all along, but now we actually have multiple implementations, it's clearer to use the name of the library.
Jeff Hostetler 06c985d8 2015-04-18T09:07:48 Rename routine to free TLS data
Jeff Hostetler d3fb7d93 2015-04-17T10:04:01 Remove DllMain now that TLS data freed by threads
Jeff Hostetler f5ffb40e 2015-04-17T09:58:09 Also fix leak of TLS data on main thread.
Jeff Hostetler 55c5f756 2015-04-17T09:30:22 Attempt to fix Windows TLS memory leak.
Edward Thomson 8e851c1e 2015-03-03T16:41:59 libgit2_shutdown: free TLS data (win32) Free TLS data on thread exit (win32)
Edward Thomson 83fe60fa 2015-03-03T14:10:50 libgit2_shutdown: clear err message on shutdown Clear the error message on git_libgit2_shutdown for all versions of the library (no threads and Win32 threads). Drop the giterr_clear in clar, as that shouldn't be necessary.
Leo Yang 3a8b69d1 2015-02-26T11:47:07 Fix leak of TLS error message in shutdown (ptherad version)
Edward Thomson 73f0278e 2014-12-23T16:40:01 global: include sys/openssl.h for GIT_EXPORT of fn The openssl setup function needs to be GIT_EXPORT'ed, be sure to include the `sys/openssl.h` header so that it is appropriately decorated as an export function.
Carlos Martín Nieto 263b1d6e 2014-12-12T08:29:43 Make the OpenSSL locking function warnings more severe Our git_openssl_set_locking() would ideally not exist. Make it clearer that we provide it as a last resort and you should prefer anything else.
Carlos Martín Nieto e79fbd9e 2014-12-05T07:09:08 Merge pull request #2743 from ethomson/init_val init: return the number of initializations
Edward Thomson 6d91dc53 2014-12-03T15:28:44 init: return the number of initializations
Stefan Widgren 2d2cd625 2014-12-03T21:01:42 Add missing else directive Add missing else directive to fix compiler warning: control reaches end of non-void function
Ungureanu Marius d6ecc311 2014-11-15T14:35:58 Plug possible leak in the openssl locks
Carlos Martín Nieto 799e22ea 2014-10-23T17:34:41 Rename git_threads_ to git_libgit2_ This describes their purpose better, as we now initialize ssl and some other global stuff in there. Calling the init function is not something which has been optional for a while now.
Edward Thomson 7bb63991 2014-11-06T10:25:23 Merge pull request #2676 from libgit2/cmn/threading Threading and crypto libraries
Jacques Germishuys bc48bcdc 2014-11-03T14:23:13 Make the Visual Studio compiler happy
Carlos Martín Nieto fe6b51ae 2014-11-01T10:45:33 ssl: separate locking init from general init Extract the lock-setting functions into their own, as we cannot assume that it's ok for us to set this unconditionally.
Carlos Martín Nieto e0836577 2014-11-01T10:26:09 ssl: clear the OpenSSL locking function We're freeing the memory which holds the locks so we must make sure that the locking function doesn't try to use it.
Edward Thomson 50aae000 2014-10-25T19:52:11 global: clean up openssl_locks on shutdown
Carlos Martín Nieto f0f97370 2014-10-18T15:52:10 ssl: dump the SSL ciphers in favour of TLS All versions of SSL are considered deprecated now, so let's ask OpenSSl to only use TLSv1. We still ask it to load those ciphers for compatibility with servers which want to use an older hello but will use TLS for encryption. For good measure we also disable compression, which can be exploitable, if the OpenSSL version supports it.
Carlos Martín Nieto fdea219a 2014-09-10T18:28:19 global: free the error message when exiting a thread When we free the global state at thread termination, we must also free the error message in order not to leak the string once per thread.
Jacques Germishuys f59a34d2 2014-07-12T14:45:34 Only create openssl_locks if thread support is enabled
Carlos Martín Nieto e6b0ae7a 2014-06-30T09:19:05 ssl: init only once without threads The OpenSSL library-loading functions do not expect to be called multiple times. Add a flag in the non-threaded libgit2 init so we only call once. This fixes #2446.
Vicent Marti e93206e0 2014-06-14T12:58:03 Merge pull request #2421 from libgit2/cmn/init-ssl-once netops: init OpenSSL once under lock
Carlos Martín Nieto 081e76ba 2014-06-12T16:20:52 ssl: init everything all the time Bring together all of the OpenSSL initialization to git_threads_init() so it's together and doesn't need locks. Moving it here also gives us libssh2 thread safety (when built against openssl).
Carlos Martín Nieto 8f897b6f 2014-06-12T14:50:08 ssl: init also without threads
Carlos Martín Nieto cf15ac8a 2014-06-12T03:20:34 ssl: cargo-cult thread safety OpenSSL's tests init everything in the main thread, so let's do that.
Carlos Martín Nieto 1d3364ac 2014-06-11T20:52:15 netops: init OpenSSL once under lock The OpenSSL init functions are not reentrant, which means that running multiple fetches in parallel can cause us to crash. Use a mutex to init OpenSSL, and since we're adding this extra checks, init it only once.
Philip Kelley fb591767 2014-06-07T12:51:48 Win32: Fix object::cache::threadmania test on x64
Vicent Marti 138af337 2014-05-19T12:20:31 Merge pull request #2303 from jacquesg/mingw-lseek WIP: Windows fixes
Anurag Gupta 0bf5430d 2014-05-06T13:33:47 Fix the issues in git_shutdown 1) Call to git_shutdown results in setting git__n_shutdown_callbacks to -1. Next call to git__on_shutdown results in ABW (Array Bound Write) for array git__shutdown_callbacks. In the current Implementation, git_atomic_dec is called git__n_shutdown_callbacks + 1 times. I have modified it to a for loop so that it is more readable. It would not set git__n_shutdown_callbacks to a negative number and reset the elements of git__shutdown_callbacks to NULL. 2) In function git_sysdir_get, shutdown function is registered only if git_sysdir__dirs_shutdown_set is set to 0. However, after this variable is set to 1, it is never reset to 0. If git_sysdir_global_init is called again from synchronized_threads_init it does not register shutdown function for this subsystem.
Anurag Gupta 001befcd 2014-05-06T12:16:24 Fix the issues in git__on_shutdown
Jacques Germishuys 6e94a1ef 2014-04-27T14:25:49 _InterlockedExchange expects a volatile LONG
Russell Belfer 3816debc 2014-03-14T14:51:04 Fix threading tests when threads disabled
Jacques Germishuys efaa342c 2014-04-11T22:07:49 Correct C90 warnings
Edward Thomson 83634d38 2014-02-24T17:43:10 Move system directory cache out of utils
Ben Straub 22661448 2013-10-04T19:35:32 Don't use git_atomic as an integer
Philip Kelley cdc95a0d 2013-10-04T18:38:37 Use InterlockedCompareExchange for the lock
Ben Straub e411b74e 2013-10-04T19:33:48 Posix synchronized init, prototype win32 version
Russell Belfer a3aa5f4d 2013-09-11T12:45:20 Add simple global shutdown hooks Increasingly there are a number of components that want to do some cleanup at global shutdown time (at least if there are not going to be memory leaks). This creates a very simple system of shutdown hooks that will be invoked by git_threads_shutdown. Right now, the maximum number of hooks is hardcoded, but since adding a hook is not a public API, it should be fine and I thought it was better to start off with really simple code.
Russell Belfer 43095341 2013-08-26T14:56:31 Load SRWLock APIs at runtime This loads SRWLock APIs at runtime and in their absence (i.e. on Windows before Vista) falls back on a regular CRITICAL_SECTION that will not permit concurrent readers.
Edward Thomson 0a1c8f55 2013-07-11T17:09:15 preload configuration paths
Russell Belfer 1a42dd17 2013-05-31T14:13:11 Mutex init can fail It is obviously quite a serious problem if this happens, but mutex initialization can fail and we should detect it. It's a bit like a memory allocation failure, in that you're probably pretty screwed if this occurs, but at least we'll catch it.
Russell Belfer 53607868 2013-04-15T00:09:03 Further threading fixes This builds on the earlier thread safety work to make it so that setting the odb, index, refdb, or config for a repository is done in a threadsafe manner with minimized locking time. This is done by adding a lock to the repository object and using it to guard the assignment of the above listed pointers. The lock is only held to assign the pointer value. This also contains some minor fixes to the other work with pack files to reduce the time that locks are being held to and fix an apparently memory leak.
Russell Belfer 41954a49 2013-03-18T14:19:35 Switch search paths to classic delimited strings This switches the APIs for setting and getting the global/system search paths from using git_strarray to using a simple string with GIT_PATH_LIST_SEPARATOR delimited paths, just as the environment PATH variable would contain. This makes it simpler to get and set the value. I also added code to expand "$PATH" when setting a new value to embed the old value of the path. This means that I no longer require separate actions to PREPEND to the value.
Edward Thomson 359fc2d2 2013-01-08T17:07:25 update copyrights
Justin Spahr-Summers c3320aca 2012-12-09T02:22:50 git__mwindow_mutex needs to be initialized even with pthreads This could also use PTHREAD_MUTEX_INITIALIZER, but a dynamic initializer seems like a more portable concept, and we won't need another #define on top of git_mutex_init()
Edward Thomson a8527429 2012-11-13T14:48:10 unload dll / destroy hash ctxs at shutdown
Edward Thomson 7ebefd22 2012-11-13T10:10:40 move hash library func ptrs to global global
Vicent Marti 93b5fabc 2012-10-01T17:59:04 threads: Assert that the global state is initialized
Carlos Martín Nieto 8cef828d 2012-08-18T22:11:49 Make the memory-window conrol structures global Up to now, the idea was that the user would do all the operations for one repository in the same thread. Thus we could have the memory-mapped window information thread-local and avoid any locking. This is not practical in a few environments, such as Apple's GCD which allocates threads arbitrarily or the .NET CLR, where the OS-level thread can change at any moment. Make the control structure global and protect it with a mutex so we don't depend on the thread currently executing the code.
Russell Belfer 2bc8fa02 2012-04-17T10:14:24 Implement git_pool paged memory allocator This adds a `git_pool` object that can do simple paged memory allocation with free for the entire pool at once. Using this, you can replace many small allocations with large blocks that can then cheaply be doled out in small pieces. This is best used when you plan to free the small blocks all at once - for example, if they represent the parsed state from a file or data stream that are either all kept or all discarded. There are two real patterns of usage for `git_pools`: either for "string" allocation, where the item size is a single byte and you end up just packing the allocations in together, or for "fixed size" allocation where you are allocating a large object (e.g. a `git_oid`) and you generally just allocation single objects that can be tightly packed. Of course, you can use it for other things, but those two cases are the easiest.
schu 5e0de328 2012-02-13T17:10:24 Update Copyright header Signed-off-by: schu <schu-github@schulog.org>
Vicent Marti a15c550d 2011-11-16T14:09:44 threads: Fix the shared global state with TLS See `global.c` for a description of what we're doing. When libgit2 is built with GIT_THREADS support, the threading system must be explicitly initialized with `git_threads_init()`.