|
5c6180b5
|
2019-11-29T11:06:11
|
|
global: convert to fiber-local storage to fix exit races
On Windows platforms, we automatically clean up the thread-local storage
upon detaching a thread via `DllMain()`. The thing is that this happens
for every thread of applications that link against the libgit2 DLL, even
those that don't have anything to do with libgit2 itself. As a result,
we cannot assume that these unsuspecting threads make use of our
`git_libgit2_init()` and `git_libgit2_shutdow()` reference counting,
which may lead to racy situations:
Thread 1 Thread 2
git_libgit2_shutdown()
DllMain(DETACH_THREAD)
git__free_tls_data()
git_atomic_dec() == 0
git__free_tls_data()
TlsFree(_tls_index)
TlsGetValue(_tls_index)
Due to the second thread never having executed `git_libgit2_init()`, the
first thread will clean up TLS data and as a result also free the
`_tls_index` variable. When detaching the second thread, we
unconditionally access the now-free'd `_tls_index` variable, which is
obviously not going to work out well.
Fix the issue by converting the code to use fiber-local storage instead
of thread-local storage. While FLS will behave the exact same as TLS if
no fibers are in use, it does allow us to specify a destructor similar
to the one that is accepted by pthread_key_create(3P). Like this, we do
not have to manually free indices anymore, but will let the FLS handle
calling the destructor. This allows us to get rid of `DllMain()`
completely, as we only used it to keep track of when threads were
exiting and results in an overall simplification of TLS cleanup.
|
|
|
b46c3594
|
2019-01-02T09:33:55
|
|
global: move init callbacks into an array
We currently have an explicit callchain of all the initialization
callbacks in our `init_common` function. This is perfectly fine, but
requires us to manually keep track of how many shutdown callbacks there
may be installed: to avoid allocations before libgit2 is fully
initialized, we assume that every initializer may register at most one
shutdown function. These shutdown functions are stored in a static array
of size `MAX_SHUTDOWN_CB`, which then needs to be updated manually
whenever a new initializer function is being added.
The situation can be easily fixed: convert the callchain of init
functions into an array and iterate over it to initialize all
subsystems. This allows us to define the `git__shutdown_callbacks` array
with the same size as the initializer array and rids us of the need to
always update `MAX_SHUTDOWN_CB`.
|
|
df2cc108
|
2018-11-18T10:29:07
|
|
stream: provide generic registration API
Update the new stream registration API to be `git_stream_register`
which takes a registration structure and a TLS boolean. This allows
callers to register non-TLS streams as well as TLS streams.
Provide `git_stream_register_tls` that takes just the init callback for
backward compatibliity.
|
|
|
21142c5a
|
2018-10-29T10:04:48
|
|
http: remove cURL
We previously used cURL to support HTTP proxies. Now that we've added
this support natively, we can remove the curl dependency.
|
|
|
43b592ac
|
2018-10-25T08:49:01
|
|
tls: introduce a wrap function
Introduce `git_tls_stream_wrap` which will take an existing `stream`
with an already connected socket and begin speaking TLS on top of it.
This is useful if you've built a connection to a proxy server and you
wish to begin CONNECT over it to tunnel a TLS connection.
Also update the pluggable TLS stream layer so that it can accept a
registration structure that provides an `init` and `wrap` function,
instead of a single initialization function.
|
|
|
9865cd16
|
2018-03-20T14:23:49
|
|
alloc: make memory allocators use function pointers
Currently, our memory allocators are being redirected to the correct
implementation at compile time by simply using macros. In order to make
them swappable at runtime, this commit reshuffles that by instead making
use of a global "git_allocator" structure, whose pointers are set up to
reference the allocator functions. Like this, it becomes easy to swap
out allocators by simply setting these function pointers.
In order to initialize a "git_allocator", our provided allocators
"stdalloc" and "crtdbg" both provide an init function. This is being
called to initialize a passed in allocator struct and set up its members
correctly.
No support is yet included to enable users of libgit2 to switch out the
memory allocator at a global level.
|
|
|
81c9894f
|
2018-05-09T14:06:57
|
|
Merge pull request #4645 from pks-t/pks/racy-init-deinit
global: adjust init count under lock
|
|
|
0933fdc5
|
2018-05-04T13:40:54
|
|
global: adjust init count under lock
Our global initialization functions `git_libgit2_init()` and
`git_libgit2_shutdown()` both adjust a global init counter to determine
whether we are the first respectively last user of libgit2. On
Unix-systems do not do so under lock, though, which opens the
possibility of a race between these two functions:
Thread 1 Thread 2
git__n_inits = 0;
git_libgit2_init();
git_atomic_inc(&git__n_inits);
/* git__n_inits == 1 */
git_libgit2_shutdown();
if (git_atomic_dec(&git__n_inits) != 0)
/* git__n_inits == 0, no early exit here */
pthread_mutex_lock(&_init_mutex);
shutdown_common();
pthread_mutex_unlock(&_init_mutex);
pthread_mutex_lock(&_init_mutex);
init_once();
pthread_mutex_unlock(&_init_mutex);
So we can end up in a situation where we try to shutdown shared data
structures before they have been initialized.
Fix the race by always locking `_init_mutex` before incrementing or
decrementing `git__n_inits`.
|
|
60e1ad92
|
2018-03-29T22:14:01
|
|
mbedtls: add global initialization
|
|
|
2022b004
|
2018-02-28T12:06:59
|
|
curl: explicitly initialize and cleanup global curl state
Our curl-based streams make use of the easy curl interface. This
interface automatically initializes and de-initializes the global curl
state by calling out to `curl_global_init` and `curl_global_cleanup`.
Thus, all global state will be repeatedly re-initialized when creating
multiple curl streams in succession. Despite being inefficient, this is
not thread-safe due to `curl_global_init` being not thread-safe itself.
Thus a multi-threaded programing handling multiple curl streams at the
same time is inherently racy.
Fix the issue by globally initializing and cleaning up curl's state.
|
|
|
e9369856
|
2017-03-21T00:25:15
|
|
stream: Gather streams to src/streams
|
|
|
0c7f49dd
|
2017-06-30T13:39:01
|
|
Make sure to always include "common.h" first
Next to including several files, our "common.h" header also declares
various macros which are then used throughout the project. As such, we
have to make sure to always include this file first in all
implementation files. Otherwise, we might encounter problems or even
silent behavioural differences due to macros or defines not being
defined as they should be. So in fact, our header and implementation
files should make sure to always include "common.h" first.
This commit does so by establishing a common include pattern. Header
files inside of "src" will now always include "common.h" as its first
other file, separated by a newline from all the other includes to make
it stand out as special. There are two cases for the implementation
files. If they do have a matching header file, they will always include
this one first, leading to "common.h" being transitively included as
first file. If they do not have a matching header file, they instead
include "common.h" as first file themselves.
This fixes the outlined problems and will become our standard practice
for header and source files inside of the "src/" from now on.
|
|
|
2ce2a48f
|
2017-05-02T13:37:15
|
|
transports: ssh: clean up after libssh2 on exit
After calling `libssh2_init`, we need to clean up after the library by
executing `libssh2_exit` as soon as we exit. Register a shutdown handler
to do so which simply calls `libssh2_exit`. This fixes several memory
leaks.
|
|
|
038f0e1b
|
2016-11-02T08:49:24
|
|
global: reset global state on shutdown without threading
When threading is not enabled for libgit2, we keep global state
in a simple static variable. When libgit2 is shut down, we clean
up the global state by freeing the global state's dynamically
allocated memory. When libgit2 is built with threading, we
additionally free the thread-local storage and thus completely
remove the global state. In a non-threaded build, though, we
simply leave the global state as-is, which may result in an error
upon reinitializing libgit2.
Fix the issue by zeroing out the variable on a shutdown, thus
returning it to its initial state.
|
|
|
59c6c286
|
2016-10-27T12:31:17
|
|
global: synchronize initialization and shutdown with pthreads
When trying to initialize and tear down global data structures
from different threads at once with `git_libgit2_init` and
`git_libgit2_shutdown`, we race around initializing data. While
we use `pthread_once` to assert that we only initilize data a
single time, we actually reset the `pthread_once_t` on the last
call to `git_libgit2_shutdown`. As resetting this variable is not
synchronized with other threads trying to access it, this is
actually racy when one thread tries to do a complete shutdown of
libgit2 while another thread tries to initialize it.
Fix the issue by creating a mutex which synchronizes `init_once`
and the library shutdown.
|
|
2381d9e4
|
2016-08-03T17:01:48
|
|
mwindow: init mwindow files in git_libgit2_init
|
|
|
aab266c9
|
2016-06-20T20:07:33
|
|
threads: add platform-independent thread initialization function
|
|
|
432af52b
|
2016-06-07T12:55:17
|
|
global: clean up crt only after freeing tls data
The thread local storage is used to hold some global state that
is dynamically allocated and should be freed upon exit. On
Windows, we clean up the C run-time right after execution of
registered shutdown callbacks and before cleaning up the TLS.
When we clean up the CRT, we also cause it to analyze for memory
leaks. As we did not free the TLS yet this will lead to false
positives.
Fix the issue by first freeing the TLS and cleaning up the CRT
only afterwards.
|
|
|
14cf05da
|
2016-05-26T12:52:29
|
|
win32: clean up unused warnings in DllMain
|
|
a177756b
|
2016-03-18T13:00:27
|
|
win32: free thread-local data on thread exit
|
|
|
ec5a43b6
|
2016-03-18T06:37:41
|
|
Merge pull request #3699 from libgit2/cmn/win32-free-tls
win32: free thread-local data on thread exit
|
|
|
967e073d
|
2016-02-27T16:42:02
|
|
merge driver: correct global initialization
|
|
fa72d6da
|
2016-03-14T12:02:00
|
|
Setup better defaults for OpenSSL ciphers
This ensures that when using OpenSSL a safe default set of ciphers
is selected. This is done so that the client communicates securely
and we don't accidentally enable unsafe ciphers like RC4, or even
worse some old export ciphers.
Implements the first part of https://github.com/libgit2/libgit2/issues/3682
|
|
|
22f3d3aa
|
2016-03-03T22:26:31
|
|
ssh: initialize libssh2
We should have been doing this, but it initializes itself upon first
use, which works as long as nobody's doing concurrent network
operations. Initialize it on our init to make sure it's not getting
initialized concurrently.
|
|
|
bf127eec
|
2016-02-19T13:24:41
|
|
global: remove an unused variable
|
|
|
8a6d6677
|
2016-02-08T16:14:03
|
|
global: make openssl registration like the rest
|
|
|
bad2702c
|
2016-02-06T11:25:47
|
|
global: refactor setup and cleanup
Move the common initialization and cleanup methods to reduce
unnecessary duplication.
|
|
|
de870533
|
2015-10-02T03:43:11
|
|
settings: add a setter for a custom user-agent
|
|
|
f85fc367
|
2015-07-26T21:12:00
|
|
error: store the error messages in a reusable buffer
Instead of allocating a brand new buffer for each error string we want
to store, we can use a per-thread buffer to store the error string and
re-use the underlying storage. We already use the buffer to format the
string, so this mostly makes that more direct.
|
|
9830fbba
|
2015-07-22T11:33:18
|
|
Merge branch 'master' into fix-init-ordering
|
|
|
cf198fdf
|
2015-07-22T10:51:38
|
|
Increment `git__n_inits` before doing `init_once`.
Fixes #3318.
|
|
93b42728
|
2015-06-09T14:38:30
|
|
Include stacktrace summary in memory leak output.
|
|
20f8edb7
|
2015-06-04T02:22:10
|
|
global: Ensure we free our SSL context.
|
|
|
24e53d2f
|
2015-03-19T09:55:20
|
|
Rename GIT_SSL to GIT_OPENSSL
This is what it's meant all along, but now we actually have multiple
implementations, it's clearer to use the name of the library.
|
|
06c985d8
|
2015-04-18T09:07:48
|
|
Rename routine to free TLS data
|
|
|
d3fb7d93
|
2015-04-17T10:04:01
|
|
Remove DllMain now that TLS data freed by threads
|
|
|
f5ffb40e
|
2015-04-17T09:58:09
|
|
Also fix leak of TLS data on main thread.
|
|
|
55c5f756
|
2015-04-17T09:30:22
|
|
Attempt to fix Windows TLS memory leak.
|
|
8e851c1e
|
2015-03-03T16:41:59
|
|
libgit2_shutdown: free TLS data (win32)
Free TLS data on thread exit (win32)
|
|
|
83fe60fa
|
2015-03-03T14:10:50
|
|
libgit2_shutdown: clear err message on shutdown
Clear the error message on git_libgit2_shutdown for all versions of
the library (no threads and Win32 threads). Drop the giterr_clear
in clar, as that shouldn't be necessary.
|
|
3a8b69d1
|
2015-02-26T11:47:07
|
|
Fix leak of TLS error message in shutdown (ptherad version)
|
|
|
73f0278e
|
2014-12-23T16:40:01
|
|
global: include sys/openssl.h for GIT_EXPORT of fn
The openssl setup function needs to be GIT_EXPORT'ed, be sure
to include the `sys/openssl.h` header so that it is appropriately
decorated as an export function.
|
|
|
263b1d6e
|
2014-12-12T08:29:43
|
|
Make the OpenSSL locking function warnings more severe
Our git_openssl_set_locking() would ideally not exist. Make it clearer
that we provide it as a last resort and you should prefer anything else.
|
|
|
e79fbd9e
|
2014-12-05T07:09:08
|
|
Merge pull request #2743 from ethomson/init_val
init: return the number of initializations
|
|
|
6d91dc53
|
2014-12-03T15:28:44
|
|
init: return the number of initializations
|
|
2d2cd625
|
2014-12-03T21:01:42
|
|
Add missing else directive
Add missing else directive to fix compiler warning: control reaches
end of non-void function
|
|
d6ecc311
|
2014-11-15T14:35:58
|
|
Plug possible leak in the openssl locks
|
|
|
799e22ea
|
2014-10-23T17:34:41
|
|
Rename git_threads_ to git_libgit2_
This describes their purpose better, as we now initialize ssl and some
other global stuff in there. Calling the init function is not something
which has been optional for a while now.
|
|
|
7bb63991
|
2014-11-06T10:25:23
|
|
Merge pull request #2676 from libgit2/cmn/threading
Threading and crypto libraries
|
|
bc48bcdc
|
2014-11-03T14:23:13
|
|
Make the Visual Studio compiler happy
|
|
|
fe6b51ae
|
2014-11-01T10:45:33
|
|
ssl: separate locking init from general init
Extract the lock-setting functions into their own, as we cannot assume
that it's ok for us to set this unconditionally.
|
|
|
e0836577
|
2014-11-01T10:26:09
|
|
ssl: clear the OpenSSL locking function
We're freeing the memory which holds the locks so we must make sure that
the locking function doesn't try to use it.
|
|
|
50aae000
|
2014-10-25T19:52:11
|
|
global: clean up openssl_locks on shutdown
|
|
|
f0f97370
|
2014-10-18T15:52:10
|
|
ssl: dump the SSL ciphers in favour of TLS
All versions of SSL are considered deprecated now, so let's ask OpenSSl
to only use TLSv1. We still ask it to load those ciphers for
compatibility with servers which want to use an older hello but will use
TLS for encryption.
For good measure we also disable compression, which can be exploitable,
if the OpenSSL version supports it.
|
|
|
fdea219a
|
2014-09-10T18:28:19
|
|
global: free the error message when exiting a thread
When we free the global state at thread termination, we must also free
the error message in order not to leak the string once per thread.
|
|
|
f59a34d2
|
2014-07-12T14:45:34
|
|
Only create openssl_locks if thread support is enabled
|
|
|
e6b0ae7a
|
2014-06-30T09:19:05
|
|
ssl: init only once without threads
The OpenSSL library-loading functions do not expect to be called
multiple times. Add a flag in the non-threaded libgit2 init so we only
call once.
This fixes #2446.
|
|
e93206e0
|
2014-06-14T12:58:03
|
|
Merge pull request #2421 from libgit2/cmn/init-ssl-once
netops: init OpenSSL once under lock
|
|
|
081e76ba
|
2014-06-12T16:20:52
|
|
ssl: init everything all the time
Bring together all of the OpenSSL initialization to
git_threads_init() so it's together and doesn't need locks.
Moving it here also gives us libssh2 thread safety (when built against
openssl).
|
|
|
8f897b6f
|
2014-06-12T14:50:08
|
|
ssl: init also without threads
|
|
|
cf15ac8a
|
2014-06-12T03:20:34
|
|
ssl: cargo-cult thread safety
OpenSSL's tests init everything in the main thread, so let's do that.
|
|
|
1d3364ac
|
2014-06-11T20:52:15
|
|
netops: init OpenSSL once under lock
The OpenSSL init functions are not reentrant, which means that running
multiple fetches in parallel can cause us to crash.
Use a mutex to init OpenSSL, and since we're adding this extra checks,
init it only once.
|
|
fb591767
|
2014-06-07T12:51:48
|
|
Win32: Fix object::cache::threadmania test on x64
|
|
|
138af337
|
2014-05-19T12:20:31
|
|
Merge pull request #2303 from jacquesg/mingw-lseek
WIP: Windows fixes
|
|
0bf5430d
|
2014-05-06T13:33:47
|
|
Fix the issues in git_shutdown
1) Call to git_shutdown results in setting git__n_shutdown_callbacks
to -1. Next call to git__on_shutdown results in ABW (Array Bound Write)
for array git__shutdown_callbacks. In the current Implementation,
git_atomic_dec is called git__n_shutdown_callbacks + 1 times. I have
modified it to a for loop so that it is more readable. It would not
set git__n_shutdown_callbacks to a negative number and reset the
elements of git__shutdown_callbacks to NULL.
2) In function git_sysdir_get, shutdown function is registered only if
git_sysdir__dirs_shutdown_set is set to 0. However, after this variable
is set to 1, it is never reset to 0. If git_sysdir_global_init is
called again from synchronized_threads_init it does not register
shutdown function for this subsystem.
|
|
|
001befcd
|
2014-05-06T12:16:24
|
|
Fix the issues in git__on_shutdown
|
|
|
6e94a1ef
|
2014-04-27T14:25:49
|
|
_InterlockedExchange expects a volatile LONG
|
|
3816debc
|
2014-03-14T14:51:04
|
|
Fix threading tests when threads disabled
|
|
|
efaa342c
|
2014-04-11T22:07:49
|
|
Correct C90 warnings
|
|
|
83634d38
|
2014-02-24T17:43:10
|
|
Move system directory cache out of utils
|
|
22661448
|
2013-10-04T19:35:32
|
|
Don't use git_atomic as an integer
|
|
cdc95a0d
|
2013-10-04T18:38:37
|
|
Use InterlockedCompareExchange for the lock
|
|
|
e411b74e
|
2013-10-04T19:33:48
|
|
Posix synchronized init, prototype win32 version
|
|
|
a3aa5f4d
|
2013-09-11T12:45:20
|
|
Add simple global shutdown hooks
Increasingly there are a number of components that want to do some
cleanup at global shutdown time (at least if there are not going
to be memory leaks). This creates a very simple system of shutdown
hooks that will be invoked by git_threads_shutdown. Right now, the
maximum number of hooks is hardcoded, but since adding a hook is
not a public API, it should be fine and I thought it was better to
start off with really simple code.
|
|
|
43095341
|
2013-08-26T14:56:31
|
|
Load SRWLock APIs at runtime
This loads SRWLock APIs at runtime and in their absence (i.e. on
Windows before Vista) falls back on a regular CRITICAL_SECTION
that will not permit concurrent readers.
|
|
|
0a1c8f55
|
2013-07-11T17:09:15
|
|
preload configuration paths
|
|
|
1a42dd17
|
2013-05-31T14:13:11
|
|
Mutex init can fail
It is obviously quite a serious problem if this happens, but mutex
initialization can fail and we should detect it. It's a bit like
a memory allocation failure, in that you're probably pretty screwed
if this occurs, but at least we'll catch it.
|
|
|
53607868
|
2013-04-15T00:09:03
|
|
Further threading fixes
This builds on the earlier thread safety work to make it so that
setting the odb, index, refdb, or config for a repository is done
in a threadsafe manner with minimized locking time. This is done
by adding a lock to the repository object and using it to guard
the assignment of the above listed pointers. The lock is only
held to assign the pointer value.
This also contains some minor fixes to the other work with pack
files to reduce the time that locks are being held to and fix an
apparently memory leak.
|
|
|
41954a49
|
2013-03-18T14:19:35
|
|
Switch search paths to classic delimited strings
This switches the APIs for setting and getting the global/system
search paths from using git_strarray to using a simple string with
GIT_PATH_LIST_SEPARATOR delimited paths, just as the environment
PATH variable would contain. This makes it simpler to get and set
the value.
I also added code to expand "$PATH" when setting a new value to
embed the old value of the path. This means that I no longer
require separate actions to PREPEND to the value.
|
|
|
359fc2d2
|
2013-01-08T17:07:25
|
|
update copyrights
|
|
c3320aca
|
2012-12-09T02:22:50
|
|
git__mwindow_mutex needs to be initialized even with pthreads
This could also use PTHREAD_MUTEX_INITIALIZER, but a dynamic initializer seems like a more portable concept, and we won't need another #define on top of git_mutex_init()
|
|
|
a8527429
|
2012-11-13T14:48:10
|
|
unload dll / destroy hash ctxs at shutdown
|
|
|
7ebefd22
|
2012-11-13T10:10:40
|
|
move hash library func ptrs to global global
|
|
93b5fabc
|
2012-10-01T17:59:04
|
|
threads: Assert that the global state is initialized
|
|
8cef828d
|
2012-08-18T22:11:49
|
|
Make the memory-window conrol structures global
Up to now, the idea was that the user would do all the operations for
one repository in the same thread. Thus we could have the
memory-mapped window information thread-local and avoid any locking.
This is not practical in a few environments, such as Apple's GCD which
allocates threads arbitrarily or the .NET CLR, where the OS-level
thread can change at any moment.
Make the control structure global and protect it with a mutex so we
don't depend on the thread currently executing the code.
|
|
|
2bc8fa02
|
2012-04-17T10:14:24
|
|
Implement git_pool paged memory allocator
This adds a `git_pool` object that can do simple paged memory
allocation with free for the entire pool at once. Using this,
you can replace many small allocations with large blocks that
can then cheaply be doled out in small pieces. This is best
used when you plan to free the small blocks all at once - for
example, if they represent the parsed state from a file or data
stream that are either all kept or all discarded.
There are two real patterns of usage for `git_pools`: either
for "string" allocation, where the item size is a single byte
and you end up just packing the allocations in together, or for
"fixed size" allocation where you are allocating a large object
(e.g. a `git_oid`) and you generally just allocation single
objects that can be tightly packed. Of course, you can use it
for other things, but those two cases are the easiest.
|
|
5e0de328
|
2012-02-13T17:10:24
|
|
Update Copyright header
Signed-off-by: schu <schu-github@schulog.org>
|
|
|
a15c550d
|
2011-11-16T14:09:44
|
|
threads: Fix the shared global state with TLS
See `global.c` for a description of what we're doing.
When libgit2 is built with GIT_THREADS support, the threading system
must be explicitly initialized with `git_threads_init()`.
|