src/unix


Log

Author Commit Date CI Message
Edward Thomson fc42c28e 2021-11-19T09:36:03 util: don't include unnecessary system libraries Remove some unnecessary includes from utility code.
Dhruva Krishnamurthy 4ce8e01a 2020-06-17T14:31:11 Support build with NO_MMAP to disable use of system mmap * Use pread/pwrite to avoid updating position in file descriptor * Emulate missing pread/pwrite on win32 using overlapped file IO
lhchavez 322c15ee 2020-08-01T18:24:41 Make the pack and mwindow implementations data-race-free This change fixes a packfile heap corruption that can happen when interacting with multiple packfiles concurrently across multiple threads. This is exacerbated by setting a lower mwindow open file limit. This change: * Renames most of the internal methods in pack.c to clearly indicate that they expect to be called with a certain lock held, making reasoning about the state of locks a bit easier. * Splits the `git_pack_file` lock in two: the one in `git_pack_file` only protects the `index_map`. The protection to `git_mwindow_file` is now in that struct. * Explicitly checks for freshness of the `git_pack_file` in `git_packfile_unpack_header`: this allows the mwindow implementation to close files whenever there is enough cache pressure, and `git_packfile_unpack_header` will reopen the packfile if needed. * After a call to `p_munmap()`, the `data` and `len` fields are poisoned with `NULL` to make use-after-frees more evident and crash rather than being open to the possibility of heap corruption. * Adds a test case to prevent this from regressing in the future. Fixes: #5591
Edward Thomson c6ebdb29 2020-11-22T09:43:06 win32: use GIT_ASSERT
Edward Thomson fe12423a 2020-05-12T13:08:22 init: move thread init to git_global_threads_init Instead of treating win32 thread initialization specially in the win32 git_libgit2_init function, add a git_global_threads_init function.
kevinlul e8ab3db9 2020-07-07T22:29:05 p_chmod: Android compatibility Fix #5565 Pre-Android 5 did not implement a virtual filesystem atop FAT partitions for Unix permissions, which causes chmod to fail. However, Unix permissions have no effect on Android anyway as file permissions are not actually managed this way, so treating it as a no-op across all Android is safe.
Sven Strickroth ff46c5d3 2020-02-20T20:47:22 Fix typo on GIT_USE_NEC Signed-off-by: Sven Strickroth <email@cs-ware.de>
Edward Thomson 6460e8ab 2019-06-23T18:13:29 internal: use off64_t instead of git_off_t Prefer `off64_t` internally.
Patrick Steinhardt 1721ab04 2019-06-16T11:25:47 unix: posix: avoid use of variadic macro `p_snprintf` The macro `p_snprintf` is implemented as a variadic macro that calls `snprintf` directly with `__VA_ARGS__`. In C89, variadic macros are not allowed, but as the arguments of `p_snprintf` and `snprintf` are matching 1:1, we can fix this by simply removing the parameter list from `p_snprintf`.
Edward Thomson 02683b20 2019-01-12T23:06:39 regexec: prefix all regexec function calls with p_ Prefix all the calls to the the regexec family of functions with `p_`. This allows us to swap out all the regular expression functions with our own implementation. Move the declarations to `posix_regex.h` for simpler inclusion.
Edward Thomson f673e232 2018-12-27T13:47:34 git_error: use new names in internal APIs and usage Move to the `git_error` name in the internal API for error-related functions.
Edward Thomson abb04caa 2018-02-01T15:55:48 consistent header guards use consistent names for the #include / #define header guard pattern.
Patrick Steinhardt 26f531d3 2017-09-12T13:35:18 features.h: allow building without CMake-generated feature header In commit a390a8464 (cmake: move defines into "features.h" header, 2017-07-01), we have introduced a new "features.h" header. This file is being generated by the CMake build system based on how the libgit2 build has been configured, replacing the preexisting method of simply setting the defines inside of the CMake build system. This was done to help splitting up the build instructions into multiple separate subdirectories. An overlooked shortcoming of this approach is that some projects making use of libgit2 build the library with custom build systems, without making use of CMake. For those users, the introduction of the "features.h" file makes their life harder as they would have to also generate this file. Fix this issue by guarding all inclusions of the generated header file by the `LIBGIT2_NO_FEATURES_H` define. Like this, other build systems can skip the feature header and simply define all used features by specifying `-D` flags for the compiler again.
Patrick Steinhardt 8341d6cf 2017-07-04T10:57:28 cmake: move regcomp and futimens checks to "features.h" In our CMakeLists.txt, we have to check multiple functions in order to determine if we have to use our own or whether we can use the platform-provided one. For two of these functions, namely `regcomp_l()` and `futimens`, the defined macro is actually used inside of the header file "src/unix/posix.h". As such, these macros are not only required by the library, but also by our test suite, which is makes use of internal headers. To prepare for the CMakeLists.txt split, move these two defines inside of the "features.h" header.
Patrick Steinhardt a390a846 2017-07-01T13:06:00 cmake: move defines into "features.h" header In a future commit, we will split out the build instructions for our library directory and move them into a subdirectory. One of the benefits is fixing scoping issues, where e.g. defines do not leak to build targets where they do not belong to. But unfortunately, this does also pose the problem of how to propagate some defines which are required by both the library and the test suite. One way would be to create another variable keeping track of all added defines and declare it inside of the parent scope. While this is the most obvious and simplest way of going ahead, it is kind of unfortunate. The main reason to not use this is that these defines become implicit dependencies between the build targets. By simply observing a define inside of the CMakeLists.txt file, one cannot reason whether this define is only required by the current target or whether it is required by different targets, as well. Another approach would be to use an internal header file keeping track of all defines shared between targets. While configuring the library, we will set various variables and let CMake configure the file, adding or removing defines based on what has been configured. Like this, one can easily keep track of the current environment by simply inspecting the header file. Furthermore, these dependencies are becoming clear inside the CMakeLists.txt, as instead of simply adding a define, we now call e.g. `SET(GIT_THREADSAFE 1)`. Having this header file though requires us to make sure it is always included before any "#ifdef"-preprocessor checks are executed. As we have already refactored code to always include the "common.h" header file before any statement inside of a file, this becomes easy: just make sure "common.h" includes the new "features.h" header file first.
Patrick Steinhardt 0c7f49dd 2017-06-30T13:39:01 Make sure to always include "common.h" first Next to including several files, our "common.h" header also declares various macros which are then used throughout the project. As such, we have to make sure to always include this file first in all implementation files. Otherwise, we might encounter problems or even silent behavioural differences due to macros or defines not being defined as they should be. So in fact, our header and implementation files should make sure to always include "common.h" first. This commit does so by establishing a common include pattern. Header files inside of "src" will now always include "common.h" as its first other file, separated by a newline from all the other includes to make it stand out as special. There are two cases for the implementation files. If they do have a matching header file, they will always include this one first, leading to "common.h" being transitively included as first file. If they do not have a matching header file, they instead include "common.h" as first file themselves. This fixes the outlined problems and will become our standard practice for header and source files inside of the "src/" from now on.
Edward Thomson e6ed0d2f 2016-12-13T11:31:38 odb_loose: fsync tests Introduce a simple counter that `p_fsync` implements. This is useful for ensuring that `p_fsync` is called when we expect it to be, for example when we have enabled an odb backend to perform `fsync`s when writing objects.
Edward Thomson cc172642 2017-02-17T12:10:19 p_snprintf: no need for arguments to a format `snprintf` requires a _format_ but does not require _arguments_ to the format. eg: `snprintf(buf, 42, "hi")` is perfectly legal. Expand the macro to match. Without this, `p_sprintf(buf, 42, "hi")` errors with: ``` error: expected expression p_snprintf(msg, 42, "hi"); ^ src/unix/posix.h:53:34: note: expanded from macro 'p_snprintf' ^ /usr/include/secure/_stdio.h:57:73: note: expanded from macro 'snprintf' __builtin___snprintf_chk (str, len, 0, __darwin_obsz(str), __VA_ARGS__) ```
Edward Thomson 909d5494 2016-12-29T12:25:15 giterr_set: consistent error messages Error messages should be sentence fragments, and therefore: 1. Should not begin with a capital letter, 2. Should not conclude with punctuation, and 3. Should not end a sentence and begin a new one
Edward Thomson 82f15896 2016-11-18T07:19:22 threads: introduce `git_thread_exit` Introduce `git_thread_exit`, which will allow threads to terminate at an arbitrary time, returning a `void *`. On Windows, this means that we need to store the current `git_thread` in TLS, so that we can set its `return` value when terminating. We cannot simply use `ExitThread`, since Win32 returns `DWORD`s from threads; we return `void *`.
Arthur Schreiber ab96ca55 2016-10-06T13:15:31 Make sure we use the `C` locale for `regcomp` on macOS.
Patrick Steinhardt aab266c9 2016-06-20T20:07:33 threads: add platform-independent thread initialization function
Patrick Steinhardt 6551004f 2016-06-20T17:49:47 threads: split up OS-dependent rwlock code
Patrick Steinhardt 139bffa0 2016-06-20T17:20:13 threads: split up OS-dependent thread-condition code
Patrick Steinhardt 1c135405 2016-06-20T17:07:14 threads: split up OS-dependent mutex code
Patrick Steinhardt faebc1c6 2016-06-20T17:44:04 threads: split up OS-dependent thread code
Carlos Martín Nieto 87c18197 2016-03-16T19:05:11 Split the page size from the mmap alignment While often similar, these are not the same on Windows. We want to use the page size on Windows for the pools, but for mmap we need to use the allocation granularity as the alignment. On the other platforms these values remain the same.
Edward Thomson 3d6a42d1 2016-02-25T11:23:19 nsec: support NDK's crazy nanoseconds Android NDK does not have a `struct timespec` in its `struct stat` for nanosecond support, instead it has a single nanosecond member inside the struct stat itself. We will use that and use a macro to expand to the `st_mtim` / `st_mtimespec` definition on other systems (much like the existing `st_mtime` backcompat definition).
Edward Thomson 04c3b35f 2016-02-23T13:02:07 map: use `giterr_set` internally Use the `giterr_set` function, which actually supports `GITERR_OS`. The `giterr_set_str` function is exposed for external users and will not append the operating system's error message.
Edward Thomson 35439f59 2016-02-11T12:24:21 win32: introduce p_timeval that isn't stupid Windows defines `timeval` with `long`, which we cannot sanely cope with. Instead, use a custom timeval struct.
Edward Thomson 8649dfd8 2015-09-29T13:36:37 p_futimes: support using futimens when available
Edward Thomson 69c8bf7e 2015-06-30T14:21:29 posix compat: include sys/stat.h for mingw
Pierre-Olivier Latour 85a5e8eb 2015-06-17T09:00:23 Fixed Xcode 6.1 build warnings
Edward Thomson 121c3171 2015-06-16T15:18:04 Introduce p_utimes and p_futimes Provide functionality to set the time on a filesystem entry, using utimes or futimes on POSIX type systems or SetFileTime on Win32.
Carlos Martín Nieto c4e6ab5f 2015-06-15T14:32:08 crlf: tick the index forward to work around racy-git behaviour In order to avoid racy-git, we zero out the file size for entries with the same timestamp as the index (or during the initial checkout). This is the case in a couple of crlf tests, as the code is fast enough to do everything in the same second. As we know that we do not perform the modification just after writing out the index, which is what this is designed to work around, tick the mtime of the index file such that it doesn't agree with the files anymore, and we do not zero out these entries.
Edward Thomson edbfc52c 2015-04-29T11:05:27 git_path: introduce 'git_path_diriter' Introduce a new `git_path_diriter` that can iterate directories efficiently for each platform.
Stefan Sperling aad0bd6b 2014-11-03T13:46:56 Fix segmentation fault observed on OpenBSD/sparc64 A non-readable mapping of a file causes an access violation in the pack tests. Always use PROT_READ to work around this.
Carlos Martín Nieto 177a29d8 2014-10-27T10:39:45 Merge commit 'refs/pull/2366/head' of github.com:libgit2/libgit2
Jacques Germishuys 662f90e6 2014-07-13T16:08:46 Move p_realpath logic to realpath.c
Jacques Germishuys 07d03d31 2014-07-13T16:40:51 Introduce some consistency in definition/declaration ordering
Jacques Germishuys 2f795d8f 2014-07-12T14:45:56 Cleanup portability/compatibility layer * Removes mingw-compat.h * Cleans up separation of compiler/platform idiosyncrasies * Unifies mingw/msvc stat structures and functions * (Tries to) hide more compiler specific implementation details (even in our internal API)
Jacques Germishuys 529c3715 2014-07-13T16:12:33 Fix unix/posix.h include guard
Albert Meltzer 62e562f9 2014-05-18T07:54:41 Fix compiler warning (git_off_t cast to size_t). Use size_t for page size, instead of long. Check result of sysconf. Use size_t for page offset so no cast to size_t (second arg to p_mmap). Use mod instead div/mult pair, so no cast to size_t is necessary.
Carlos Martín Nieto f7310540 2014-05-13T02:41:48 indexer: use mmap for writing Some OSs cannot keep their ideas about file content straight when mixing standard IO with file mapping. As we use mmap for reading from the packfile, let's make writing to the pack file use mmap.
Philip Kelley c2c81615 2014-04-19T18:05:31 Win32: UTF-8 <-> WCHAR conversion overhaul
Russell Belfer 7697e541 2013-12-11T15:02:20 Test cancel from indexer progress callback This adds tests that try canceling an indexer operation from within the progress callback. After writing the tests, I wanted to run this under valgrind and had a number of errors in that situation because mmap wasn't working. I added a CMake option to force emulation of mmap and consolidated the Amiga-specific code into that new place (so we don't actually need separate Amiga code now, just have to turn on -DNO_MMAP). Additionally, I made the indexer code propagate error codes more reliably than it used to.
Jasper Lievisse Adriaanse 0d4a5b13 2013-04-22T00:13:35 Add missing prototype for p_realpath().
Carlos Martín Nieto 872ca1d3 2013-04-15T20:00:42 Fix compilation on OpenBSD
Russell Belfer e40f1c2d 2013-03-08T16:39:57 Make tree iterator handle icase equivalence There is a serious bug in the previous tree iterator implementation. If case insensitivity resulted in member elements being equivalent to one another, and those member elements were trees, then the children of the colliding elements would be processed in sequence instead of in a single flattened list. This meant that the tree iterator was not truly acting like a case-insensitive list. This completely reworks the tree iterator to manage lists with case insensitive equivalence classes and advance through the items in a unified manner in a single sorted frame. It is possible that at a future date we might want to update this to separate the case insensitive and case sensitive tree iterators so that the case sensitive one could be a minimal amount of code and the insensitive one would always know what it needed to do without checking flags. But there would be so much shared code between the two, that I'm not sure it that's a win. For now, this gets what we need. More tests are needed, though.
Carlos Martín Nieto 67fcac56 2013-01-29T18:00:32 Fix p_realpath on OpenBSD OpenBSD's realpath(3) doesn't require the last part of the path to exist. Override p_realpath in this OS to bring it in line with the library's assumptions.
Edward Thomson 359fc2d2 2013-01-08T17:07:25 update copyrights
Russell Belfer cccacac5 2012-11-14T22:41:51 Add POSIX compat lstat() variant for win32 The existing p_lstat implementation on win32 is not quite POSIX compliant when setting errno to ENOTDIR. This adds an option to make is be compliant so that code (such as checkout) that cares to have separate behavior for ENOTDIR can use it portably. This also contains a couple of other minor cleanups in the posix_w32.c implementations to avoid unnecessary work.
Eduardo Bart 345eef23 2012-11-07T16:10:57 Move inet_pton to posix platform-compatibility layer
Russell Belfer 824d5e4d 2012-10-11T11:58:00 Always use internal fnmatch, not system
Vicent Marti 17f7bde2 2012-08-23T15:47:08 posix: Always set a default mapping mode
Russell Belfer 85bd1746 2012-08-22T16:03:35 Some cleanup suggested during review This cleans up a number of items suggested during code review with @vmg, including: * renaming "outside repo" config API to `git_config_open_default` * killing the `git_config_open_global` API * removing the `git_` prefix from the static functions in fileops * removing some unnecessary functionality from the "cp" command
Russell Belfer ca1b6e54 2012-07-31T17:02:54 Add template dir and set gid to repo init This extends git_repository_init_ext further with support for initializing the repository from an external template directory and with support for the "create shared" type flags that make a set GID repository directory. This also adds tests for much of the new functionality to the existing `repo/init.c` test suite. Also, this adds a bunch of new utility functions including a very general purpose `git_futils_mkdir` (with the ability to make paths and to chmod the paths post-creation) and a file tree copying function `git_futils_cp_r`. Also, this includes some new path functions that were useful to keep the code simple.
Ben Straub bfc65634 2012-07-17T08:08:34 Merge branch 'development' into clone
Ben Straub 1d68fcd0 2012-07-16T16:16:11 Checkout: handle symlinks. Includes unfinished win32 implementation.
Chris Young 96ef3d84 2012-06-13T23:16:14 Make this more generic and mergeable. Needs AmigaOS.cmake now from CMake package at OS4Depot, or contents below: --8<-- SET(AMIGA 1) SET(CMAKE_SHARED_LIBRARY_C_FLAGS "-fPIC") SET(CMAKE_SHARED_LIBRARY_CREATE_C_FLAGS "-shared") --8<--
Chris Young 2aeadb9c 2012-06-12T19:25:09 Actually do the mmap... unsurprisingly, this makes the indexer work on SFS On RAM: the .idx and .pack files become links to a .lock and the original download respectively. Assume some feature (such as record locking) supported by SFS but not JXFS or RAM: is required.
Chris Young 90490113 2012-06-10T18:08:15 Basic mmap/munmap compatiblity
Chris Young 2774ccb8 2012-06-07T20:40:34 no fnmatch.h
Chris Young c3f35902 2012-06-07T20:29:22 Merge remote-tracking branch 'source/development' into update-test Merging main libgit2! Conflicts: CMakeLists.txt src/unix/map.c
Chris Young 82c23c58 2012-06-05T12:06:40 Assume this is irrelevant for now
Scott J. Goldman 6fb1c0b4 2012-05-09T23:45:55 Fix readdir_r() usage for Solaris On Solaris, struct dirent is defined differently than Linux. The field containing the path name is of size 0, rather than NAME_MAX. So, we need to use a properly sized buffer on Solaris to avoid a stack overflow. Also fix some DIR* leaks on cleanup.
Scott J. Goldman ec42eafd 2012-05-09T22:30:57 Hook up Windows compat fnmatch() for Solaris Since Solaris does not support some of the same flags as glibc fnmatch(), we just use the implementation we have for Windows. Now that it's no longer a windows-specific thing, I moved it into compat/ instead of win32/
Russell Belfer e3c47510 2012-03-13T14:23:24 Resolve comments from pull request This converts the map validation function into a macro, tweaks the GITERR_OS system error automatic appending, and adds a tentative new error access API and some quick unit tests for both the old and new error APIs.
Russell Belfer e1de726c 2012-03-12T22:55:40 Migrate ODB files to new error handling This migrates odb.c, odb_loose.c, odb_pack.c and pack.c to the new style of error handling. Also got the unix and win32 versions of map.c. There are some minor changes to other files but no others were completely converted. This also contains an update to filebuf so that a zeroed out filebuf will not think that the fd (== 0) is actually open (and inadvertently call close() on fd 0 if cleaned up). Lastly, this was built and tested on win32 and contains a bunch of fixes for the win32 build which was pretty broken.
Russell Belfer 74fa4bfa 2012-02-28T16:14:47 Update diff to use iterators This is a major reorganization of the diff code. This changes the diff functions to use the iterators for traversing the content. This allowed a lot of code to be simplified. Also, this moved the functions relating to outputting a diff into a new file (diff_output.c). This includes a number of other changes - adding utility functions, extending iterators, etc. plus more tests for the diff code. This also takes the example diff.c program much further in terms of emulating git-diff command line options.
Russell Belfer 290f240e 2012-02-23T11:16:47 Fix readdir usage across platforms This fixes the missing readdir_r from win32 and fixes other platforms to always use the reentrant readdir_r form for reading directory contents.
schu 5e0de328 2012-02-13T17:10:24 Update Copyright header Signed-off-by: schu <schu-github@schulog.org>
Paul Betts 222d057c 2011-09-19T10:34:52 Create cross-platform setenv
Vicent Marti bb742ede 2011-09-19T01:54:32 Cleanup legal data 1. The license header is technically not valid if it doesn't have a copyright signature. 2. The COPYING file has been updated with the different licenses used in the project. 3. The full GPLv2 header in each file annoys me.
Vicent Marti f978b748 2011-08-30T13:34:14 compat: Move `mkstemp` to the POSIX compat layer
Vicent Marti 84dd3820 2011-08-18T02:13:51 posix: Properly handle `snprintf` in all platforms
schu 7fade6c6 2011-08-17T12:14:12 unix/posix.h: remove redundant include Signed-off-by: schu <schu-github@schulog.org>
Vicent Marti 2fc78e70 2011-07-08T23:01:37 posix: Portable `vsnprintf` Our good, lovely folks at Microsoft decided that there was no good reason to make `vsnprintf` compilant with the C standard, so that function in Windows returns -1 on overflow, instead of returning the actual byte count needed to write the full string. We now handle this situation more gracefully with the POSIX compatibility layer, by returning the needed byte size using an auxiliary method instead of blindly resizing the target buffer until it fits. This means we can now support `printf`s of any size by allocating a temporary buffer. That's good.
Vicent Marti e9c6571d 2011-07-06T01:04:04 fnmatch: Use native on Unix, emulate on Win32
Vicent Marti 5ad739e8 2011-07-04T20:05:11 fileops: Drop `git_fileops_prettify_path` The old `git_fileops_prettify_path` has been replaced with `git_path_prettify`. This is a much simpler method that uses the OS's `realpath` call to obtain the full path for directories and resolve symlinks. The `realpath` syscall is the original POSIX call in Unix system and an emulated version under Windows using the Windows API.
Vicent Marti f79026b4 2011-07-04T11:43:34 fileops: Cleanup Cleaned up the structure of the whole OS-abstraction layer. fileops.c now contains a set of utility methods for file management used by the library. These are abstractions on top of the original POSIX calls. There's a new file called `posix.c` that contains emulations/reimplementations of all the POSIX calls the library uses. These are prefixed with `p_`. There's a specific posix file for each platform (win32 and unix). All the path-related methods have been moved from `utils.c` to `path.c` and have their own prefix.
Vicent Marti 678e9e04 2011-07-03T13:33:43 build: Move OS-specific compat to their own folders
Vicent Marti 17d52304 2011-07-01T17:26:23 build: Simplify build structure This will make libgit2 more suitable for embedding.
Jakob Pfender 450ac186 2011-05-19T15:29:22 unix/map.c: Move to new error handling mechanism
Alex Budovski f0bde7fa 2011-01-11T16:07:45 Revised platform types to use 'best supported' size. This will allow graceful migration to 64 bit file sizes and timestamps should git's binary interface be extended to allow this.
Vicent Marti 9f54fe48 2010-12-23T00:15:09 Remove git_errno It was not being used by any methods (only by malloc and calloc), and since it needs to be TLS, it cannot be exported on DLLs on Windows. Burn it with fire. The API always returns error codes! Signed-off-by: Vicent Marti <tanoku@gmail.com>
Ramsay Jones 79ca2edc 2009-03-20T19:51:48 win32: Add routines to abstract memory-mapped file functions In particular, the git__mmap() and git__munmap() routines provide the interface to platform specific memory-mapped file facilities. We provide implementations for unix and win32, which can be found in their own sub-directories. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>