|
88731e3c
|
2019-06-14T12:34:37
|
|
examples: implement git-stash example
Implement a new example that resembles the git-stash(1) command.
Right now, it only provides the apply, list, save and pop
subcommands without any options.
This example is mostly used to test libgit2's stashing
performance on big repositories.
|
|
1f44079c
|
2019-07-20T18:08:40
|
|
Merge pull request #5179 from pks-t/pks/patch-parse-free
patch_parse: fix segfault due to line containing static contents
|
|
a613832e
|
2019-07-20T18:49:48
|
|
patch_parse: fix segfault due to line containing static contents
With commit dedf70ad2 (patch_parse: do not depend on parsed buffer's
lifetime, 2019-07-05), all lines of the patch are allocated with
`strdup` to make lifetime of the parsed patch independent of the buffer
that is currently being parsed. In patch b08932824 (patch_parse: ensure
valid patch output with EOFNL, 2019-07-11), we introduced another
code location where we add lines to the parsed patch. But as that one
was implemented via a separate pull request, it wasn't converted to use
`strdup`, as well. As a consequence, we generate a segfault when trying
to deallocate the potentially static buffer that's now in some of the
lines.
Use `git__strdup` to fix the issue.
|
|
e07dbc92
|
2019-07-20T11:26:00
|
|
Merge pull request #5173 from pks-t/pks/gitignore-wildmatch-error
ignore: fix determining whether a shorter pattern negates another
|
|
fd7a384b
|
2019-07-20T11:24:37
|
|
Merge pull request #5159 from pks-t/pks/patch-parse-old-missing-nl
patch_parse: handle missing newline indicator in old file
|
|
f33ca472
|
2019-07-20T11:06:23
|
|
Merge pull request #5158 from pks-t/pks/patch-parsed-lifetime
patch_parse: do not depend on parsed buffer's lifetime
|
|
d78a1b18
|
2019-07-20T11:04:53
|
|
Merge pull request #5174 from pks-t/pks/winhttp-hash
sha1: fix compilation of WinHTTP backend
|
|
964c1c60
|
2019-07-20T11:02:30
|
|
Merge pull request #5176 from pks-t/pks/repo-template-head
repository: do not initialize HEAD if it's provided by templates
|
|
9d46f167
|
2019-07-19T10:50:51
|
|
repository: do not initialize HEAD if it's provided by templates
When using templates to initialize a git repository, then git-init(1)
will copy over all contents of the template directory. These will be
preferred over the default ones created by git-init(1). While we mostly
do the same, there is the exception of "HEAD". While we do copy over the
template's HEAD file, afterwards we'll immediately re-initialize its
contents with either the default "ref: refs/origin/master" or the init
option's `initial_head` field.
Let's fix the inconsistency with upstream git-init(1) by not overwriting
the template HEAD, but only if the user hasn't set `opts.initial_head`.
If the `initial_head` field has been supplied, we should use that
indifferent from whether the template contained a HEAD file or not. Add
tests to verify we correctly use the template directory's HEAD file and
that `initial_head` overrides the template.
|
|
f3134a84
|
2019-07-19T10:41:10
|
|
repository: update error handling in `init_ext`
Update `git_repository_init_ext` to use our typical style of error
handling. The function had multiple statements which didn't `goto out`
immediately but instead deferred it to later calls combined with `if`
statements.
|
|
869ae5a3
|
2019-07-19T10:15:43
|
|
repository: avoid swallowing error codes in `create_head`
The error handling in `git_repository_create_head` completely swallows
all error codes. While probably not too much of a problem, this also
violates our usual coding style.
Refactor the code to use a local `error` variable with the typical `goto
out` statements.
|
|
0d12b8dd
|
2019-07-19T09:43:34
|
|
tests: repo: refactor setup of templates and repos
All tests in repo::template have a common pattern of first setting up
templates, then settung up the repository that makes use of those
templates via several init options. Refactor this pattern into two
functions `setup_templates` and `setup_repo` that handle most of that
logic to make it easier to spot what a test actually wants to check.
Furthermore, this also refactors how we clean up after the tests.
Previously, it was a combination of manually calling
`cl_fixture_cleanup` and `cl_set_cleanup`, which really is kind of hard
to read. This commit refactors this to instead provide the cleaning
parameters in the setup functions. All cleanups are then performed in
the suite's cleanup function.
|
|
3b79ceaf
|
2019-07-19T08:58:12
|
|
tests: repo: refactor template path handling
The repo::template test suite makes use of quite a few local variables
that could be consolidated. Do so to make the code easier to read.
|
|
ee193480
|
2019-07-19T08:45:45
|
|
tests: repo: move template tests into their own suite
There's quite a lot of supporting code for our templates and they are an
obvious standalone feature. Thus, let's extract those tests into their
own suite to also make refactoring of them easier.
|
|
3424c210
|
2019-07-19T08:00:13
|
|
Merge pull request #5138 from libgit2/ethomson/cvar
configuration: cvar -> configmap
|
|
a33c0de2
|
2019-07-18T19:17:40
|
|
Merge pull request #5172 from bk2204/cache-efficient-eviction
Evict cache items more efficiently
|
|
e86d75f3
|
2019-07-18T19:00:42
|
|
Merge pull request #5175 from pks-t/pks/clar-fix-suite-count
clar: fix suite count
|
|
92109976
|
2019-07-18T14:20:18
|
|
tests: fix undercounting of suites
With the introduction of data variants for suites, we started
undercounting the number of suites as we didn't account for those that
were executed twice. This was then adjusted to count the number of
initializers instead, but this fails to account for suites without any
initializers at all.
Fix the suite count by counting either the number of initializers or, if
there is no initializer, count it as a single suite, only.
|
|
29fe79e6
|
2019-07-18T14:07:22
|
|
Merge pull request #5163 from csware/gitignore-vs2017
Ignore VS2017 specific files and folders
|
|
36558513
|
2019-06-24T23:31:23
|
|
configuration: deprecate git_cvar safely
|
|
658022c4
|
2019-07-18T13:53:41
|
|
configuration: cvar -> configmap
`cvar` is an unhelpful name. Refactor its usage to `configmap` for more
clarity.
|
|
343fb83a
|
2019-07-18T13:50:47
|
|
Merge pull request #5156 from pks-t/pks/attr-macros-in-subdir
gitattributes: ignore macros defined in subdirectories
|
|
270fd807
|
2019-07-18T13:44:10
|
|
azure: compile one Windows platform with the WinHTTP SHA1 backend
We currently have no job that compiles libgit2 with the WinHTTP backend
for SHA1. Due to this, a compile error has been introduced and not
noticed for several months. Change the x86 MSVC job to use the HTTPS
backend for SHA1. The x86 job was chosen with no particular reason.
|
|
7574564e
|
2019-07-18T13:40:34
|
|
sha1: win32: fix compilation due to unknown type
In commit bbf034ab9 (hash: move `git_hash_prov` into Win32 backend,
2019-02-22), the `git_hash_prov`'s structure name has been removed in
favour of its typedef'ed name. But as we have no CI that compiles with
the WinHTTPS hashing backend right now, it wasn't noticed that the
implementation that uses this struct wasn't changed correctly.
Fix the struct type to make it compile again.
|
|
b7c247b3
|
2019-07-18T13:37:02
|
|
cmake: include SHA1 headers into our source files
When selecting the SHA1 backend, we only include the respective C
implementation of the selected backend. But since commit bd48bf3fb
(hash: introduce source files to break include circles, 2019-06-14), we
have introduced separate headers and compilation units for all hashes.
So by not including the headers, we may not honor them to compute
whether a file needs to be recompiled and they also will not be
displayed in IDEs.
Add the header files to fix this problem.
|
|
6f6340af
|
2019-07-18T11:57:55
|
|
ignore: fix determining whether a shorter pattern negates another
When computing whether we need to store a negative pattern, we iterate
through all previously known patterns and check whether the negative
pattern undoes any of the previous ones. In doing so we call `wildmatch`
and check it's return for any negative error values. If there was a
negative return, we will abort and bubble up that error to the caller.
In fact, this check for negative values stems from the time where we
still used `fnmatch` instead of `wildmatch`. For `fnmatch`, negative
values indicate a "real" error, while for `wildmatch` a negative value
may be returned if the matching was prematurely aborted. A premature
abort may for example also happen if the pattern matches a prefix of the
haystack if the pattern is shorter. Returning an error in that case is
the wrong thing to do.
Fix the code to compare for equality with `WM_MATCH`, only. Negative
values returned by `wildmatch` are perfectly fine and thus should be
ignored. Add a test that verifies we do not see the error.
|
|
368b9795
|
2019-07-18T11:27:21
|
|
Merge pull request #5168 from tiennou/clar/fix-data-suite-count
clar: correctly account for "data" suites when counting
|
|
51124a5b
|
2019-07-17T17:33:34
|
|
Merge pull request #5170 from bk2204/packbuilder-efficient-realloc
Allocate memory more efficiently when packing objects
|
|
770b91b1
|
2019-07-17T15:59:54
|
|
cache: evict items more efficiently
When our object cache is full, we pick eight items (or the whole cache,
if there are fewer) and evict them. For small cache sizes, this is fine,
but when we're dealing with a large number of objects, we can repeatedly
exhaust the cache and spend a large amount of time in git_oidmap_iterate
trying to find items to evict.
Instead, let's assume that if the cache gets full, we have a large
number of objects that we're handling, and be more aggressive about
evicting items. Let's remove one item for every 2048 items, but not less
than 8. This causes us to scale our evictions in proportion to the size
of the cache and significantly reduces the time we spend in
git_oidmap_iterate.
Before this change, a full pack of all the non-blob objects in the Linux
repository took in excess of 30 minutes and spent 62.3% of total runtime
in odb_read_1 and its children, and 44.3% of the time in
git_oidmap_iterate. With this change, the same operation now takes 14
minutes and 44 seconds, and odb_read_1 accounts for only 35.9% of total
time, whereas git_oidmap_iterate consists of 6.2%.
Note that we do spend a little more time inflating objects and a decent
amount more time in memcmp. However, overall, the time taken is
significantly improved, and time in pack building is now dominated by
git_delta_create_from_index (33.7%), which is what we would expect.
|
|
c4df926b
|
2019-07-16T21:54:10
|
|
pack-objects: allocate memory more efficiently
The packbuilder code allocates memory in chunks. When it needs to
allocate, it tries to add 1024 to the number of objects and multiply by
3/2. However, it actually multiplies by 1 instead, since it performs an
integral division in the expression "3 / 2" and only then multiplies by
the increased number of objects.
The current behavior causes the code to waste massive amounts of time
copying memory when it reallocates, causing inserting all non-blob
objects in the Linux repository into a new pack to take some
indeterminate time greater than 5 minutes instead of 52 seconds.
Correct this error by first dividing by two, and only then multiplying
by 3. We still check for overflow for the multiplication, which is the
only part that can overflow. This appears to be the only place in the
code base which has this problem.
|
|
4cd8dfaa
|
2019-07-16T20:20:55
|
|
clar: correctly account for "data" suites when counting
Failing to do that makes clar miss the last of the suites, as all
duplicated "data" would have not been accounted for.
|
|
5f22f8d2
|
2019-07-12T17:25:50
|
|
Ignore VS2017 specific files and folders
Signed-off-by: Sven Strickroth <email@cs-ware.de>
|
|
f92d495d
|
2019-07-12T10:48:14
|
|
Merge pull request #5131 from pks-t/pks/fileops-mkdir-in-root
fileops: fix creation of directory in filesystem root
|
|
f8346905
|
2019-07-12T09:03:33
|
|
attr_file: ignore macros defined in subdirectories
Right now, we are unconditionally applying all macros found in a
gitatttributes file. But quoting gitattributes(5):
Custom macro attributes can be defined only in top-level
gitattributes files ($GIT_DIR/info/attributes, the .gitattributes
file at the top level of the working tree, or the global or
system-wide gitattributes files), not in .gitattributes files in
working tree subdirectories. The built-in macro attribute "binary"
is equivalent to:
So gitattribute files in subdirectories of the working tree may
explicitly _not_ contain macro definitions, but we do not currently
enforce this limitation.
This patch introduces a new parameter to the gitattributes parser that
tells whether macros are allowed in the current file or not. If set to
`false`, we will still parse macros, but silently ignore them instead of
adding them to the list of defined macros. Update all callers to
correctly determine whether the to-be-parsed file may contain macros or
not. Most importantly, when walking up the directory hierarchy, we will
only set it to `true` once it reaches the root directory of the repo
itself.
Add a test that verifies that we are indeed not applying macros from
subdirectories. Previous to these changes, the test would've failed.
|
|
97968529
|
2019-07-05T08:05:16
|
|
attr_file: refactor `parse_buffer` function
The gitattributes code is one of our oldest and most-untouched codebases
in libgit2, and as such its code style doesn't quite match our current
best practices. Refactor the function `git_attr_file__parse_buffer` to
better match them.
|
|
dbc7e4b1
|
2019-07-05T07:53:02
|
|
attr_file: refactor `load_standalone` function
The gitattributes code is one of our oldest and most-untouched codebases
in libgit2, and as such its code style doesn't quite match our current
best practices. Refactor the function `git_attr_file__lookup_standalone`
to better match them.
|
|
be8f9bb1
|
2019-07-05T13:33:10
|
|
attrcache: fix memory leak if inserting invalid macro to cache
A macro without any assignments is considered an invalid macro by the
attributes cache and is thus not getting added to the macro map at all.
But as `git_attr_cache__insert_macro` returns success with neither
free'ing nor adopting the macro into its map, this will cause a memory
leak.
Fix this by freeing the macro in the function if it's not going to be
added. This is perfectly fine to do, as callers assume that the
attrcache will have the macro adopted on success anyway.
|
|
7277bf83
|
2019-07-05T13:33:05
|
|
attrcache: fix multiple memory leaks when inserting macros
The function `git_attr_cache__insert_macro` is responsible for adopting
macros in the per-repo macro cache. When adding a macro that replaces an
already existing macro (e.g. because of re-parsing gitattributes files),
then we do not free the previous macro and thus cause a memory leak.
Fix this leak by first checking if the cache already has a macro defined
with the same name. If so, free it before replacing the cache entry with
the new instance.
|
|
df417a43
|
2019-07-12T09:02:16
|
|
tests: attr: verify that in-memory macros are respected
Add some tests to ensure that the `git_attr_add_macro` function works as
expected.
|
|
4a7f704f
|
2019-07-05T08:10:33
|
|
tests: attr: implement tests to verify attribute rewriting behaviour
Implement some tests that verify that we are correctly updating
gitattributes when rewriting or unlinking the corresponding files.
|
|
ed854aa0
|
2019-07-05T07:45:22
|
|
tests: attr: extract macro tests into their own suite
As macros are a specific functionality in the gitattributes code, it
makes sense to extract them into their own test suite, too. This makes
finding macro-related tests easier.
|
|
dacac9e1
|
2019-07-12T08:30:07
|
|
Merge pull request #5160 from pks-t/pks/win32-fuzzers
win32: fix fuzzers and have CI build them
|
|
5ae22a63
|
2019-06-21T08:13:31
|
|
fileops: fix creation of directory in filesystem root
In commit 45f24e787 (git_repository_init: stop traversing at
windows root, 2019-04-12), we have fixed `git_futils_mkdir` to
correctly handle the case where we create a directory in
Windows-style filesystem roots like "C:\repo".
The problem here is an off-by-one: previously, to that commit,
we've been checking wether the parent directory's length is equal
to the root directory's length incremented by one. When we call
the function with "/example", then the parent directory's length
("/") is 1, but the root directory offset is 0 as the path is
directly rooted without a drive prefix. This resulted in `1 == 0 +
1`, which was true. With the change, we've stopped incrementing
the root directory length, and thus now compare `1 <= 0`, which
is false.
The previous way of doing it was kind of finicky any non-obvious,
which is also why the error was introduced. So instead of just
re-adding the increment, let's explicitly add a condition that
aborts finding the parent if the current parent path is "/".
Making this change causes Azure Pipelines to fail the testcase
repo::init::nonexistent_paths on Unix-based systems. This is because we
have just fixed creating directories in the filesystem root, which
previously didn't work. As Docker-based tests are running as root user,
we are thus able to create the non-existing path and will now succeed to
create the repository that was expected to actually fail.
Let's split this up into three different tests:
- A test to verify that we do not create repos in a non-existing parent
directoy if the flag `GIT_REPOSITORY_INIT_MKPATH` is not set.
- A test to verify that we fail if the root directory does not exist. As
there is a common root directory on Unix-based systems that always
exist, we can only test for this on Windows-based systems.
- A test to verify that we fail if trying to create a repository in an
unwriteable parent directory. We can only test this if not running
tests as root user, as CAP_DAC_OVERRIDE will cause us to ignore
permissions when creating files.
|
|
a6ad9e8a
|
2019-07-11T14:03:21
|
|
Merge pull request #5134 from pks-t/pks/config-parser-separation
Config parser separation
|
|
b0893282
|
2019-07-11T12:12:04
|
|
patch_parse: ensure valid patch output with EOFNL
|
|
3f855fe8
|
2019-07-05T11:06:33
|
|
patch_parse: handle missing newline indicator in old file
When either the old or new file contents have no newline at the end of
the file, then git-diff(1) will print out a "\ No newline at end of
file" indicator. While we do correctly handle this in the case where the
new file has this indcator, we fail to parse patches where the old file
is missing a newline at EOF.
Fix this bug by handling and missing newline indicators in the old file.
Add tests to verify that we can parse such files.
|
|
b30dab8f
|
2019-07-11T12:10:48
|
|
apply: refactor to use a switch statement
|
|
001d76e1
|
2019-07-11T11:34:40
|
|
diff: ignore EOFNL for computing patch IDs
The patch ID is supposed to be mostly context-insignificant and
thus only includes added or deleted lines. As such, we shouldn't honor
end-of-file-without-newline markers in diffs.
Ignore such lines to fix how we compute the patch ID for such diffs.
|
|
dbeadf8a
|
2019-07-11T10:56:05
|
|
config_parse: provide parser init and dispose functions
Right now, all configuration file backends are expected to
directly mess with the configuration parser's internals in order
to set it up. Let's avoid doing that by implementing both a
`git_config_parser_init` and `git_config_parser_dispose` function
to clearly define the interface between configuration backends
and the parser.
Ideally, we would make the `git_config_parser` structure
definition private to its implementation. But as that would
require an additional memory allocation that was not required
before we just live with it being visible to others.
|
|
32157526
|
2019-07-11T11:10:02
|
|
config_file: refactor error handling in `config_write`
Error handling in `config_write` is rather convoluted and does
not match our current code style. Refactor it to make it easier
to understand.
|
|
820fa1a3
|
2019-07-11T11:04:33
|
|
config_file: internalize `git_config_file` struct
With the previous commits, we have finally separated the config
parsing logic from the specific configuration file backend. Due
to that, we can now move the `git_config_file` structure into the
config file backend's implementation so that no other code may
accidentally start using it again. Furthermore, we rename the
structure to `diskfile` to make it obvious that it is internal,
only, and to unify it with naming scheme of the other diskfile
structures.
|
|
6e6da75f
|
2019-07-11T11:00:05
|
|
config_parse: remove use of `git_config_file`
The config parser code needs to keep track of the current parsed
file's name so that we are able to provide proper error messages
to the user. Right now, we do that by storing a `git_config_file`
in the parser structure, but as that is a specific backend and
the parser aims to be generic, it is a layering violation.
Switch over to use a simple string to fix that.
|
|
54d350e0
|
2019-06-21T12:53:43
|
|
config_file: embed file in diskfile parse data
The config file code needs to keep track of the actual
`git_config_file` structure, as it not only contains the path
of the current configuration file, but it also keeps tracks of
all includes of that file. Right now, we keep track of that
structure via the `git_config_parser`, but as that's supposed to
be a backend generic implementation of configuration parsing it's
a layering violation to have it in there.
Switch over the config file backend to use its own config file
structure that's embedded in the backend parse data. This allows
us to switch over the generic config parser to avoid using the
`git_config_file` structure.
|
|
76749dfb
|
2019-06-21T12:33:31
|
|
config_parse: rename `data` parameter to `payload` for clarity
By convention, parameters that get passed to callbacks are
usually named `payload` in our codebase. Rename the `data`
parameters in the configuration parser callbacks to `payload` to
avoid confusion.
|
|
ba9725a2
|
2019-07-11T10:48:49
|
|
Merge pull request #5132 from pks-t/pks/config-stat-cache
config_file: implement stat cache to avoid repeated rehashing
|
|
2ba7020f
|
2019-06-27T09:23:59
|
|
config_file: avoid re-reading files on write
When we rewrite the configuration file due to any of its values
being modified, we call `config_refresh` to update the in-memory
representation of our config file backend. This is needlessly
wasteful though, as `config_refresh` will always open the on-disk
representation to reads the file contents while we already know
the complete file contents at this point in time as we have just
written it to disk.
Implement a new function `config_refresh_from_buffer` that will
refresh the backend's config entries from a buffer instead of
from the config file itself. Note that this will thus _not_
update the backend's timestamp, which will cause us to re-read
the buffer when performing a read operation on it. But this is
still an improvement as we now lazily re-read the contents, and
most importantly we will avoid constantly re-reading the contents
if we perform multiple write operations.
The following strace demonstrates this if we're re-writing a key
multiple times. It uses our config example with `config_set`
changed to update the file 10 times with different keys:
$ strace lg2 config x.x z |& grep '^open.*config'
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
And now with the optimization of `config_refresh_from_buffer`:
$ strace lg2 config x.x z |& grep '^open.*config'
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
open("/tmp/repo/.git/config.lock", O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 4
As can be seen, this is quite a lot of `open` calls less.
|
|
a0dc3027
|
2019-06-27T08:54:51
|
|
config_file: split out function that sets config entries
Updating a config file backend's config entries is a bit more
involved, as it requires clearing of the old config entries as
well as handling locking correctly. As we will need this
functionality in a future patch to refresh config entries from a
buffer, let's extract this into its own function
`config_set_entries`.
|
|
985f5cdf
|
2019-06-27T08:41:16
|
|
config_file: split out function that reads entries from a buffer
The `config_read` function currently performs both reading the
on-disk config file as well as parsing the retrieved buffer
contents. To optimize how we refresh our config entries from an
in-memory buffer, we need to be able to directly parse buffers,
though, without involving any on-disk files at all.
Extract a new function `config_read_buffer` that sets up the
parsing logic and then parses config entries from a buffer, only.
Have `config_read` use it to avoid duplicated logic.
|
|
3e1c137a
|
2019-06-27T08:24:21
|
|
config_file: move refresh into `write` function
We are quite lazy in how we refresh our config file backend when
updating any of its keys: instead of just updating our in-memory
representation of the keys, we just discard the old set of keys
and then re-read the config file contents from disk. This refresh
currently happens separately at every callsite of `config_write`,
but it is clear that we _always_ want to refresh if we have
written the config file to disk. If we didn't, then we'd run
around with an outdated config file backend that does not
represent what we have on disk.
By moving the refresh into `config_write`, we are also able to
optimize the case where the config file is currently locked.
Before, we would've tried to re-read the file even if we have
only updated its cached contents without touching the on-disk
file. Thus we'd have unnecessarily stat'd the file, even though
we know that it shouldn't have been modified in the meantime due
to its lock.
|
|
d7f58eab
|
2019-06-21T11:55:21
|
|
config_file: implement stat cache to avoid repeated rehashing
To decide whether a config file has changed, we always hash its
complete contents. This is unnecessarily expensive, as
well-behaved filesystems will always update stat information for
files which have changed. So before computing the hash, we should
first check whether the stat info has actually changed for either
the configuration file or any of its includes. This avoids having
to re-read the configuration file and its includes every time
when we check whether it's been modified.
Tracing the for-each-ref example previous to this commit, one can
see that we repeatedly re-open both the repo configuration as
well as the global configuration:
$ strace lg2 for-each-ref |& grep config
access("/home/pks/.gitconfig", F_OK) = -1 ENOENT (No such file or directory)
access("/home/pks/.config/git/config", F_OK) = 0
access("/etc/gitconfig", F_OK) = -1 ENOENT (No such file or directory)
stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
access("/tmp/repo/.git/config", F_OK) = 0
stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
stat("/home/pks/.gitconfig", 0x7ffd15c05290) = -1 ENOENT (No such file or directory)
access("/home/pks/.gitconfig", F_OK) = -1 ENOENT (No such file or directory)
stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
access("/home/pks/.config/git/config", F_OK) = 0
stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
stat("/home/pks/.gitconfig", 0x7ffd15c051f0) = -1 ENOENT (No such file or directory)
stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
stat("/home/pks/.gitconfig", 0x7ffd15c05090) = -1 ENOENT (No such file or directory)
stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
stat("/home/pks/.gitconfig", 0x7ffd15c05090) = -1 ENOENT (No such file or directory)
stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
stat("/home/pks/.gitconfig", 0x7ffd15c05090) = -1 ENOENT (No such file or directory)
stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
With the change, we only do stats for those files and open them a
single time, only:
$ strace lg2 for-each-ref |& grep config
access("/home/pks/.gitconfig", F_OK) = -1 ENOENT (No such file or directory)
access("/home/pks/.config/git/config", F_OK) = 0
access("/etc/gitconfig", F_OK) = -1 ENOENT (No such file or directory)
stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
access("/tmp/repo/.git/config", F_OK) = 0
stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
open("/tmp/repo/.git/config", O_RDONLY|O_CLOEXEC) = 3
stat("/home/pks/.gitconfig", 0x7ffe70540d20) = -1 ENOENT (No such file or directory)
access("/home/pks/.gitconfig", F_OK) = -1 ENOENT (No such file or directory)
stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
access("/home/pks/.config/git/config", F_OK) = 0
stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
open("/home/pks/.config/git/config", O_RDONLY|O_CLOEXEC) = 3
stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
stat("/home/pks/.gitconfig", 0x7ffe70540ca0) = -1 ENOENT (No such file or directory)
stat("/home/pks/.gitconfig", 0x7ffe70540c80) = -1 ENOENT (No such file or directory)
stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
stat("/home/pks/.gitconfig", 0x7ffe70540b40) = -1 ENOENT (No such file or directory)
stat("/home/pks/.gitconfig", 0x7ffe70540b20) = -1 ENOENT (No such file or directory)
stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
stat("/home/pks/.gitconfig", 0x7ffe70540b40) = -1 ENOENT (No such file or directory)
stat("/home/pks/.gitconfig", 0x7ffe70540b20) = -1 ENOENT (No such file or directory)
stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
stat("/tmp/repo/.git/config", {st_mode=S_IFREG|0644, st_size=92, ...}) = 0
stat("/home/pks/.gitconfig", 0x7ffe70540b40) = -1 ENOENT (No such file or directory)
stat("/home/pks/.gitconfig", 0x7ffe70540b20) = -1 ENOENT (No such file or directory)
stat("/home/pks/.config/git/config", {st_mode=S_IFREG|0644, st_size=1154, ...}) = 0
The following benchmark has been performed with and without the
stat cache in a best-of-ten run:
```
int lg2_repro(git_repository *repo, int argc, char **argv)
{
git_config *cfg;
int32_t dummy;
int i;
UNUSED(argc);
UNUSED(argv);
check_lg2(git_repository_config(&cfg, repo),
"Could not obtain config", NULL);
for (i = 1; i < 100000; ++i)
git_config_get_int32(&dummy, cfg, "foo.bar");
git_config_free(cfg);
return 0;
}
```
Without stat cache:
$ time lg2 repro
real 0m1.528s
user 0m0.568s
sys 0m0.944s
With stat cache:
$ time lg2 repro
real 0m0.526s
user 0m0.268s
sys 0m0.258s
This benchmark shows a nearly three-fold performance improvement.
This change requires that we check our configuration stress tests
as we're now in fact becoming more racy. If somebody is writing a
configuration file at nearly the same time (there is a window of
100ns on Windows-based systems), then it might be that we realize
that this file has actually changed and thus may not re-read it.
This will only happen if either an external process is rewriting
the configuration file or if the same process has multiple
`git_config` structures pointing to the same time, where one of
both is being used to write and the other one is used to read
values.
|
|
d0868646
|
2019-06-21T11:43:09
|
|
config: use `git_config_file` in favor of `struct config_file`
|
|
8ee3d39a
|
2019-06-27T09:18:19
|
|
examples: implement config example
Implement a new example that resembles git-config(1). Right now,
this example can both read and set configuration keys, only.
|
|
df54c7fb
|
2019-06-27T07:34:43
|
|
cmake: report whether we are using sub-second stat information
Depending on the platform and on build options, we may or may not
build libgit2 with support for nanoseconds when using `stat`
calls. It's currently unclear though whether sub-second stat
information is used at all.
Add feature info for this to tell at configure time whether it's
being used or not.
|
|
eb27fb9b
|
2019-07-05T11:59:17
|
|
ci: build fuzzers on Powershell based build jobs
In order to guarantee that our fuzzers build just fine on the Windows
platform, let's enable building fuzzers on all Powershell-based builds.
|
|
3c966fb4
|
2019-06-28T10:53:03
|
|
fuzzers: clean up header includes
There's multiple headers included in our fuzzers that aren't required at
all. Furthermore, some of them are not available on Win32, causing
builds to fail. Remove them to fix this.
|
|
9d43d45b
|
2019-06-28T12:10:51
|
|
fuzzers: use `git_buf_printf` instead of `snprintf`
The `snprintf` function does not exist on Win32, it only has
`_snprintf_s` available. Let's just avoid any cross-platform hassle and
use our own `git_buf` functionality instead.
|
|
a6b2fffd
|
2019-06-28T11:04:21
|
|
fuzzers: use POSIX emulation layer to unlink files
Use `p_unlink` instead of `unlink` to remove the generated packfiles in
our packfile fuzzer. Like this, we do not have to worry about using
proper includes that are known on all platforms, especially Win32.
|
|
69055813
|
2019-06-28T10:50:01
|
|
fuzzers: make printf formatters cross-platform compatible
The `printf` formatters in our standalone fuzzing driver are currently
using the "%m" specifier, which is a GNU extension that prints the error
message for the error code in `errno`. As we're using libgit2 functions
in both cases anyway, let's just use `git_error_last` instead to make
this valid on all platforms.
|
|
48d56328
|
2019-06-28T10:47:37
|
|
fuzzers: implement `mkdtemp` alternative for Win32
The `mkdtemp` function is not available on Windows, so our download_refs
fuzzer will fail to compile on Windows. Provide an alternative
implementation to fix it.
|
|
398412cc
|
2019-07-05T11:56:16
|
|
Merge pull request #5143 from libgit2/ethomson/warnings
ci: build with ENABLE_WERROR on Windows
|
|
a3afda9f
|
2019-06-28T11:50:32
|
|
tests: trace: fix parameter type of aux callback
The function `git_win32__stack__set_aux_cb` expects the second parameter
to be a function callback of type `git_win32__stack__aux_cb_lookup`,
which expects a `size_t` parameter. In our test suite
trace::windows::stacktrace, we declare the callback with `unsigned int`
as parameter, though, causing a compiler warning.
Correct the parameter type to silence the warning.
|
|
2f14c4fc
|
2019-06-28T14:39:20
|
|
w32_stack: convert buffer length param to `size_t`
In both `git_win32__stack_format` and `git_win32__stack`, we handle
buffer lengths via an integer variable. As we only ever pass buffer
sizes to it, this should be a `size_t` though to avoid loss of
precision. As we also use it to compare with other `size_t` variables,
this also silences signed/unsigned comparison warnings.
|
|
77d7e5eb
|
2019-06-27T15:29:36
|
|
clar: use `size_t` to keep track of current line number
We use the `__LINE__` macro in several places throughout clar to allow
easier traceability when e.g. a test fails. While `__LINE__` is of type
`size_t`, the clar functions all accept an integer and thus may loose
precision. While unlikely that any file in our codebase will exceed a
linecount of `INT_MAX`, let's convert it anyway to silence any compiler
warnings.
|
|
2dea4736
|
2019-06-27T15:27:29
|
|
examples: avoid warning when iterating over index entries
When iterating over index entries, we store the indices in an unsigned
int. As the index entrycount is a `size_t` though, this may be a loss of
precision which a compiler might rightfully complain about.
Use `size_t` instead to fix any warnings.
|
|
abf24a30
|
2019-06-27T15:25:17
|
|
examples: avoid conversion warnings when calculating progress
When computing the progress, we perform some arithmetics that are
implicitly converting from `size_t` to `int`. In one case we're
calclulating a percentage, so we know that it should always be in the
range of [0,100] and thus we're fine. In the other case we convert from
bytes to kilobytes -- this should be stored in a `size_t` to avoid loss
of precision, even though it probably won't matter due to limited
download rates.
|
|
e7bb1fe8
|
2019-06-27T15:14:08
|
|
examples: avoid passing signed integer to `memchr`
The memchr(3P) function expects a `size_t` as its last parameter, but we
do pass it an object size, which is of signed type `git_off_t`. As we
can be sure that the result will be non-negative, let's just cast the
parameter to a `size_t`.
|
|
976eed80
|
2019-06-27T15:12:11
|
|
examples: cast away constness for reallocating head arrays
When reallocating commit arrays in `opts_add_commit` and
`opts_add_refish`, respectively, we simply pass the const pointer to
`xrealloc`. As `xrealloc` expects a non-const pointer, though, this will
generate a warning with some compilers.
Cast away the constness to silence compilers.
|
|
dedf70ad
|
2019-07-05T09:35:43
|
|
patch_parse: do not depend on parsed buffer's lifetime
When parsing a patch from a buffer, we let the patch lines point into
the original buffer. While this is efficient use of resources, this also
ties the lifetime of the parsed patch to the parsed buffer. As this
behaviour is not documented anywhere in our API it is very surprising to
its users.
Untie the lifetime by duplicating the lines into the parsed patch. Add a
test that verifies that lifetimes are indeed independent of each other.
|
|
5c87b5a8
|
2019-07-04T12:19:07
|
|
Merge pull request #5152 from csware/attr-system-attr-file
Fix Regression: attr: Correctly load system attr file (on Windows)
|
|
c87abeca
|
2019-07-04T11:45:02
|
|
tests: attr: add tests for system-level attributes
There were no tests that verified that system-level gitattributes files
get handled correctly. In fact, we have recently introduced a regression
that caused us to abort if there was a system-level gitattributes file
available.
Add two tests that verify that we're able to handle system-level
gitattributes files. The test attr::repo::sysdir_with_session would've
failed without the fix to the described regression.
|
|
1bbec26d
|
2019-07-04T11:41:21
|
|
attr_file: completely initialize attribute sessions
The function `git_attr_session__init` is currently only initializing
setting up the attribute's session key by incrementing the repo-global
key by one. Most notably, all other members of the `git_attr_session`
struct are not getting initialized at all. So if one is to allocate a
session on the stack and then calls `git_attr_session__init`, the
session will still not be fully initialized. We have fared just fine
with that until now as all users of the function have allocated the
session structure as part of bigger structs with `calloc`, and thus its
contents have been zero-initialized implicitly already.
Fix this by explicitly zeroing out the session to enable allocation of
sessions on the stack.
|
|
18a6d9f3
|
2019-06-29T16:19:08
|
|
attr: Don't fail in attr_setup if there exists a system attributes file
Regression introduced in commit 5452e49fce21f726bec19519da7f012e3f19e736 on PR #4967.
Signed-off-by: Sven Strickroth <email@cs-ware.de>
|
|
c4c1500a
|
2019-06-27T14:18:19
|
|
Merge pull request #5145 from pks-t/pks/hash-algo-uninit-return
hash: fix missing error return on production builds
|
|
7fd3f32b
|
2019-06-27T13:54:55
|
|
hash: fix missing error return on production builds
When no hash algorithm has been initialized in a given hash context,
then we will simply `assert` and not return a value at all. This works
just fine in debug builds, but on non-debug builds the assert will be
converted to a no-op and thus we do not have a proper return value.
Fix this by returning an error code in addition to the asserts.
|
|
73427b94
|
2019-06-27T13:23:17
|
|
Merge pull request #5142 from scottfurry/StaticChkFixExamples
Resolve static check warnings in example code
|
|
2ba7dbbe
|
2019-06-24T14:55:15
|
|
Resolve static check warnings in example code
Using cppcheck on libgit2 sources indicated two warnings in
example code.
merge.c was reported as having a memory leak. Fix applied
was to `free()` memory pointed to by `parents`.
init.c was reported as having a null pointer dereference
on variable arg. Function 'usage' was being called with
a null variable. Changed supplied parameter to empty string.
|
|
e9102def
|
2019-06-27T11:38:04
|
|
Merge pull request #4438 from pks-t/pks/hash-algorithm
Multiple hash algorithms
|
|
b6625a3b
|
2019-06-27T10:12:16
|
|
Merge pull request #5128 from tiennou/fix/docs
More documentation
|
|
3d22394a
|
2019-06-27T10:11:23
|
|
Merge pull request #4967 from tiennou/fix/4671
Incomplete commondir support
|
|
33448b45
|
2019-06-19T19:46:12
|
|
docs: More of it
|
|
501c51b2
|
2019-06-26T14:49:50
|
|
repo: commondir resolution can sometimes fallback to the repodir
For example, https://git-scm.com/docs/gitrepository-layout says:
info
Additional information about the repository is recorded in this directory.
This directory is ignored if $GIT_COMMON_DIR is set
and "$GIT_COMMON_DIR/info" will be used instead.
So when looking for `info/attributes`, we need to check the commondir first,
or fallback to "our" `info/attributes`.
|
|
9f723c97
|
2019-06-26T14:49:37
|
|
docs: fixups
|
|
b883d370
|
2019-06-26T14:49:30
|
|
ignore: fix a missing commondir causing failures
As with the preceding commit, the ignore code tries to load code from
info/exclude, and we fail to ignore a non-existent file here.
|
|
82c7a9bc
|
2019-06-26T14:49:24
|
|
attr: fix attribute lookup if repo has no common directory
If creating a repository without a common directory (e.g. by
using `git_repository_new`), then `git_repository_item_path` will
return `GIT_ENOTFOUND` for every file that's usually located in
this directory. While we do not care for this case when looking
up the "info/attributes" file, we fail to properly ignore these
errors when setting up or collecting attributes files. Thus, the
gitattributes lookup is broken and will only ever return
`GIT_ENOTFOUND`.
Fix this issue by properly ignoring `GIT_ENOTFOUND` returned by
`git_repository_item_path`.
|
|
5452e49f
|
2019-06-26T14:49:17
|
|
attr: refactor setup to match current coding style
The code in the `attr_setup` function is not really matching our
current coding style. Besides alignment issues, it's also hard to
see what functions calls depend on one another because they're
split up over multiple conditional statements.
Fix these issues by grouping together dependent function calls
and adjusting the alignment.
|
|
b6b2d9d7
|
2019-06-25T15:05:23
|
|
examples: ssize_t is signed, not unsigned
|
|
cd67a903
|
2019-06-25T14:55:51
|
|
examples: cast away const-ness
|
|
1118dd9a
|
2019-06-25T14:50:12
|
|
examples: don't lose `const`
|
|
ede458b4
|
2019-06-25T14:48:10
|
|
example: use `git_off_t` for the object size
|
|
f48cf5b3
|
2019-06-25T14:46:31
|
|
w32_stack: treat a len as an size_t
|