|
d8896bda
|
2018-01-03T16:07:36
|
|
diff_generate: avoid excessive stats of .gitattribute files
When generating a diff between two trees, for each file that is to be
diffed we have to determine whether it shall be treated as text or as
binary files. While git has heuristics to determine which kind of diff
to generate, users can also that default behaviour by setting or
unsetting the 'diff' attribute for specific files.
Because of that, we have to query gitattributes in order to determine
how to diff the current files. Instead of hitting the '.gitattributes'
file every time we need to query an attribute, which can get expensive
especially on networked file systems, we try to cache them instead. This
works perfectly fine for every '.gitattributes' file that is found, but
we hit cache invalidation problems when we determine that an attribuse
file is _not_ existing. We do create an entry in the cache for missing
'.gitattributes' files, but as soon as we hit that file again we
invalidate it and stat it again to see if it has now appeared.
In the case of diffing large trees with each other, this behaviour is
very suboptimal. For each pair of files that is to be diffed, we will
repeatedly query every directory component leading towards their
respective location for an attributes file. This leads to thousands or
even hundreds of thousands of wasted syscalls.
The attributes cache already has a mechanism to help in that scenario in
form of the `git_attr_session`. As long as the same attributes session
is still active, we will not try to re-query the gitmodules files at all
but simply retain our currently cached results. To fix our problem, we
can create a session at the top-most level, which is the initialization
of the `git_diff` structure, and use it in order to look up the correct
diff driver. As the `git_diff` structure is used to generate patches for
multiple files at once, this neatly solves our problem by retaining the
session until patches for all files have been generated.
The fix has been tested with linux.git by calling
`git_diff_tree_to_tree` and `git_diff_to_buf` with v4.10^{tree} and
v4.14^{tree}.
| time | .gitattributes stats
without fix | 33.201s | 844614
with fix | 30.327s | 4441
While execution only improved by roughly 10%, the stat(3) syscalls for
.gitattributes files decreased by 99.5%. The benchmarks were quite
simple with best-of-three timings on Linux ext4 systems. One can assume
that for network based file systems the performance gain will be a lot
larger due to a much higher latency.
|
|
0c7f49dd
|
2017-06-30T13:39:01
|
|
Make sure to always include "common.h" first
Next to including several files, our "common.h" header also declares
various macros which are then used throughout the project. As such, we
have to make sure to always include this file first in all
implementation files. Otherwise, we might encounter problems or even
silent behavioural differences due to macros or defines not being
defined as they should be. So in fact, our header and implementation
files should make sure to always include "common.h" first.
This commit does so by establishing a common include pattern. Header
files inside of "src" will now always include "common.h" as its first
other file, separated by a newline from all the other includes to make
it stand out as special. There are two cases for the implementation
files. If they do have a matching header file, they will always include
this one first, leading to "common.h" being transitively included as
first file. If they do not have a matching header file, they instead
include "common.h" as first file themselves.
This fixes the outlined problems and will become our standard practice
for header and source files inside of the "src/" from now on.
|
|
b859faa6
|
2016-08-23T23:38:39
|
|
Teach `git_patch_from_diff` about parsed diffs
Ensure that `git_patch_from_diff` can return the patch for parsed diffs,
not just generate a patch for a generated diff.
|
|
7166bb16
|
2016-04-25T00:35:48
|
|
introduce `git_diff_from_buffer` to parse diffs
Parse diff files into a `git_diff` structure.
|
|
9be638ec
|
2016-04-19T15:12:18
|
|
git_diff_generated: abstract generated diffs
|
|
25e84f95
|
2015-11-23T15:49:54
|
|
checkout: only consider nsecs when built that way
When examining the working directory and determining whether it's
up-to-date, only consider the nanoseconds in the index entry when
built with `GIT_USE_NSEC`. This prevents us from believing that
the working directory is always dirty when the index was originally
written with a git client that uinderstands nsecs (like git 2.x).
|
|
c2e1b058
|
2015-06-05T18:26:49
|
|
Only write index if updated when passing GIT_DIFF_UPDATE_INDEX
When diffing the index with the workdir and GIT_DIFF_UPDATE_INDEX has been passed,
the previous implementation was always writing the index to disk even if it wasn't
modified.
|
|
90177111
|
2015-06-23T16:27:33
|
|
stash: save the workdir file when deleted in index
When stashing the workdir tree, examine the index as well. Using
a mechanism similar to `git_diff_tree_to_workdir_with_index`
allows us to determine that a file was added in the index and
subsequently modified in the working directory. Without examining
the index, we would erroneously believe that this file was
untracked and fail to include it in the working directory tree.
Use a slightly modified `git_diff_tree_to_workdir_with_index` in
order to avoid some of the behavior custom to `git diff`. In
particular, be sure to include the working directory side of a
file when it was deleted in the index.
|
|
5ef43d41
|
2015-06-23T10:29:59
|
|
git_diff__merge: allow pluggable diff merges
|
|
96dd171e
|
2015-06-19T08:32:26
|
|
diff: preserve original mode in the index
When updating the index during a diff, preserve the original mode,
which prevents us from dropping the mode to what we have interpreted
as on our system (eg, what the working directory claims it to be,
which may be a lie on some systems.)
|
|
9c8ed499
|
2014-04-29T15:05:58
|
|
Remove trace / add git_diff_perfdata struct + api
|
|
cd424ad5
|
2014-04-28T16:39:53
|
|
Add GIT_STATUS_OPT_UPDATE_INDEX and use trace API
This adds an option to refresh the stat cache while generating
status. It also rips out the GIT_PERF stuff I had an makes use
of the trace API to keep statistics about what happens during diff.
|
|
94fb4aad
|
2014-04-28T14:48:41
|
|
Add diff option to update index stat cache
When diff is scanning the working directory, if it finds a file
where it is not sure if the index entry matches the working dir,
it will recalculate the OID (which is pretty expensive). This
adds a new flag to diff so that if the OID calculation finds that
the file actually has not changed (i.e. just the modified time was
altered or such), then it will refresh the stat cache in the index
so that future calls to diff will not have to check the oid again.
|
|
0fc8e1f6
|
2014-04-28T14:34:55
|
|
Lay groundwork for updating stat cache in diff
This reorganized the diff OID calculation to make it easier to
correctly update the stat cache during a diff once the flags to
do so are enabled.
This includes marking the path of a git_index_entry as const so
we can make a "fake" git_index_entry with a "const char *" path
and not get warnings. I was a little surprised at how unobtrusive
this change was, but I think it's probably a good thing.
|
|
240f4af3
|
2014-04-28T14:04:29
|
|
Add build option for diff internal statistics
|
|
d8cc1fb6
|
2014-04-11T19:15:15
|
|
Introduce git_diff_format_email and git_diff_commit_as_email
|
|
9950bb4e
|
2014-01-24T20:23:17
|
|
diff: rename the file's 'oid' to 'id'
In the same vein as the previous commits in this series.
|
|
3e57069e
|
2013-11-01T13:49:43
|
|
Fix --assume-unchanged support
This was never really working right because we were checking the
wrong flag and not checking it in all the places that we need to
be checking it. I finally got around to writing a test and adding
actual support for it.
|
|
3ff1d123
|
2013-10-11T14:51:54
|
|
Rename diff objects and split patch.h
This makes no functional change to diff but renames a couple of
the objects and splits the new git_patch (formerly git_diff_patch)
into a new header file.
|
|
5e96f316
|
2013-08-08T08:54:38
|
|
Merge pull request #1738 from libgit2/diff-patch-content-size
Add API for getting at git_diff_patch->content_size
|
|
effdbeb3
|
2013-07-24T17:48:37
|
|
Make rename detection file size fix better
The previous fix for checking file sizes with rename detection
always loads the blob. In this version, if the odb backend can
get the object header without loading the whole thing into memory,
then we'll just use that, so that we can eliminate possible rename
sources & targets without loading them.
|
|
197b8966
|
2013-07-23T14:34:31
|
|
Add hunk/file headers to git_diff_patch_size
This allows git_diff_patch_size to account for hunk headers and
file headers in the returned size. This required some refactoring
of the code that is used to print file headers so that it could be
invoked by the git_diff_patch_size API.
Also this increases the test coverage and fixes an off-by-one bug
in the size calculation when newline changes happen at the end of
the file.
|
|
2b672d5b
|
2013-07-08T22:46:36
|
|
Add git_pathspec_match_diff API
This adds an additional pathspec API that will match a pathspec
against a diff object. This is convenient if you want to handle
renames (so you need the whole diff and can't use the pathspec
constraint built into the diff API) but still want to tell if the
diff had any files that matched the pathspec.
When the pathspec is matched against a diff, instead of keeping
a list of filenames that matched, instead the API keeps the list
of git_diff_deltas that matched and they can be retrieved via a
new API git_pathspec_match_list_diff_entry.
There are a couple of other minor API extensions here that were
mostly for the sake of convenience and to reduce dependencies
on knowing the internal data structure between files inside the
library.
|
|
351888cf
|
2013-06-13T15:37:06
|
|
Improve case handling in git_diff__paired_foreach
This commit reinstates some changes to git_diff__paired_foreach
that were discarded during the rebase (because the diff_output.c
file had gone away), and also adjusts the case insensitively
logic slightly to hopefully deal with either mismatched icase
diffs and other case insensitivity scenarios.
|
|
a1683f28
|
2013-06-14T16:18:04
|
|
More tests and bug fixes for status with rename
This changes the behavior of the status RENAMED flags so that they
will be combined with the MODIFIED flags if appropriate. If a file
is modified in the index and also renamed, then the status code
will have both the GIT_STATUS_INDEX_MODIFIED and INDEX_RENAMED bits
set. If it is renamed but the OID has not changed, then just the
GIT_STATUS_INDEX_RENAMED bit will be set. Similarly, the flags
GIT_STATUS_WT_MODIFIED and GIT_STATUS_WT_RENAMED can both be set
independently of one another.
This fixes a serious bug where the check for unmodified files that
was done at data load time could end up erasing the RENAMED state
of a file that was renamed with no changes.
Lastly, this contains a bunch of new tests for status with renames,
including tests where the only rename changes are case changes.
The expected results of these tests have to vary by whether the
platform uses a case sensitive filesystem or not, so the expected
data covers those platform differences separately.
|
|
114f5a6c
|
2013-06-10T10:10:39
|
|
Reorganize diff and add basic diff driver
This is a significant reorganization of the diff code to break it
into a set of more clearly distinct files and to document the new
organization. Hopefully this will make the diff code easier to
understand and to extend.
This adds a new `git_diff_driver` object that looks of diff driver
information from the attributes and the config so that things like
function content in diff headers can be provided. The full driver
spec is not implemented in the commit - this is focused on the
reorganization of the code and putting the driver hooks in place.
This also removes a few #includes from src/repository.h that were
overbroad, but as a result required extra #includes in a variety
of places since including src/repository.h no longer results in
pulling in the whole world.
|
|
67db583d
|
2013-05-23T15:06:07
|
|
More diff rename tests; better split swap handling
This adds a couple more tests of different rename scenarios.
Also, this fixes a problem with the case where you have two
"split" deltas and the left half of one matches the right half of
the other. That case was already being handled, but in the wrong
order in a way that could result in bad output. Also, if the swap
also happened to put the other two halves into the correct place
(i.e. two files exchanged places with each other), then the second
delta was left with the SPLIT flag set when it really should be
cleared.
|
|
a21cbb12
|
2013-05-22T10:37:12
|
|
Significant rename detection rewrite
This flips rename detection around so instead of creating a
forward mapping from deltas to possible rename targets, instead
it creates a reverse mapping, looking at possible targets and
trying to find a source that they could have been renamed or
copied from. This is important because each output can only
have a single source, but a given source could map to multiple
outputs (in the form of COPIED records).
Additionally, this makes a couple of tweaks to the public rename
detection APIs, mostly renaming a couple of options that control
the behavior to make more sense and to be more like core Git.
I walked through the tests looking at the exact results and
updated the expectations based on what I saw. The new code is
different from the old because it cannot give some nonsense
results (like A was renamed to both B and C) which were part of
the outputs previously.
|
|
e35e2684
|
2013-05-07T04:32:17
|
|
Add GIT_DIFF_LINE_CONTEXT_EOFNL
This adds a new line origin constant for the special line that
is used when both files end without a newline.
In the course of writing the tests for this, I was having problems
with modifying a file but not having diff notice because it was
the same size and modified less than one second from the start of
the test, so I decided to start working on nanosecond timestamp
support. This commit doesn't contain the nanosecond support, but
it contains the reorganization of maybe_modified and the hooks so
that if the nanosecond data were being read by stat() (or rather
being copied by git_index_entry__init_from_stat), then the nsec
would be taken into account.
This new stuff could probably use some more tests, although there
is some amount of it here.
|
|
0462fba5
|
2013-04-30T14:56:41
|
|
renames!
|
|
71a3d27e
|
2013-02-08T10:06:47
|
|
Replace diff delta binary with flags
Previously the git_diff_delta recorded if the delta was binary.
This replaces that (with no net change in structure size) with
a full set of flags. The flag values that were already in use
for individual git_diff_file objects are reused for the delta
flags, too (along with renaming those flags to make it clear that
they are used more generally).
This (a) makes things somewhat more consistent (because I was
using a -1 value in the "boolean" binary field to indicate unset,
whereas now I can just use the flags that are easier to understand),
and (b) will make it easier for me to add some additional flags to
the delta object in the future, such as marking the results of a
copy/rename detection or other deltas that might want a special
indicator.
While making this change, I officially moved some of the flags that
were internal only into the private diff header.
This also allowed me to remove a gross hack in rename/copy detect
code where I was overwriting the status field with an internal
value.
|
|
359fc2d2
|
2013-01-08T17:07:25
|
|
update copyrights
|
|
9950d27a
|
2012-12-06T13:26:58
|
|
Clean up iterator APIs
This removes the need to explicitly pass the repo into iterators
where the repo is implied by the other parameters. This moves
the repo to be owned by the parent struct. Also, this has some
iterator related updates to the internal diff API to lay the
groundwork for checkout improvements.
|
|
c7231c45
|
2012-11-30T16:31:42
|
|
Deploy GITERR_CHECK_VERSION
|
|
2f8d30be
|
2012-11-29T15:05:04
|
|
Deploy GIT_DIFF_OPTIONS_INIT
|
|
bbe6dbec
|
2012-11-14T23:29:48
|
|
Add explicit git_index ptr to diff and checkout
A number of diff APIs and the `git_checkout_index` API take a
`git_repository` object an operate on the index. This updates
them to take a `git_index` pointer explicitly and only fall back
on the `git_repository` index if the index input is NULL. This
makes it easier to operate on a temporary index.
|
|
bad68c0a
|
2012-11-13T14:02:59
|
|
Add iterator for git_index object
The index iterator could previously only be created from a repo
object, but this allows creating an iterator from a `git_index`
object instead (while keeping, though renaming, the old function).
|
|
55cbd05b
|
2012-11-08T16:56:34
|
|
Some diff refactorings to help code reuse
There are some diff functions that are useful in a rewritten
checkout and this lays some groundwork for that. This contains
three main things:
1. Share the function diff uses to calculate the OID for a file
in the working directory (now named `git_diff__oid_for_file`
2. Add a `git_diff__paired_foreach` function to iterator over
two diff lists concurrently. Convert status to use it.
3. Move all the string/prefix/index entry comparisons into
function pointers inside the `git_diff_list` object so they
can be switched between case sensitive and insensitive
versions. This makes them easier to reuse in various
functions without replicating logic. As part of this, move
a couple of index functions out of diff.c and into index.c.
|
|
0f3def71
|
2012-11-09T11:19:46
|
|
Fix various cross-platform build issues
This fixes a number of warnings and problems with cross-platform
builds. Among other things, it's not safe to name a member of a
structure "strcmp" because that may be #defined.
|
|
db106d01
|
2012-10-30T09:40:50
|
|
Move rename detection into new file
This improves the naming for the rename related functionality
moving it to be called `git_diff_find_similar()` and renaming
all the associated constants, etc. to make more sense.
I also moved the new code (plus the existing `git_diff_merge`)
into a new file `diff_tform.c` where I can put new functions
related to manipulating git diff lists.
This also updates the implementation significantly from the
last revision fixing some ordering issues (where break-rewrite
needs to be handled prior to copy and rename detection) and
improving config option handling.
|
|
b4f5bb07
|
2012-10-23T16:40:51
|
|
Initial implementation of diff rename detection
This implements the basis for diff rename and copy detection,
although it is based on simple SHA comparison right now instead
of using a matching algortihm. Just as `git_diff_merge` can be
used as a post-pass on diffs to emulate certain command line
behaviors, there is a new API `git_diff_detect` which will
update a diff list in-place, adjusting some deltas to RENAMED
or COPIED state (and also, eventually, splitting MODIFIED deltas
where the change is too large into DELETED/ADDED pairs).
This also adds a new test repo that will hold rename/copy/split
scenarios. Right now, it just has exact-match rename and copy,
but the tests are written to use tree diffs, so we should be able
to add new test scenarios easily without breaking tests.
|
|
52a61bb8
|
2012-10-17T14:10:23
|
|
Fix minor bugs
Fixed no-submodule speedup of new checkout code. Fixed missing
final update to progress (which may go away, I realize). Fixed
unused structure in header and incorrect comment.
|
|
5f69a31f
|
2012-09-24T20:52:34
|
|
Initial implementation of new diff patch API
Replacing the `git_iterator` object, this creates a simple API
for accessing the "patch" for any file pair in a diff list and
then gives indexed access to the hunks in the patch and the lines
in the hunk. This is the initial implementation of this revised
API - it is still broken, but at least builds cleanly.
|
|
64286308
|
2012-09-25T10:48:50
|
|
Fix bugs in new diff patch code
This fixes all the bugs in the new diff patch code. The only
really interesting one is that when we merge two diffs, we now
have to actually exclude diff delta records that are not supposed
to be tracked, as opposed to before where they could be included
because they would be skipped silently by `git_diff_foreach()`.
Other than that, there are just minor errors.
|
|
bae957b9
|
2012-09-25T16:31:46
|
|
Add const to all shared pointers in diff API
There are a lot of places where the diff API gives the user access
to internal data structures and many of these were being exposed
through non-const pointers. This replaces them all with const
pointers for any object that the user can access but is still
owned internally to the git_diff_list or git_diff_patch objects.
This will probably break some bindings... Sorry!
|
|
b36effa2
|
2012-09-10T09:59:14
|
|
Replace git_diff_iterator_num_files with progress
The `git_diff_iterator_num_files` API was problematic, since we
don't actually know the exact number of files to be iterated over
until we load those files into memory. This replaces it with a
new `git_diff_iterator_progress` API that goes from 0 to 1, and
moves and renamed the old API for the internal places that can
tolerate a max value instead of an exact value.
|
|
60b9d3fc
|
2012-09-05T15:00:40
|
|
Implement filters for status/diff blobs
This adds support to diff and status for running filters (a la crlf)
on blobs in the workdir before computing SHAs and before generating
text diffs. This ended up being a bit more code change than I had
thought since I had to reorganize some of the diff logic to minimize
peak memory use when filtering blobs in a diff.
This also adds a cap on the maximum size of data that will be loaded
to diff. I set it at 512Mb which should match core git. Right now
it is a #define in src/diff.h but it could be moved into the public
API if desired.
|
|
f335ecd6
|
2012-08-30T14:24:16
|
|
Diff iterators
This refactors the diff output code so that an iterator object
can be used to traverse and generate the diffs, instead of just
the `foreach()` style with callbacks. The code has been rearranged
so that the two styles can still share most functions.
This also replaces `GIT_REVWALKOVER` with `GIT_ITEROVER` and uses
that as a common error code for marking the end of iteration when
using a iterator style of object.
|
|
0abd7244
|
2012-06-04T16:17:41
|
|
Fix filemode comparison in diffs
File modes were both not being ignored properly on platforms
where they should be ignored, nor be diffed consistently on
platforms where they are supported.
This change adds a number of diff and status filemode change
tests. This also makes sure that filemode-only changes are
included in the diff output when they occur and that filemode
changes are ignored successfully when core.filemode is false.
There is no code that automatically toggles core.filemode
based on the capabilities of the current platform, so the user
still needs to be careful in their .git/config file.
|
|
16b83019
|
2012-03-04T23:28:36
|
|
Fix usage of "new" for fieldname in public header
This should restore the ability to include libgit2 headers
in C++ projects.
Cherry picked 2de60205dfea2c4a422b2108a5e8605f97c2e895 from
development into new-error-handling.
|
|
eb3d71a5
|
2012-04-25T22:23:35
|
|
diff: fix generation of the header of a removal patch
|
|
19fa2bc1
|
2012-04-17T15:12:50
|
|
Convert attrs and diffs to use string pools
This converts the git attr related code (including ignores) and
the git diff related code (and implicitly the status code) to use
`git_pools` for storing strings. This reduces the number of small
blocks allocated dramatically.
|
|
14a513e0
|
2012-04-13T15:00:29
|
|
Add support for pathspec to diff and status
This adds preliminary support for pathspecs to diff and status.
The implementation is not very optimized (it still looks at
every single file and evaluated the the pathspec match against
them), but it works.
|
|
95dfb031
|
2012-03-30T14:40:50
|
|
Improve config handling for diff,submodules,attrs
This adds support for a bunch of core.* settings that affect
diff and status, plus fixes up some incorrect implementations
of those settings from before. Also, this cleans up the
handling of config settings in the new submodules code and
in the old attrs/ignore code.
|
|
1db12b00
|
2012-03-25T23:04:26
|
|
Eliminate hairy COITERATE macro
I decided that the COITERATE macro was, in the end causing
more confusion that it would save and decided just to write
out the loops that I needed for parallel diff list iteration.
It is not that much code and this just feels less obfuscated.
|
|
a48ea31d
|
2012-03-21T12:33:09
|
|
Reimplment git_status_foreach using git diff
This is an initial reimplementation of status using diff a la
the way that core git does it.
|
|
74fa4bfa
|
2012-02-28T16:14:47
|
|
Update diff to use iterators
This is a major reorganization of the diff code. This changes
the diff functions to use the iterators for traversing the
content. This allowed a lot of code to be simplified. Also,
this moved the functions relating to outputting a diff into a
new file (diff_output.c).
This includes a number of other changes - adding utility
functions, extending iterators, etc. plus more tests for the
diff code. This also takes the example diff.c program much
further in terms of emulating git-diff command line options.
|
|
e47329b6
|
2012-02-13T17:29:30
|
|
First pass of diff index to workdir implementation
This is an initial version of git_diff_workdir_to_index. It
also includes renaming some structures and some refactoring
of the existing code so that it could be shared better with
the new function.
This is not complete since it needs a rebase to get some
new odb functions from the upstream branch.
|
|
a2e895be
|
2012-02-07T12:14:28
|
|
Continue implementation of git-diff
* Implemented git_diff_index_to_tree
* Reworked git_diff_options structure to handle more options
* Made most of the options in git_diff_options actually work
* Reorganized code a bit to remove some redundancy
* Added option parsing to examples/diff.c to test most options
|
|
65b09b1d
|
2012-02-02T18:03:43
|
|
Implement diff lists and formatters
This reworks the diff API to separate the steps of producing
a diff descriptions from formatting the diff. This will allow
us to share diff output code with the various diff creation
scenarios and will allow us to implement rename detection as
an optional pass that can be run on a diff list.
|