|
3e6533ba
|
2017-12-10T17:25:00
|
|
odb_loose: reject objects that cannot fit in memory
Check the size of objects being read from the loose odb backend and
reject those that would not fit in memory with an error message that
reflects the actual problem, instead of error'ing later with an
unintuitive error message regarding truncation or invalid hashes.
|
|
ddefea75
|
2017-11-30T15:55:59
|
|
odb: support large loose objects
zlib will only inflate/deflate an `int`s worth of data at a time.
We need to loop through large files in order to ensure that we inflate
the entire file, not just an `int`s worth of data. Thankfully, we
already have this loop in our `git_zstream` layer. Handle large objects
using the `git_zstream`.
|
|
0c7f49dd
|
2017-06-30T13:39:01
|
|
Make sure to always include "common.h" first
Next to including several files, our "common.h" header also declares
various macros which are then used throughout the project. As such, we
have to make sure to always include this file first in all
implementation files. Otherwise, we might encounter problems or even
silent behavioural differences due to macros or defines not being
defined as they should be. So in fact, our header and implementation
files should make sure to always include "common.h" first.
This commit does so by establishing a common include pattern. Header
files inside of "src" will now always include "common.h" as its first
other file, separated by a newline from all the other includes to make
it stand out as special. There are two cases for the implementation
files. If they do have a matching header file, they will always include
this one first, leading to "common.h" being transitively included as
first file. If they do not have a matching header file, they instead
include "common.h" as first file themselves.
This fixes the outlined problems and will become our standard practice
for header and source files inside of the "src/" from now on.
|
|
6c23704d
|
2017-06-08T21:40:18
|
|
settings: rename `GIT_OPT_ENABLE_SYNCHRONOUS_OBJECT_CREATION`
Initially, the setting has been solely used to enable the use of
`fsync()` when creating objects. Since then, the use has been extended
to also cover references and index files. As the option is not yet part
of any release, we can still correct this by renaming the option to
something more sensible, indicating not only correlation to objects.
This commit renames the option to `GIT_OPT_ENABLE_FSYNC_GITDIR`. We also
move the variable from the object to repository source code.
|
|
1dc89aab
|
2017-05-01T21:34:21
|
|
object validation: free some memleaks
|
|
2a5ad7d0
|
2017-02-17T16:42:40
|
|
fsync: call it "synchronous" object writing
Rename `GIT_OPT_ENABLE_SYNCHRONIZED_OBJECT_CREATION` ->
`GIT_OPT_ENABLE_SYNCHRONOUS_OBJECT_CREATION`.
|
|
6d3ad7e0
|
2016-12-13T10:58:43
|
|
Add `ENABLE_SYNCHRONIZED_OBJECT_CREATION` option
Allow users to enable `SYNCHRONIZED_OBJECT_CREATION` with a setting.
|
|
fc27fe21
|
2016-12-13T10:35:05
|
|
odb_loose: actually honor the fsync option
We've had an fsync option for a long time, but it was "ignored".
Stop ignoring it.
|
|
909d5494
|
2016-12-29T12:25:15
|
|
giterr_set: consistent error messages
Error messages should be sentence fragments, and therefore:
1. Should not begin with a capital letter,
2. Should not conclude with punctuation, and
3. Should not end a sentence and begin a new one
|
|
27051d4e
|
2016-07-22T13:34:19
|
|
odb: only freshen pack files every 2 seconds
Since writing multiple objects may all already exist in a single
packfile, avoid freshening that packfile repeatedly in a tight loop.
Instead, only freshen pack files every 2 seconds.
|
|
8f09a98e
|
2016-07-14T16:23:24
|
|
odb: freshen existing objects when writing
When writing an object, we calculate its OID and see if it exists in the
object database. If it does, we need to freshen the file that contains
it.
|
|
6a2d2f8a
|
2015-06-17T06:42:20
|
|
delta: move delta application to delta.c
Move the delta application functions into `delta.c`, next to the
similar delta creation functions. Make the `git__delta_apply`
functions adhere to other naming and parameter style within the
library.
|
|
7f407710
|
2016-05-02T16:24:14
|
|
odb_loose: fix undefined behavior when computing size
An object's size is computed by reading the object header's size
field until the most significant bit is not set anymore. To get
the total size, we increase the shift on each iteration and add
the shifted value to the total size.
We read the current value into a variable of type `unsigned
char`, from which we then take all bits except the most
significant bit and shift the result. We will end up with a
maximum shift of 60, but this exceeds the width of the value's
type, resulting in undefined behavior.
Fix the issue by instead reading the values into a variable of
type `unsigned long`, which matches the required width. This is
equivalent to git.git, which uses an `unsigned long` as well.
|
|
e10144ae
|
2016-03-04T01:18:30
|
|
odb: improved not found error messages
When looking up an abbreviated oid, show the actual (abbreviated) oid
the caller passed instead of a full (but ambiguously truncated) oid.
|
|
ac2fba0e
|
2015-09-16T15:07:27
|
|
git_futils_mkdir_*: make a relative-to-base mkdir
Untangle git_futils_mkdir from git_futils_mkdir_ext - the latter
assumes that we own everything beneath the base, as if it were
being called with a base of the repository or working directory,
and is tailored towards checkout and ensuring that there is no
bogosity beneath the base that must be cleaned up.
This is (at best) slow and (at worst) unsafe in the larger context
of a filesystem where we do not own things and cannot do things like
unlink symlinks that are in our way.
|
|
77b339f7
|
2015-05-12T13:06:33
|
|
odb: make the writestream's size a git_off_t
Restricting files to size_t is a silly limitation. The loose backend
writes to a file directly, so there is no issue in using 63 bits for the
size.
We still assume that the header is going to fit in 64 bytes, which does
mean quite a bit smaller files due to the run-length encoding, but it's
still a much larger size than you would want Git to handle.
|
|
f1453c59
|
2015-02-12T12:19:37
|
|
Make our overflow check look more like gcc/clang's
Make our overflow checking look more like gcc and clang's, so that
we can substitute it out with the compiler instrinsics on platforms
that support it. This means dropping the ability to pass `NULL` as
an out parameter.
As a result, the macros also get updated to reflect this as well.
|
|
392702ee
|
2015-02-09T23:41:13
|
|
allocations: test for overflow of requested size
Introduce some helper macros to test integer overflow from arithmetic
and set error message appropriately.
|
|
b874629b
|
2014-12-04T21:06:59
|
|
Spelling fixes
|
|
3b2cb2c9
|
2014-09-16T11:49:25
|
|
Factor 40 and 41 constants from source.
|
|
ee311907
|
2014-05-05T16:04:14
|
|
odb: ignore files in the objects dir
We assume that everything under GIT_DIR/objects/ is a directory. This is
not necessarily the case if some process left a stray file in there.
Check beforehand if we do have a directory and ignore the entry
otherwise.
|
|
26875825
|
2014-03-05T13:06:22
|
|
Check short OID len in odb, not in backends
|
|
f5753999
|
2014-03-04T15:34:23
|
|
Add exists_prefix to ODB backend and ODB API
|
|
26c1cb91
|
2013-12-09T09:44:03
|
|
One more rename/cleanup for callback err functions
|
|
25e0b157
|
2013-12-06T15:07:57
|
|
Remove converting user error to GIT_EUSER
This changes the behavior of callbacks so that the callback error
code is not converted into GIT_EUSER and instead we propagate the
return value through to the caller. Instead of using the
giterr_capture and giterr_restore functions, we now rely on all
functions to pass back the return value from a callback.
To avoid having a return value with no error message, the user
can call the public giterr_set_str or some such function to set
an error message. There is a new helper 'giterr_set_callback'
that functions can invoke after making a callback which ensures
that some error message was set in case the callback did not set
one.
In places where the sign of the callback return value is
meaningful (e.g. positive to skip, negative to abort), only the
negative values are returned back to the caller, obviously, since
the other values allow for continuing the loop.
The hardest parts of this were in the checkout code where positive
return values were overloaded as meaningful values for checkout.
I fixed this by adding an output parameter to many of the internal
checkout functions and removing the overload. This added some
code, but it is probably a better implementation.
There is some funkiness in the network code where user provided
callbacks could be returning a positive or a negative value and
we want to rely on that to cancel the loop. There are still a
couple places where an user error might get turned into GIT_EUSER
there, I think, though none exercised by the tests.
|
|
96869a4e
|
2013-12-03T16:45:39
|
|
Improve GIT_EUSER handling
This adds giterr_user_cancel to return GIT_EUSER and clear any
error message that is sitting around. As a result of using that
in places, we need to be more thorough with capturing errors that
happen inside a callback when used internally. To help with that,
this also adds giterr_capture and giterr_restore so that when we
internally use a foreach-type function that clears errors and
converts them to GIT_EUSER, it is easier to restore not just the
return value, but the actual error message text.
|
|
1d3a8aeb
|
2013-11-04T18:28:57
|
|
move mode_t to filebuf_open instead of _commit
|
|
f966acd1
|
2013-11-04T15:46:32
|
|
Take umask into account in filebuf_commit
|
|
dd64c71c
|
2013-11-04T14:50:25
|
|
Allow backend consumers to specify file mode
|
|
219d3457
|
2013-10-01T16:12:15
|
|
Initial iconv hookup for precomposed unicode
This hooks up git_path_direach and git_path_dirload so that they
will take a flag indicating if directory entry names should be
tested and converted from decomposed unicode to precomposed form.
This code will only come into play on the Apple platform and even
then, only when certain types of filesystems are used.
This involved adding a flag to these functions which involved
changing a lot of places in the code.
This was an opportunity to do a bit of code cleanup here and there,
for example, getting rid of the git_futils_cleanupdir_r function in
favor of a simple flag to git_futils_rmdir_r to not remove the top
level entry. That ended up adding depth tracking during rmdir_r
which led to a safety check for infinite directory recursion. Yay.
This hasn't actually been tested on the Mac filesystems where the
issue occurs. I still need to get test environment for that.
|
|
209f9b67
|
2013-09-08T18:25:17
|
|
odb: Teach loose backend to return EAMBIGUOUS
|
|
4047950f
|
2013-08-29T14:19:34
|
|
odb: Prevent stream_finalize_write() from overwriting
Now that #1785 is merged, git_odb_stream_finalize_write() calculates the object id before invoking the odb backend.
This commit gives a chance to the backend to check if it already knows this object.
|
|
520287f6
|
2013-08-19T02:17:00
|
|
Merge pull request #1785 from libgit2/cmn/odb-hash-frontend
odb: move hashing to the frontend for streaming
|
|
d19dd9cf
|
2013-08-18T23:38:51
|
|
odb: Straighten oid prefix handling
|
|
fe0c6d4e
|
2013-08-17T01:41:08
|
|
odb: make it clearer that the id is calculated in the frontend
The frontend is in charge of calculating the id of the objects. Thus
the backends should treat it as a read-only value. The positioning in
the function signature made it seem as though it was an output
parameter.
Make the id const and move it from the front to behind the subject
(backend or stream).
|
|
d4e6cf0c
|
2013-08-15T14:32:47
|
|
odb: remove a duplicate object header formatting function
|
|
8380b39a
|
2013-08-15T14:29:39
|
|
odb: perform the stream hashing in the frontend
Hash the data as it's coming into the stream and tell the backend what
its name is when finalizing the write. This makes it consistent with
the way a plain git_odb_write() performs the write.
|
|
8294e8cf
|
2013-06-22T17:15:31
|
|
Constrain mkdir calls to avoid extra mkdirs
This updates the calls that make the subdirectories for objects
to use a base directory above which git_futils_mkdir won't walk
any higher. This prevents attempts to mkdir all the way up to
the root of the filesystem.
Also, this moves the objects_dir into the loose backend structure
and removes the separate allocation, plus does some preformatting
of the objects_dir value to guarantee a trailing slash, etc.
|
|
83cc70d9
|
2013-04-19T12:48:33
|
|
Move odb_backend implementors stuff into git2/sys
This moves some of the odb_backend stuff that is related to the
internals of an odb_backend implementation into include/git2/sys.
Some of the stuff related to streaming I left in include/git2
because it seemed like it would be reasonably needed by a normal
user who wanted to stream objects into and out of the ODB.
Also, I added APIs for traversing the list of backends so that
some of the tests would not need to access ODB internals.
|
|
359fc2d2
|
2013-01-08T17:07:25
|
|
update copyrights
|
|
4d185dd9
|
2012-12-19T14:30:06
|
|
odb: check if object exists before writing
Update the procondition of git_odb_backend::write.
It may now be assumed that the object has already been hashed.
|
|
55f6f21b
|
2012-11-29T19:59:18
|
|
Deploy versioned git_odb_backend structure
|
|
c3fb7d04
|
2012-11-27T15:00:49
|
|
Make git_odb_foreach_cb take const param
This makes the first OID param of the ODB callback a const pointer
and also propogates that change all the way to the backends.
|
|
51e1d808
|
2012-08-06T12:41:08
|
|
Merge remote-tracking branch 'arrbee/tree-walk-fixes' into development
Conflicts:
src/notes.c
src/transports/git.c
src/transports/http.c
src/transports/local.c
tests-clar/odb/foreach.c
|
|
5dca2010
|
2012-08-03T17:08:01
|
|
Update iterators for consistency across library
This updates all the `foreach()` type functions across the library
that take callbacks from the user to have a consistent behavior.
The rules are:
* A callback terminates the loop by returning any non-zero value
* Once the callback returns non-zero, it will not be called again
(i.e. the loop stops all iteration regardless of state)
* If the callback returns non-zero, the parent fn returns GIT_EUSER
* Although the parent returns GIT_EUSER, no error will be set in
the library and `giterr_last()` will return NULL if called.
This commit makes those changes across the library and adds tests
for most of the iteration APIs to make sure that they follow the
above rules.
|
|
b8457baa
|
2012-07-24T07:57:58
|
|
portability: Improve x86/amd64 compatibility
|
|
b7158c53
|
2012-07-12T20:48:46
|
|
Use GIT_INLINE instead of inline
|
|
521aedad
|
2012-06-05T14:48:51
|
|
odb: add git_odb_foreach()
Go through each backend and list every objects that exists in
them. This allows fsck-like uses.
|
|
904b67e6
|
2012-05-18T01:48:50
|
|
errors: Rename error codes
|
|
e172cf08
|
2012-05-18T01:21:06
|
|
errors: Rename the generic return codes
|
|
282283ac
|
2012-05-04T16:46:46
|
|
Fix valgrind issues
There are three changes here:
- correctly propogate error code from failed object lookups
- make zlib inflate use our allocators
- add OID to notfound error in ODB lookups
|
|
fa6420f7
|
2012-04-29T21:46:33
|
|
buf: deploy git_buf_len()
|
|
8e8b6b01
|
2012-04-04T13:13:43
|
|
Clean up valgrind warnings
|
|
0d0fa7c3
|
2012-03-16T15:56:01
|
|
Convert attr, ignore, mwindow, status to new errors
Also cleaned up some previously converted code that still had
little things to polish.
|
|
e1de726c
|
2012-03-12T22:55:40
|
|
Migrate ODB files to new error handling
This migrates odb.c, odb_loose.c, odb_pack.c and pack.c to
the new style of error handling. Also got the unix and win32
versions of map.c. There are some minor changes to other
files but no others were completely converted.
This also contains an update to filebuf so that a zeroed out
filebuf will not think that the fd (== 0) is actually open
(and inadvertently call close() on fd 0 if cleaned up).
Lastly, this was built and tested on win32 and contains a
bunch of fixes for the win32 build which was pretty broken.
|
|
cb8a7961
|
2012-03-07T00:02:55
|
|
error-handling: Repository
This also includes droping `git_buf_lasterror` because it makes no sense
in the new system. Note that in most of the places were it has been
dropped, the code needs cleanup. I.e. GIT_ENOMEM is going away, so
instead it should return a generic `-1` and obviously not throw
anything.
|
|
9d160ba8
|
2012-03-06T01:37:56
|
|
diff: Fix rebase breackage
|
|
1a481123
|
2012-02-17T00:13:34
|
|
error-handling: References
Yes, this is error handling solely for `refs.c`, but some of the
abstractions leak all ofer the code base.
|
|
13224ea4
|
2012-02-27T04:28:31
|
|
buffer: Unify `git_fbuffer` and `git_buf`
This makes so much sense that I can't believe it hasn't been done
before. Kill the old `git_fbuffer` and read files straight into
`git_buf` objects.
Also: In order to fully support 4GB files in 32-bit systems, the
`git_buf` implementation has been changed from using `ssize_t` for
storage and storing negative values on allocation failure, to using
`size_t` and changing the buffer pointer to a magical pointer on
allocation failure.
Hopefully this won't break anything.
|
|
0c3bae62
|
2012-02-15T16:56:56
|
|
zlib: Remove custom `git2/zlib.h` header
This is legacy compat stuff for when `deflateBound` is not defined, but
we're not embedding zlib and that function is always available. Kill
that with fire.
|
|
5e0de328
|
2012-02-13T17:10:24
|
|
Update Copyright header
Signed-off-by: schu <schu-github@schulog.org>
|
|
1744fafe
|
2012-01-17T15:49:47
|
|
Move path related functions from fileops to path
This takes all of the functions that look up simple data about
paths (such as `git_futils_isdir`) and moves them over to path.h
(becoming `git_path_isdir`). This leaves fileops.h just with
functions that actually manipulate the filesystem or look at
the file contents in some way.
As part of this, the dir.h header which is really just for win32
support was moved into win32 (with some minor changes).
|
|
97769280
|
2011-11-30T11:27:15
|
|
Use git_buf for path storage instead of stack-based buffers
This converts virtually all of the places that allocate GIT_PATH_MAX
buffers on the stack for manipulating paths to use git_buf objects
instead. The patch is pretty careful not to touch the public API
for libgit2, so there are a few places that still use GIT_PATH_MAX.
This extends and changes some details of the git_buf implementation
to add a couple of extra functions and to make error handling easier.
This includes serious alterations to all the path.c functions, and
several of the fileops.c ones, too. Also, there are a number of new
functions that parallel existing ones except that use a git_buf
instead of a stack-based buffer (such as git_config_find_global_r
that exists alongsize git_config_find_global).
This also modifies the win32 version of p_realpath to allocate whatever
buffer size is needed to accommodate the realpath instead of hardcoding
a GIT_PATH_MAX limit, but that change needs to be tested still.
|
|
b762e576
|
2011-11-17T15:10:27
|
|
filebuf: add GIT_FILEBUF_INIT and protect multiple opens and cleanups
Update all stack allocations of git_filebuf to use GIT_FILEBUF_INIT
and make git_filebuf_open and git_filebuf_cleanup safe to be called
multiple times on the same buffer.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
|
|
472d4d85
|
2011-11-17T20:32:04
|
|
Don't overwrite existing objects
It's redundant to do this (git doesn't) and Windows doesn't allow us
to overwrite a read-only file (which objects are).
Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>
|
|
89fb8f02
|
2011-10-28T19:04:23
|
|
Merge pull request #456 from brodie/perm-fixes
Create objects, indexes, and directories with the right file permissions
|
|
3286c408
|
2011-10-28T14:51:13
|
|
global: Properly use `git__` memory wrappers
Ensure that all memory related functions (malloc, calloc, strdup, free,
etc) are using their respective `git__` wrappers.
|
|
c51065e3
|
2011-10-24T14:39:03
|
|
Tolerate zlib deflation with window size < 32Kb
libgit2 currently identifies loose objects as corrupt if they've been
deflated using a window size less than 32Kb, because the
is_zlib_compressed_data() function doesn't recognise the header
byte as a zlib header. This patch makes the method tolerant of
all valid window sizes (15-bit to 8-bit) - but doesn't sacrifice
it's accuracy in distingushing the standard loose-object format
from the experimental (now abandoned) format. It's based on a patch
which has been merged into C-Git master branch:
https://github.com/git/git/commit/7f684a2aff636f44a506
On memory constrained systems zlib may use a much smaller window
size - working on Agit, I found that Android uses a 4KB window;
giving a header byte of 0x48, not 0x78. Consequently all loose
objects generated by the Android platform appear 'corrupt' :(
It might appear that this patch changes isStandardFormat() to the
point where it could incorrectly identify the experimental format as
the standard one, but the two criteria (bitmask & checksum) can only
give a false result for an experimental object where both of the
following are true:
1) object size is exactly 8 bytes when uncompressed (bitmask)
2) [single-byte in-pack git type&size header] * 256
+ [1st byte of the following zlib header] % 31 = 0 (checksum)
As it happens, for all possible combinations of valid object type
(1-4) and window bits (0-7), the only time when the checksum will be
divisible by 31 is for 0x1838 - ie object type *1*, a Commit - which,
due the fields all Commit objects must contain, could never be as
small as 8 bytes in size.
Given this, the combination of the two criteria (bitmask & checksum)
always correctly determines the buffer format, and is more tolerant
than the previous version.
References:
Android uses a 4KB window for deflation:
http://android.git.kernel.org/?p=platform/libcore.git;a=blob;f=luni/src/main/native/java_util_zip_Deflater.cpp;h=c0b2feff196e63a7b85d97cf9ae5bb258
Code snippet searching for false positives with the zlib checksum:
https://gist.github.com/1118177
Change-Id: Ifd84cd2bd6b46f087c9984fb4cbd8309f483dec0
|
|
01ad7b3a
|
2011-09-06T15:48:45
|
|
*: correct and codify various file permissions
The following files now have 0444 permissions:
- loose objects
- pack indexes
- pack files
- packs downloaded by fetch
- packs downloaded by the HTTP transport
And the following files now have 0666 permissions:
- config files
- repository indexes
- reflogs
- refs
This brings libgit2 more in line with Git.
Note that git_filebuf_commit() and git_filebuf_commit_at() have both
gained a new mode parameter.
The latter change fixes an important issue where filebufs created with
GIT_FILEBUF_TEMPORARY received 0600 permissions (due to mkstemp(3)
usage). Now we chmod() the file before renaming it into place.
Tests have been added to confirm that new commit, tag, and tree
objects are created with the right permissions. I don't have access to
Windows, so for now I've guarded the tests with "#ifndef GIT_WIN32".
|
|
ce8cd006
|
2011-09-07T15:32:44
|
|
fileops/repository: create (most) directories with 0777 permissions
To further match how Git behaves, this change makes most of the
directories libgit2 creates in a git repo have a file mode of
0777. Specifically:
- Intermediate directories created with git_futils_mkpath2file() have
0777 permissions. This affects odb_loose, reflog, and refs.
- The top level folder for bare repos is created with 0777
permissions.
- The top level folder for non-bare repos is created with 0755
permissions.
- /objects/info/, /objects/pack/, /refs/heads/, and /refs/tags/ are
created with 0777 permissions.
Additionally, the following changes have been made:
- fileops functions that create intermediate directories have grown a
new dirmode parameter. The only exception to this is filebuf's
lock_file(), which unconditionally creates intermediate directories
with 0777 permissions when GIT_FILEBUF_FORCE is set.
- The test runner now sets the umask to 0 before running any
tests. This ensurses all file mode checks are consistent across
systems.
- t09-tree.c now does a directory permissions check. I've avoided
adding this check to other tests that might reuse existing
directories from the prefabricated test repos. Because they're
checked into the repo, they have 0755 permissions.
- Other assorted directories created by tests have 0777 permissions.
|
|
c103d7b4
|
2011-09-29T15:49:28
|
|
odb: Pass compression settings to filebuf
|
|
8af4d074
|
2011-09-29T15:34:17
|
|
odb: Let users decide compression level for the loose ODB
|
|
71a4c1f1
|
2011-09-18T20:07:59
|
|
Merge pull request #384 from kiryl/warnings
Add more -W flags to CFLAGS
|
|
87d9869f
|
2011-09-19T03:34:49
|
|
Tabify everything
There were quite a few places were spaces were being used instead of
tabs. Try to catch them all. This should hopefully not break anything.
Except for `git blame`. Oh well.
|
|
bb742ede
|
2011-09-19T01:54:32
|
|
Cleanup legal data
1. The license header is technically not valid if it doesn't have a
copyright signature.
2. The COPYING file has been updated with the different licenses used in
the project.
3. The full GPLv2 header in each file annoys me.
|
|
1c3fac4d
|
2011-09-08T14:31:37
|
|
Add casts to get rid of some warnings when filling zlib structures
|
|
d568d585
|
2011-08-30T23:55:22
|
|
CMakefile: add -Wmissing-prototypes and fix warnings
Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>
|
|
afeecf4f
|
2011-07-09T02:10:46
|
|
odb: Direct writes are back
DIRECT WRITES ARE BACK AND FASTER THAN EVER. The streaming writer to the
ODB was an overkill for the smaller objects like Commit and Tags; most
of the streaming logic was taking too long.
This commit makes Commits, Tags and Trees to be built-up in memory, and
then written to disk in 2 pushes (header + data), instead of streaming
everything.
This is *always* faster, even for big files (since the git_filebuf class
still does streaming writes when the memory cache overflows). This is
also a gazillion lines of code smaller, because we don't have to
precompute the final size of the object before starting the stream (this
was kind of defeating the point of streaming, anyway).
Blobs are still written with full streaming instead of loading them in
memory, since this is still the fastest way.
A new `git_buf` class has been added. It's missing some features, but
it'll get there.
|
|
f79026b4
|
2011-07-04T11:43:34
|
|
fileops: Cleanup
Cleaned up the structure of the whole OS-abstraction layer.
fileops.c now contains a set of utility methods for file management used
by the library. These are abstractions on top of the original POSIX
calls.
There's a new file called `posix.c` that contains
emulations/reimplementations of all the POSIX calls the library uses.
These are prefixed with `p_`. There's a specific posix file for each
platform (win32 and unix).
All the path-related methods have been moved from `utils.c` to `path.c`
and have their own prefix.
|
|
f1d01851
|
2011-06-16T02:48:48
|
|
oid: Uniformize ncmp methods
Drop redundant methods. The ncmp method is now public
|
|
fa48608e
|
2011-06-16T02:36:21
|
|
oid: Rename methods
Yeah. Finally. Fuck the old names, this ain't POSIX
and they don't make any sense at all.
|
|
c09093cc
|
2011-06-06T10:55:36
|
|
Renamed git_oid_match to git_oid_ncmp.
As suggested by carlosmn, git_oid_ncmp would probably
be a better name than git_oid_match, for it does the same
as git_oid_cmp but only up to a certain amount of hex digits.
|
|
aea8a638
|
2011-05-29T18:00:35
|
|
Implemented read_unique_short_oid method for loose backend.
|
|
d0323a5f
|
2011-06-01T21:25:56
|
|
short-oid: Cleanup
|
|
ecd6fdf1
|
2011-05-27T18:49:09
|
|
Added a read_unique_short_oid method to backends, to make it possible to find objects from sha1 prefixes in the future. Default implementations throw GIT_ENOTIMPLEMENTED for strict prefixes (i.e. length < GIT_OID_HEXSZ).
|
|
60e1b49a
|
2011-05-23T21:12:18
|
|
odb_loose: Reword errors
|
|
dfb12cd5
|
2011-05-19T12:28:46
|
|
odb_loose.c: Move to new error handling mechanism
|
|
f93f8ec5
|
2011-05-19T12:27:43
|
|
odb_loose.c: Return GIT_ENOMEM when allocation fails
When trying to inflate a buffer, a GIT_ERROR was returned when malloc()
failed.
Fix this to return GIT_ENOMEM.
|
|
d8e1d038
|
2011-05-06T12:47:21
|
|
Fix two warnings from Clang
Both are about not reading the value stored in a variable.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
|
|
a3ced637
|
2011-04-22T17:36:28
|
|
Fix going into infinite loop in read_header_loose()
read_header_loose causes infinite loop on this file:
$ cat ../libcppgit/bin/sample-repo/test_mailbox/.git/objects/8f/e274605cbc740a2a957f44b2722a8a73915a09 | base64
eAErKUpNVTAzYzA0MDAzMVHISUxKzSlmWLgkuyN5+rxr6juMPR2EmN8s7Vl9D6oiN7UkkcHJdLbl
7Z3N/oxfE0W8wrSbuFRkAwDFfBn1
|
|
f6f72d7e
|
2011-03-23T18:44:53
|
|
Improve the ODB writing backend
Temporary files when doing streaming writes are now stored inside the
Objects folder, to prevent issues when moving files between
disks/partitions.
Add support for block writes to the ODB again (for those backends that
cannot implement streaming).
|
|
72a3fe42
|
2011-03-18T19:38:49
|
|
I broke your bindings
Hey. Apologies in advance -- I broke your bindings.
This is a major commit that includes a long-overdue redesign of the
whole object-database structure. This is expected to be the last major
external API redesign of the library until the first non-alpha release.
Please get your bindings up to date with these changes. They will be
included in the next minor release. Sorry again!
Major features include:
- Real caching and refcounting on parsed objects
- Real caching and refcounting on objects read from the ODB
- Streaming writes & reads from the ODB
- Single-method writes for all object types
- The external API is now partially thread-safe
The speed increases are significant in all aspects, specially when
reading an object several times from the ODB (revwalking) and when
writing big objects to the ODB.
Here's a full changelog for the external API:
blob.h
------
- Remove `git_blob_new`
- Remove `git_blob_set_rawcontent`
- Remove `git_blob_set_rawcontent_fromfile`
- Rename `git_blob_writefile` -> `git_blob_create_fromfile`
- Change `git_blob_create_fromfile`:
The `path` argument is now relative to the repository's working dir
- Add `git_blob_create_frombuffer`
commit.h
--------
- Remove `git_commit_new`
- Remove `git_commit_add_parent`
- Remove `git_commit_set_message`
- Remove `git_commit_set_committer`
- Remove `git_commit_set_author`
- Remove `git_commit_set_tree`
- Add `git_commit_create`
- Add `git_commit_create_v`
- Add `git_commit_create_o`
- Add `git_commit_create_ov`
tag.h
-----
- Remove `git_tag_new`
- Remove `git_tag_set_target`
- Remove `git_tag_set_name`
- Remove `git_tag_set_tagger`
- Remove `git_tag_set_message`
- Add `git_tag_create`
- Add `git_tag_create_o`
tree.h
------
- Change `git_tree_entry_2object`:
New signature is `(git_object **object_out, git_repository *repo, git_tree_entry *entry)`
- Remove `git_tree_new`
- Remove `git_tree_add_entry`
- Remove `git_tree_remove_entry_byindex`
- Remove `git_tree_remove_entry_byname`
- Remove `git_tree_clearentries`
- Remove `git_tree_entry_set_id`
- Remove `git_tree_entry_set_name`
- Remove `git_tree_entry_set_attributes`
object.h
------------
- Remove `git_object_new
- Remove `git_object_write`
- Change `git_object_close`:
This method is now *mandatory*. Not closing an object causes a
memory leak.
odb.h
-----
- Remove type `git_rawobj`
- Remove `git_rawobj_close`
- Rename `git_rawobj_hash` -> `git_odb_hash`
- Change `git_odb_hash`:
New signature is `(git_oid *id, const void *data, size_t len, git_otype type)`
- Add type `git_odb_object`
- Add `git_odb_object_close`
- Change `git_odb_read`:
New signature is `(git_odb_object **out, git_odb *db, const git_oid *id)`
- Change `git_odb_read_header`:
New signature is `(size_t *len_p, git_otype *type_p, git_odb *db, const git_oid *id)`
- Remove `git_odb_write`
- Add `git_odb_open_wstream`
- Add `git_odb_open_rstream`
odb_backend.h
-------------
- Change type `git_odb_backend`:
New internal signatures are as follows
int (* read)(void **, size_t *, git_otype *, struct git_odb_backend *, const git_oid *)
int (* read_header)(size_t *, git_otype *, struct git_odb_backend *, const git_oid *)
int (* writestream)(struct git_odb_stream **, struct git_odb_backend *, size_t, git_otype)
int (* readstream)( struct git_odb_stream **, struct git_odb_backend *, const git_oid *)
- Add type `git_odb_stream`
- Add enum `git_odb_streammode`
Signed-off-by: Vicent Marti <tanoku@gmail.com>
|
|
19a30a3f
|
2011-03-03T19:53:17
|
|
Add new move function, `gitfo_mv_force`
Forces a move by creating the folder for the destination file, if it
doesn't exist.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
|
|
d4b5a4e2
|
2011-02-09T19:49:02
|
|
Internal changes on the backend system
The priority value for different backends has been removed from the
public `git_odb_backend` struct. We handle that internally. The priority
value is specified on the `git_odb_add_alternate`.
This is convenient because it allows us to poll a backend twice with
different priorities without having to instantiate it twice.
We also differentiate between main backends and alternates; alternates have
lower priority and cannot be written to.
These changes come with some unit tests to make sure that the backend
sorting is consistent.
The libgit2 version has been bumped to 0.4.0.
This commit changes the external API:
CHANGED:
struct git_odb_backend
No longer has a `priority` attribute; priority for the backend
in managed internally by the library.
git_odb_add_backend(git_odb *odb, git_odb_backend *backend, int priority)
Now takes an additional priority parameter, the priority that
will be given to the backend.
ADDED:
git_odb_add_alternate(git_odb *odb, git_odb_backend *backend, int priority)
Add a backend as an alternate. Alternate backends have always
lower priority than main backends, and writing is disabled on
them.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Signed-off-by: Vicent Marti <tanoku@gmail.com>
|
|
f725931b
|
2011-02-05T12:42:41
|
|
Fix directory/path manipulation methods
The `dirname` and `dirbase` methods have been replaced with the Android
implementation, which is actually compilant to some kind of standard.
A new method `topdir` has been added, which returns the topmost
directory in a path.
These changes fix issue #49:
`gitfo_prettify_dir_path` converts "./.git/" to ".git/", so
the code at src/repository.c:190 goes out of bounds when
trying to find the topmost directory.
The new `git__topdir` method handles this gracefully, and the
fixed `git__dirname` now returns the proper value for the
repository's working dir.
E.g.
/repo/.git/ ==> working dir '/repo/'
.git/ ==> working dir '.'
Signed-off-by: Vicent Marti <tanoku@gmail.com>
|
|
44908fe7
|
2010-12-06T23:03:16
|
|
Change the library include file
Libgit2 is now officially include as
#include "<git2.h>"
or indidividual files may be included as
#include <git2/index.h>
Signed-off-by: Vicent Marti <tanoku@gmail.com>
|
|
d12299fe
|
2010-12-03T22:22:10
|
|
Change include structure for the project
The maze with include dependencies has been fixed.
There is now a global include:
#include <git.h>
The git_odb_backend API has been exposed.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
|
|
7d7cd885
|
2010-12-03T18:01:30
|
|
Decouple storage from ODB logic
Comes with two default backends: loose object and packfiles.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
|