kmx git

Commit	Date	Message
a0a1b19a	2015-10-14T19:31:54	odb: Prioritize alternate backends For most real use cases, repositories with alternates use them as main object storage. Checking the alternate for objects before the main repository should result in measurable speedups. Because of this, we're changing the sorting algorithm to prioritize alternates in cases where two backends have the same priority. This means that the pack backend for the alternate will be checked before the pack backend for the main repository but both of them will be checked before any loose backends.
43820f20	2015-10-14T19:24:07	odb: Be smarter when refreshing backends In the current implementation of ODB backends, each backend is tasked with refreshing itself after a failed lookup. This is standard Git behavior: we want to e.g. reload the packfiles on disk in case they have changed and that's the reason we can't find the object we're looking for. This behavior, however, becomes pathological in repositories where multiple alternates have been loaded. Given that each alternate counts as a separate backend, a miss in the main repository (which can potentially be very frequent in cases where object storage comes from the alternate) will result in refreshing all its packfiles before we move on to the alternate backend where the object will most likely be found. To fix this, the code in `odb.c` has been refactored as to perform the refresh of all the backends externally, once we've verified that the object is nowhere to be found. If the refresh is successful, we then perform the lookup sequentially through all the backends, skipping the ones that we know for sure weren't refreshed (because they have no refresh API). The on-disk pack backend has been adjusted accordingly: it no longer performs refreshes internally.
d3b29fb9	2015-10-01T00:50:37	refdb and odb backends must provide `free` function As refdb and odb backends can be allocated by client code, libgit2 can’t know whether an alternative memory allocator was used, and thus should not try to call `git__free` on those objects. Instead, odb and refdb backend implementations must always provide their own `free` functions to ensure memory gets freed correctly.
e5f9df7b	2015-06-29T21:45:04	odb: cast to long long for printf
9f3c18e2	2015-06-02T08:36:15	Fixed build warnings on Xcode 6.1
a6f2ceaf	2015-05-13T12:11:55	Merge pull request #3118 from libgit2/cmn/stream-size odb: make the writestream's size a git_off_t
b0d7f329	2015-05-13T10:23:19	odb: reverse the default backend priorities We currently first look in the loose object dir and then in the packs for objects. When performing operations on recent history this has a higher likelihood of hitting, but when we deal with operations which look further back into the past, we start spending a large amount of time getting ENOTENT from `access`. Reversing the priorities means that long-running operations can get to their objects faster, as we can look at the index data we have in memory (or rather mapped) to figure out whether we have an object, which is faster than going out to the filesystem. The packed backend already implements an optimistic read algorithm by first looking at the packs we know about and only going out to disk to referesh if the object is not found which means that in the case where we do have the object (which will be in the majority for anything that traverses the graph) we can avoid going to to disk entirely to determine whether an object exists. Operations which look at recent history may take a slight impact, but these would be operations which look a lot less at object and thus take less time regardless.
77b339f7	2015-05-12T13:06:33	odb: make the writestream's size a git_off_t Restricting files to size_t is a silly limitation. The loose backend writes to a file directly, so there is no issue in using 63 bits for the size. We still assume that the header is going to fit in 64 bytes, which does mean quite a bit smaller files due to the run-length encoding, but it's still a much larger size than you would want Git to handle.
7dd22538	2015-05-11T10:19:25	centralizing all IO buffer size values
f1453c59	2015-02-12T12:19:37	Make our overflow check look more like gcc/clang's Make our overflow checking look more like gcc and clang's, so that we can substitute it out with the compiler instrinsics on platforms that support it. This means dropping the ability to pass `NULL` as an out parameter. As a result, the macros also get updated to reflect this as well.
15d54fdd	2015-02-10T22:34:03	odb__hashlink: check st.st_size before casting
392702ee	2015-02-09T23:41:13	allocations: test for overflow of requested size Introduce some helper macros to test integer overflow from arithmetic and set error message appropriately.
c251f3bb	2014-12-08T16:05:47	win32: remember to cleanup our hash_ctx
e0156651	2014-11-21T13:50:46	odb: `git_odb_object` contents are never NULL This is a contract that we made in the library and that we need to uphold. The contents of a blob can never be NULL because several parts of the library (including the filter and attributes code) expect `git_blob_rawcontent` to always return a valid pointer.
e1ac0101	2014-11-08T14:40:53	odb: hardcode the empty blob and tree git hardocodes these as objects which exist regardless of whether they are in the odb and uses them in the shell interface as a way of expressing the lack of a blob or tree for one side of e.g. a diff. In the library we use each language's natural way of declaring a lack of value which makes a workaround like this unnecessary. Since git uses it, it does however mean each shell application would need to perform this check themselves. This makes it common work across a range of applications and an issue with compatibility with git, which fits right into what the library aims to provide. Thus we introduce the hard-coded empty blob and tree in the odb frontend. These hard-coded objects are checked for before going to the backends, but after the cache check, which means the second time they're used, they will be treated as normal cached objects instead of creating new ones.
530594c0	2014-05-23T05:53:41	odb: clear backend errors on successful read We go through the different backends in order, so it's not an error if at least one of the backends has the data we want.
bc91347b	2014-04-30T11:16:31	Fix remaining init_options inconsistencies There were a couple of "init_opts()" functions a few more cases of structure initialization that I somehow missed.
48e60ae7	2014-04-21T11:23:29	Don't redefine the same callback types, their signatures may change
3ab57816	2014-03-31T23:23:32	Merge pull request #2178 from libgit2/rb/fix-short-id Fix git_odb_short_id and git_odb_exists_prefix bugs
31a14982	2014-03-21T17:36:34	Fix wrong assertion Fixes issue #2196
89499078	2014-03-10T10:53:39	Fix a number of git_odb_exists_prefix bugs The git_odb_exists_prefix API was not dealing correctly when a later backend returned GIT_ENOTFOUND even if an earlier backend had found the object. Additionally, the unit tests were not properly exercising the API and had a couple mistakes in checking the results. Lastly, since the backends are not expected to behavior correctly unless all bytes of the short id are zero except for the prefix, this makes the ODB prefix APIs explicitly clear out the extra bytes so the user doesn't have to be as careful.
b9f81997	2014-03-05T21:49:23	Added function-based initializers for every options struct. The basic structure of each function is courtesy of arrbee.
a064dc2d	2014-03-06T00:47:05	Merge pull request #2159 from libgit2/rb/odb-exists-prefix Add ODB API to check for existence by prefix and object id shortener
26875825	2014-03-05T13:06:22	Check short OID len in odb, not in backends
7bd2f401	2014-03-05T11:35:47	ODB writing fails gracefully when unsupported If no ODB backends support writing, we should fail gracefully.
f5753999	2014-03-04T15:34:23	Add exists_prefix to ODB backend and ODB API
ae3b6d61	2014-01-12T23:31:13	odb: handle NULL pointers passed to git_odb_stream_free Signed-off-by: Brodie Rao <brodie@sf.io>
dd64c71c	2013-11-04T14:50:25	Allow backend consumers to specify file mode
5c50f22a	2013-10-28T09:25:44	Merge pull request #1891 from libgit2/cmn/fix-thin-packs Add support for thin packs
98fec8a9	2013-10-22T16:05:47	Implement `git_odb_object_dup`
0b33fca0	2013-10-02T13:39:35	indexer: fix thin packs When given an ODB from which to read objects, the indexer will attempt to inject the missing bases at the end of the pack and update the header and trailer to reflect the new contents.
92d19d16	2013-09-21T09:34:03	Merge pull request #1840 from linquize/warning Fix warning
66566516	2013-09-08T17:15:42	Fix warning
a9f51e43	2013-09-11T22:00:36	Merge git_buf and git_buffer This makes the git_buf struct that was used internally into an externally available structure and eliminates the git_buffer. As part of that, some of the special cases that arose with the externally used git_buffer were blended into the git_buf, such as being careful about git_buf objects that may have a NULL ptr and allowing for bufs with a valid ptr and size but zero asize as a way of referring to externally owned data.
2a7d224f	2013-09-10T16:33:32	Extend public filter api with filter lists This moves the git_filter_list into the public API so that users can create, apply, and dispose of filter lists. This allows more granular application of filters to user data outside of libgit2 internals. This also converts all the internal usage of filters to the public APIs along with a few small tweaks to make it easier to use the public git_buffer stuff alongside the internal git_buf.
85d54812	2013-08-28T16:44:04	Create public filter object and use it This creates include/sys/filter.h with a basic definition of a git_filter and then converts the internal code to use it. There are related internal objects (git_filter_list) that we will want to publish at some point, but this is a first step.
8cf80525	2013-09-11T20:13:59	errors: Fix format of some error messages
031f3f80	2013-09-07T22:39:05	odb: Error when streaming in too [few\|many] bytes
4047950f	2013-08-29T14:19:34	odb: Prevent stream_finalize_write() from overwriting Now that #1785 is merged, git_odb_stream_finalize_write() calculates the object id before invoking the odb backend. This commit gives a chance to the backend to check if it already knows this object.
b1a6c316	2013-08-30T17:36:00	odb: Move the auto refresh logic to the pack backend Previously, `git_object_read()`, `git_object_read_prefix()` and `git_object_exists()` were implementing an auto refresh logic. When the expected object couldn't be found in any backend, a call to `git_odb_refresh()` was triggered and the lookup was once again performed against all backends. This commit removes this auto-refresh logic from the odb layer and pushes it down into the pack-backend (as it's the only one currently exposing a `refresh()` endpoint).
a12e069a	2013-08-30T16:31:52	odb: Honor the non refreshing capability of a backend
090a07d2	2013-08-17T02:12:04	odb: avoid hashing twice in and edge case If none of the backends support direct writes and we must stream the whole file, we already know what the object's id should be; so use the stream's functions directly, bypassing the frontend's hashing and overwriting of our existing id.
fe0c6d4e	2013-08-17T01:41:08	odb: make it clearer that the id is calculated in the frontend The frontend is in charge of calculating the id of the objects. Thus the backends should treat it as a read-only value. The positioning in the function signature made it seem as though it was an output parameter. Make the id const and move it from the front to behind the subject (backend or stream).
8380b39a	2013-08-15T14:29:39	odb: perform the stream hashing in the frontend Hash the data as it's coming into the stream and tell the backend what its name is when finalizing the write. This makes it consistent with the way a plain git_odb_write() performs the write.
376e6c9f	2013-08-15T13:48:35	odb: wrap the stream reading and writing functions This is in preparation for moving the hashing to the frontend, which requires us to handle the incoming data before passing it to the backend's stream.
e54cfb9b	2013-08-12T11:50:27	odb: free object data when id is ambiguous By the time we recognise this as an ambiguous id, the object's data has been loaded into memory. Free it when returning EABMIGUOUS.
c6451624	2013-07-15T16:00:07	Fix some more memory leaks in error path
6de9b2ee	2013-06-12T21:10:33	util: It's called `memzero`
3e9e6cda	2013-06-07T09:54:33	Add safe memset and use it This adds a `git__memset` routine that will not be optimized away and updates the places where I memset() right before a free() call to use it.
f658dc43	2013-05-31T14:09:58	Zero memory for major objects before freeing By zeroing out the memory when we free larger objects (i.e. those that serve as collections of other data, such as repos, odb, refdb), I'm hoping that it will be easier for libgit2 bindings to find errors in their object management code.
03c28d92	2013-05-06T06:45:53	Merge pull request #1526 from arrbee/cleanup-error-return-without-msg Make sure error messages are set for most error returns
dfec726b	2013-05-03T23:30:54	odb: Do not error out if an alternate ODB is missing
f063f578	2013-05-01T14:48:35	Catch some odd odb backend corner case errors There are some cases, particularly where no loaded ODB backends support a particular operation, where we would return an error code without having set an error. This catches those cases and reports that no ODB backends support the operation in question.
cd2ed9f0	2013-04-30T04:02:52	Merge pull request #1518 from arrbee/export-oid-comparison Remove most inlines from the public API
b7f167da	2013-04-29T13:52:12	Make git_oid_cmp public and add git_oid__cmp
c8a4e8a5	2013-04-29T11:14:56	don't use uninitialized struct stat in win32
78606263	2013-04-15T00:05:44	Add callback to git_objects_table This adds create and free callback to the git_objects_table so that more of the creation and destruction of objects can be table driven instead of using switch statements. This also makes the semantics of certain object creation functions consistent so that we can make better use of function pointers. This also fixes a theoretical error case where an object allocation fails and we end up storing NULL into the cache.
8842c75f	2013-04-03T22:30:07	What has science done.
5df18424	2013-04-01T19:38:23	lol this worked first try wtf
0edad3cc	2013-04-22T16:41:56	Merge branch 'development' into vmg/dupe-odb-backends Conflicts: src/odb.c
4ef2c79c	2013-04-22T16:37:40	odb: Disable inode checks for Win32
83cc70d9	2013-04-19T12:48:33	Move odb_backend implementors stuff into git2/sys This moves some of the odb_backend stuff that is related to the internals of an odb_backend implementation into include/git2/sys. Some of the stuff related to streaming I left in include/git2 because it seemed like it would be reasonably needed by a normal user who wanted to stream objects into and out of the ODB. Also, I added APIs for traversing the list of backends so that some of the tests would not need to access ODB internals.
a29c6b5f	2013-04-19T23:51:18	odb: Do not allow duplicate on-disk backends
f5e28202	2013-03-25T13:38:43	opts: allow configuration of odb cache size Currently, the odb cache has a fixed size of 128 slots as defined by GIT_DEFAULT_CACHE_SIZE. Allow users to set the size of the cache via git_libgit2_opts(). Fixes #1035.
10c06114	2013-03-17T04:46:46	Several warnings detected by static code analyzer fixed Implicit type conversion argument of function to size_t type Suspicious sequence of types castings: size_t -> int -> size_t Consider reviewing the expression of the 'A = B == C' kind. The expression is calculated as following: 'A = (B == C)' Unsigned type is never < 0
8fe6bc5c	2013-01-10T15:43:08	odb: Refresh on `exists` query too
4a863c06	2013-01-03T20:36:26	Sane refresh logic All the ODB backends have a specific refresh interface. When reading an object, first we attempt every single backend: if the read fails, then we refresh all the backends and retry the read one more time to see if the object has appeared.
891a4681	2013-01-04T17:42:41	dat errorcode
359fc2d2	2013-01-08T17:07:25	update copyrights
4d185dd9	2012-12-19T14:30:06	odb: check if object exists before writing Update the procondition of git_odb_backend::write. It may now be assumed that the object has already been hashed.
0249a503	2012-12-07T09:40:21	Merge pull request #1091 from carlosmn/stream-object Indexer speedup with large objects
c7231c45	2012-11-30T16:31:42	Deploy GITERR_CHECK_VERSION
55f6f21b	2012-11-29T19:59:18	Deploy versioned git_odb_backend structure
f56f8585	2012-11-19T22:23:16	indexer: use the packfile streaming API The new API allows us to read the object bit by bit from the packfile, instead of needing it all at once in the packfile. This also allows us to hash the object as it comes in from the network instead of having to try to read it all and failing repeatedly for larger objects. This is only the first step, but it already shows huge improvements when dealing with objects over a few megabytes in size. It reduces the memory needs in some cases, but delta objects still need to be completely in memory and the old inefficent method is still used for that.
9507a434	2012-11-28T10:47:10	odb: Add `git_odb_add_disk_alternate` Loads a disk alternate by path to the ODB. Mimics the `GIT_ALTERNATE_OBJECT_DIRECTORIES` shell var.
2e76b5fc	2012-11-27T09:49:16	API updates for odb.h
85e7efa1	2012-11-14T13:35:43	odb: recursively load alternates The maximum depth is 5, like in git
d6fb0924	2012-11-05T12:37:15	Win32 CryptoAPI and CNG support for SHA1
603bee07	2012-11-12T19:22:49	Remove git_hash_ctx_new - callers now _ctx_init()
09cc0b92	2012-11-05T11:33:10	create callback to handle packs from fetch, move the indexer to odb_pack
edca6c8f	2012-07-01T19:44:22	git_odb_object_free: don't segfault w/ arg == NULL
addc9be4	2012-09-26T17:21:32	Fix error hashing empty file.
e8776d30	2012-09-16T00:10:07	odb: don't overflow the link path buffer Allocate a buffer large enough to store the path plus the terminator instead of letting readlink write beyond the end.
9be2261e	2012-09-13T09:24:12	Merge pull request #927 from arrbee/hashfile-with-filters Add git_repository_hashfile to hash with filters
13faa77c	2012-09-13T17:57:45	Fix -Wuninitialized warning
a13fb55a	2012-09-11T17:26:21	Add tests and improve param checks Fixed some minor `git_repository_hashfile` issues: - Fixed incorrect doc (saying that repo could be NULL) - Added checking of object type value to acceptable ones - Added more tests for various parameter permutations
c859184b	2012-09-11T23:05:24	Properly handle p_reads
c6ac28fd	2012-09-10T12:24:05	Reorg internal odb read header and object lookup Often `git_odb_read_header` will "fail" and have to read the entire object into memory instead of just the header. When this happens, the object is loaded and then disposed of immediately, which makes it difficult to efficiently use the header information to decide if the object should be loaded (since attempting to do so will often result in loading the object twice). This commit takes the existing code and reorganizes it to have two new functions: - `git_odb__read_header_or_object` which acts just like the old read header function except that it returns the object, too, if it was forced to load the whole thing. It then becomes the callers responsibility to free the `git_odb_object`. - `git_object__from_odb_object` which was extracted from the old `git_object_lookup` and creates a subclass of `git_object` from an existing `git_odb_object` (separating the ODB lookup from the `git_object` creation). This allows you to use the first header reading function efficiently without instantiating the `git_odb_object` twice. There is no net change to the behavior of any of the existing functions, but this allows internal code to tap into the ODB lookup and object creation to be more efficient.
60b9d3fc	2012-09-05T15:00:40	Implement filters for status/diff blobs This adds support to diff and status for running filters (a la crlf) on blobs in the workdir before computing SHAs and before generating text diffs. This ended up being a bit more code change than I had thought since I had to reorganize some of the diff logic to minimize peak memory use when filtering blobs in a diff. This also adds a cap on the maximum size of data that will be loaded to diff. I set it at 512Mb which should match core git. Right now it is a #define in src/diff.h but it could be moved into the public API if desired.
0e9f2fce	2012-09-06T11:35:09	odb: mark unused variable
c49d328c	2012-08-27T09:59:13	Expose a malloc function to 3rd party ODB backends
c07d9c95	2012-08-09T15:33:04	oid: Explicitly include `oid.h` for the inlined CMP
51e1d808	2012-08-06T12:41:08	Merge remote-tracking branch 'arrbee/tree-walk-fixes' into development Conflicts: src/notes.c src/transports/git.c src/transports/http.c src/transports/local.c tests-clar/odb/foreach.c
5dca2010	2012-08-03T17:08:01	Update iterators for consistency across library This updates all the `foreach()` type functions across the library that take callbacks from the user to have a consistent behavior. The rules are: * A callback terminates the loop by returning any non-zero value * Once the callback returns non-zero, it will not be called again (i.e. the loop stops all iteration regardless of state) * If the callback returns non-zero, the parent fn returns GIT_EUSER * Although the parent returns GIT_EUSER, no error will be set in the library and `giterr_last()` will return NULL if called. This commit makes those changes across the library and adds tests for most of the iteration APIs to make sure that they follow the above rules.
b8457baa	2012-07-24T07:57:58	portability: Improve x86/amd64 compatibility
521aedad	2012-06-05T14:48:51	odb: add git_odb_foreach() Go through each backend and list every objects that exists in them. This allows fsck-like uses.
c06e0003	2012-06-20T01:41:30	odb: don't leak when detecting id ambiguity If we find several objects with the same prefix, we need to free the memory where we stored the earlier object. Keep track of the raw.data pointer across read_prefix calls and free it if we find another object.
904b67e6	2012-05-18T01:48:50	errors: Rename error codes
e172cf08	2012-05-18T01:21:06	errors: Rename the generic return codes
24634c6f	2012-05-12T15:01:39	Handle duplicate objects from different backends in git_odb_read_prefix().

a0a1b19a

2015-10-14T19:31:54

odb: Prioritize alternate backends For most real use cases, repositories with alternates use them as main object storage. Checking the alternate for objects before the main repository should result in measurable speedups. Because of this, we're changing the sorting algorithm to prioritize alternates *in cases where two backends have the same priority*. This means that the pack backend for the alternate will be checked before the pack backend for the main repository *but* both of them will be checked before any loose backends.

43820f20

2015-10-14T19:24:07

odb: Be smarter when refreshing backends In the current implementation of ODB backends, each backend is tasked with refreshing itself after a failed lookup. This is standard Git behavior: we want to e.g. reload the packfiles on disk in case they have changed and that's the reason we can't find the object we're looking for. This behavior, however, becomes pathological in repositories where multiple alternates have been loaded. Given that each alternate counts as a separate backend, a miss in the main repository (which can potentially be very frequent in cases where object storage comes from the alternate) will result in refreshing all its packfiles before we move on to the alternate backend where the object will most likely be found. To fix this, the code in `odb.c` has been refactored as to perform the refresh of all the backends externally, once we've verified that the object is nowhere to be found. If the refresh is successful, we then perform the lookup sequentially through all the backends, skipping the ones that we know for sure weren't refreshed (because they have no refresh API). The on-disk pack backend has been adjusted accordingly: it no longer performs refreshes internally.

d3b29fb9

2015-10-01T00:50:37

refdb and odb backends must provide `free` function As refdb and odb backends can be allocated by client code, libgit2 can’t know whether an alternative memory allocator was used, and thus should not try to call `git__free` on those objects. Instead, odb and refdb backend implementations must always provide their own `free` functions to ensure memory gets freed correctly.

e5f9df7b

2015-06-29T21:45:04

odb: cast to long long for printf

9f3c18e2

2015-06-02T08:36:15

Fixed build warnings on Xcode 6.1

a6f2ceaf

2015-05-13T12:11:55

Merge pull request #3118 from libgit2/cmn/stream-size odb: make the writestream's size a git_off_t

b0d7f329

2015-05-13T10:23:19

odb: reverse the default backend priorities We currently first look in the loose object dir and then in the packs for objects. When performing operations on recent history this has a higher likelihood of hitting, but when we deal with operations which look further back into the past, we start spending a large amount of time getting ENOTENT from `access`. Reversing the priorities means that long-running operations can get to their objects faster, as we can look at the index data we have in memory (or rather mapped) to figure out whether we have an object, which is faster than going out to the filesystem. The packed backend already implements an optimistic read algorithm by first looking at the packs we know about and only going out to disk to referesh if the object is not found which means that in the case where we do have the object (which will be in the majority for anything that traverses the graph) we can avoid going to to disk entirely to determine whether an object exists. Operations which look at recent history may take a slight impact, but these would be operations which look a lot less at object and thus take less time regardless.

77b339f7

2015-05-12T13:06:33

odb: make the writestream's size a git_off_t Restricting files to size_t is a silly limitation. The loose backend writes to a file directly, so there is no issue in using 63 bits for the size. We still assume that the header is going to fit in 64 bytes, which does mean quite a bit smaller files due to the run-length encoding, but it's still a much larger size than you would want Git to handle.

7dd22538

2015-05-11T10:19:25

centralizing all IO buffer size values

f1453c59

2015-02-12T12:19:37

Make our overflow check look more like gcc/clang's Make our overflow checking look more like gcc and clang's, so that we can substitute it out with the compiler instrinsics on platforms that support it. This means dropping the ability to pass `NULL` as an out parameter. As a result, the macros also get updated to reflect this as well.

15d54fdd

2015-02-10T22:34:03

odb__hashlink: check st.st_size before casting

392702ee

2015-02-09T23:41:13

allocations: test for overflow of requested size Introduce some helper macros to test integer overflow from arithmetic and set error message appropriately.

c251f3bb

2014-12-08T16:05:47

win32: remember to cleanup our hash_ctx

e0156651

2014-11-21T13:50:46

odb: `git_odb_object` contents are never NULL This is a contract that we made in the library and that we need to uphold. The contents of a blob can never be NULL because several parts of the library (including the filter and attributes code) expect `git_blob_rawcontent` to always return a valid pointer.

e1ac0101

2014-11-08T14:40:53

odb: hardcode the empty blob and tree git hardocodes these as objects which exist regardless of whether they are in the odb and uses them in the shell interface as a way of expressing the lack of a blob or tree for one side of e.g. a diff. In the library we use each language's natural way of declaring a lack of value which makes a workaround like this unnecessary. Since git uses it, it does however mean each shell application would need to perform this check themselves. This makes it common work across a range of applications and an issue with compatibility with git, which fits right into what the library aims to provide. Thus we introduce the hard-coded empty blob and tree in the odb frontend. These hard-coded objects are checked for before going to the backends, but after the cache check, which means the second time they're used, they will be treated as normal cached objects instead of creating new ones.

530594c0

2014-05-23T05:53:41

odb: clear backend errors on successful read We go through the different backends in order, so it's not an error if at least one of the backends has the data we want.

bc91347b

2014-04-30T11:16:31

Fix remaining init_options inconsistencies There were a couple of "init_opts()" functions a few more cases of structure initialization that I somehow missed.

48e60ae7

2014-04-21T11:23:29

Don't redefine the same callback types, their signatures may change

3ab57816

2014-03-31T23:23:32

Merge pull request #2178 from libgit2/rb/fix-short-id Fix git_odb_short_id and git_odb_exists_prefix bugs

31a14982

2014-03-21T17:36:34

Fix wrong assertion Fixes issue #2196

89499078

2014-03-10T10:53:39

Fix a number of git_odb_exists_prefix bugs The git_odb_exists_prefix API was not dealing correctly when a later backend returned GIT_ENOTFOUND even if an earlier backend had found the object. Additionally, the unit tests were not properly exercising the API and had a couple mistakes in checking the results. Lastly, since the backends are not expected to behavior correctly unless all bytes of the short id are zero except for the prefix, this makes the ODB prefix APIs explicitly clear out the extra bytes so the user doesn't have to be as careful.

b9f81997

2014-03-05T21:49:23

Added function-based initializers for every options struct. The basic structure of each function is courtesy of arrbee.

a064dc2d

2014-03-06T00:47:05

Merge pull request #2159 from libgit2/rb/odb-exists-prefix Add ODB API to check for existence by prefix and object id shortener

26875825

2014-03-05T13:06:22

Check short OID len in odb, not in backends

7bd2f401

2014-03-05T11:35:47

ODB writing fails gracefully when unsupported If no ODB backends support writing, we should fail gracefully.

f5753999

2014-03-04T15:34:23

Add exists_prefix to ODB backend and ODB API

ae3b6d61

2014-01-12T23:31:13

odb: handle NULL pointers passed to git_odb_stream_free Signed-off-by: Brodie Rao <brodie@sf.io>

dd64c71c

2013-11-04T14:50:25

Allow backend consumers to specify file mode

5c50f22a

2013-10-28T09:25:44

Merge pull request #1891 from libgit2/cmn/fix-thin-packs Add support for thin packs

98fec8a9

2013-10-22T16:05:47

Implement `git_odb_object_dup`

0b33fca0

2013-10-02T13:39:35

indexer: fix thin packs When given an ODB from which to read objects, the indexer will attempt to inject the missing bases at the end of the pack and update the header and trailer to reflect the new contents.

92d19d16

2013-09-21T09:34:03

Merge pull request #1840 from linquize/warning Fix warning

66566516

2013-09-08T17:15:42

Fix warning

a9f51e43

2013-09-11T22:00:36

Merge git_buf and git_buffer This makes the git_buf struct that was used internally into an externally available structure and eliminates the git_buffer. As part of that, some of the special cases that arose with the externally used git_buffer were blended into the git_buf, such as being careful about git_buf objects that may have a NULL ptr and allowing for bufs with a valid ptr and size but zero asize as a way of referring to externally owned data.

2a7d224f

2013-09-10T16:33:32

Extend public filter api with filter lists This moves the git_filter_list into the public API so that users can create, apply, and dispose of filter lists. This allows more granular application of filters to user data outside of libgit2 internals. This also converts all the internal usage of filters to the public APIs along with a few small tweaks to make it easier to use the public git_buffer stuff alongside the internal git_buf.

85d54812

2013-08-28T16:44:04

Create public filter object and use it This creates include/sys/filter.h with a basic definition of a git_filter and then converts the internal code to use it. There are related internal objects (git_filter_list) that we will want to publish at some point, but this is a first step.

8cf80525

2013-09-11T20:13:59

errors: Fix format of some error messages

031f3f80

2013-09-07T22:39:05

odb: Error when streaming in too [few|many] bytes

4047950f

2013-08-29T14:19:34

odb: Prevent stream_finalize_write() from overwriting Now that #1785 is merged, git_odb_stream_finalize_write() calculates the object id before invoking the odb backend. This commit gives a chance to the backend to check if it already knows this object.

b1a6c316

2013-08-30T17:36:00

odb: Move the auto refresh logic to the pack backend Previously, `git_object_read()`, `git_object_read_prefix()` and `git_object_exists()` were implementing an auto refresh logic. When the expected object couldn't be found in any backend, a call to `git_odb_refresh()` was triggered and the lookup was once again performed against all backends. This commit removes this auto-refresh logic from the odb layer and pushes it down into the pack-backend (as it's the only one currently exposing a `refresh()` endpoint).

a12e069a

2013-08-30T16:31:52

odb: Honor the non refreshing capability of a backend

090a07d2

2013-08-17T02:12:04

odb: avoid hashing twice in and edge case If none of the backends support direct writes and we must stream the whole file, we already know what the object's id should be; so use the stream's functions directly, bypassing the frontend's hashing and overwriting of our existing id.

fe0c6d4e

2013-08-17T01:41:08

odb: make it clearer that the id is calculated in the frontend The frontend is in charge of calculating the id of the objects. Thus the backends should treat it as a read-only value. The positioning in the function signature made it seem as though it was an output parameter. Make the id const and move it from the front to behind the subject (backend or stream).

8380b39a

2013-08-15T14:29:39

odb: perform the stream hashing in the frontend Hash the data as it's coming into the stream and tell the backend what its name is when finalizing the write. This makes it consistent with the way a plain git_odb_write() performs the write.

376e6c9f

2013-08-15T13:48:35

odb: wrap the stream reading and writing functions This is in preparation for moving the hashing to the frontend, which requires us to handle the incoming data before passing it to the backend's stream.

e54cfb9b

2013-08-12T11:50:27

odb: free object data when id is ambiguous By the time we recognise this as an ambiguous id, the object's data has been loaded into memory. Free it when returning EABMIGUOUS.

c6451624

2013-07-15T16:00:07

Fix some more memory leaks in error path

6de9b2ee

2013-06-12T21:10:33

util: It's called `memzero`

3e9e6cda

2013-06-07T09:54:33

Add safe memset and use it This adds a `git__memset` routine that will not be optimized away and updates the places where I memset() right before a free() call to use it.

f658dc43

2013-05-31T14:09:58

Zero memory for major objects before freeing By zeroing out the memory when we free larger objects (i.e. those that serve as collections of other data, such as repos, odb, refdb), I'm hoping that it will be easier for libgit2 bindings to find errors in their object management code.

03c28d92

2013-05-06T06:45:53

Merge pull request #1526 from arrbee/cleanup-error-return-without-msg Make sure error messages are set for most error returns

dfec726b

2013-05-03T23:30:54

odb: Do not error out if an alternate ODB is missing

f063f578

2013-05-01T14:48:35

Catch some odd odb backend corner case errors There are some cases, particularly where no loaded ODB backends support a particular operation, where we would return an error code without having set an error. This catches those cases and reports that no ODB backends support the operation in question.

cd2ed9f0

2013-04-30T04:02:52

Merge pull request #1518 from arrbee/export-oid-comparison Remove most inlines from the public API

b7f167da

2013-04-29T13:52:12

Make git_oid_cmp public and add git_oid__cmp

c8a4e8a5

2013-04-29T11:14:56

don't use uninitialized struct stat in win32

78606263

2013-04-15T00:05:44

Add callback to git_objects_table This adds create and free callback to the git_objects_table so that more of the creation and destruction of objects can be table driven instead of using switch statements. This also makes the semantics of certain object creation functions consistent so that we can make better use of function pointers. This also fixes a theoretical error case where an object allocation fails and we end up storing NULL into the cache.

8842c75f

2013-04-03T22:30:07

What has science done.

5df18424

2013-04-01T19:38:23

lol this worked first try wtf

0edad3cc

2013-04-22T16:41:56

Merge branch 'development' into vmg/dupe-odb-backends Conflicts: src/odb.c

4ef2c79c

2013-04-22T16:37:40

odb: Disable inode checks for Win32

83cc70d9

2013-04-19T12:48:33

Move odb_backend implementors stuff into git2/sys This moves some of the odb_backend stuff that is related to the internals of an odb_backend implementation into include/git2/sys. Some of the stuff related to streaming I left in include/git2 because it seemed like it would be reasonably needed by a normal user who wanted to stream objects into and out of the ODB. Also, I added APIs for traversing the list of backends so that some of the tests would not need to access ODB internals.

a29c6b5f

2013-04-19T23:51:18

odb: Do not allow duplicate on-disk backends

f5e28202

2013-03-25T13:38:43

opts: allow configuration of odb cache size Currently, the odb cache has a fixed size of 128 slots as defined by GIT_DEFAULT_CACHE_SIZE. Allow users to set the size of the cache via git_libgit2_opts(). Fixes #1035.

10c06114

2013-03-17T04:46:46

Several warnings detected by static code analyzer fixed Implicit type conversion argument of function to size_t type Suspicious sequence of types castings: size_t -> int -> size_t Consider reviewing the expression of the 'A = B == C' kind. The expression is calculated as following: 'A = (B == C)' Unsigned type is never < 0

8fe6bc5c

2013-01-10T15:43:08

odb: Refresh on `exists` query too

4a863c06

2013-01-03T20:36:26

Sane refresh logic All the ODB backends have a specific refresh interface. When reading an object, first we attempt every single backend: if the read fails, then we refresh all the backends and retry the read one more time to see if the object has appeared.

891a4681

2013-01-04T17:42:41

dat errorcode

359fc2d2

2013-01-08T17:07:25

update copyrights

4d185dd9

2012-12-19T14:30:06

odb: check if object exists before writing Update the procondition of git_odb_backend::write. It may now be assumed that the object has already been hashed.

0249a503

2012-12-07T09:40:21

Merge pull request #1091 from carlosmn/stream-object Indexer speedup with large objects

c7231c45

2012-11-30T16:31:42

Deploy GITERR_CHECK_VERSION

55f6f21b

2012-11-29T19:59:18

Deploy versioned git_odb_backend structure

f56f8585

2012-11-19T22:23:16

indexer: use the packfile streaming API The new API allows us to read the object bit by bit from the packfile, instead of needing it all at once in the packfile. This also allows us to hash the object as it comes in from the network instead of having to try to read it all and failing repeatedly for larger objects. This is only the first step, but it already shows huge improvements when dealing with objects over a few megabytes in size. It reduces the memory needs in some cases, but delta objects still need to be completely in memory and the old inefficent method is still used for that.

9507a434

2012-11-28T10:47:10

odb: Add `git_odb_add_disk_alternate` Loads a disk alternate by path to the ODB. Mimics the `GIT_ALTERNATE_OBJECT_DIRECTORIES` shell var.

2e76b5fc

2012-11-27T09:49:16

API updates for odb.h

85e7efa1

2012-11-14T13:35:43

odb: recursively load alternates The maximum depth is 5, like in git

d6fb0924

2012-11-05T12:37:15

Win32 CryptoAPI and CNG support for SHA1

603bee07

2012-11-12T19:22:49

Remove git_hash_ctx_new - callers now _ctx_init()

09cc0b92

2012-11-05T11:33:10

create callback to handle packs from fetch, move the indexer to odb_pack

edca6c8f

2012-07-01T19:44:22

git_odb_object_free: don't segfault w/ arg == NULL

addc9be4

2012-09-26T17:21:32

Fix error hashing empty file.

e8776d30

2012-09-16T00:10:07

odb: don't overflow the link path buffer Allocate a buffer large enough to store the path plus the terminator instead of letting readlink write beyond the end.

9be2261e

2012-09-13T09:24:12

Merge pull request #927 from arrbee/hashfile-with-filters Add git_repository_hashfile to hash with filters

13faa77c

2012-09-13T17:57:45

Fix -Wuninitialized warning

a13fb55a

2012-09-11T17:26:21

Add tests and improve param checks Fixed some minor `git_repository_hashfile` issues: - Fixed incorrect doc (saying that repo could be NULL) - Added checking of object type value to acceptable ones - Added more tests for various parameter permutations

c859184b

2012-09-11T23:05:24

Properly handle p_reads

c6ac28fd

2012-09-10T12:24:05

Reorg internal odb read header and object lookup Often `git_odb_read_header` will "fail" and have to read the entire object into memory instead of just the header. When this happens, the object is loaded and then disposed of immediately, which makes it difficult to efficiently use the header information to decide if the object should be loaded (since attempting to do so will often result in loading the object twice). This commit takes the existing code and reorganizes it to have two new functions: - `git_odb__read_header_or_object` which acts just like the old read header function except that it returns the object, too, if it was forced to load the whole thing. It then becomes the callers responsibility to free the `git_odb_object`. - `git_object__from_odb_object` which was extracted from the old `git_object_lookup` and creates a subclass of `git_object` from an existing `git_odb_object` (separating the ODB lookup from the `git_object` creation). This allows you to use the first header reading function efficiently without instantiating the `git_odb_object` twice. There is no net change to the behavior of any of the existing functions, but this allows internal code to tap into the ODB lookup and object creation to be more efficient.

60b9d3fc

2012-09-05T15:00:40

Implement filters for status/diff blobs This adds support to diff and status for running filters (a la crlf) on blobs in the workdir before computing SHAs and before generating text diffs. This ended up being a bit more code change than I had thought since I had to reorganize some of the diff logic to minimize peak memory use when filtering blobs in a diff. This also adds a cap on the maximum size of data that will be loaded to diff. I set it at 512Mb which should match core git. Right now it is a #define in src/diff.h but it could be moved into the public API if desired.

0e9f2fce

2012-09-06T11:35:09

odb: mark unused variable

c49d328c

2012-08-27T09:59:13

Expose a malloc function to 3rd party ODB backends

c07d9c95

2012-08-09T15:33:04

oid: Explicitly include `oid.h` for the inlined CMP

51e1d808

2012-08-06T12:41:08

Merge remote-tracking branch 'arrbee/tree-walk-fixes' into development Conflicts: src/notes.c src/transports/git.c src/transports/http.c src/transports/local.c tests-clar/odb/foreach.c

5dca2010

2012-08-03T17:08:01

Update iterators for consistency across library This updates all the `foreach()` type functions across the library that take callbacks from the user to have a consistent behavior. The rules are: * A callback terminates the loop by returning any non-zero value * Once the callback returns non-zero, it will not be called again (i.e. the loop stops all iteration regardless of state) * If the callback returns non-zero, the parent fn returns GIT_EUSER * Although the parent returns GIT_EUSER, no error will be set in the library and `giterr_last()` will return NULL if called. This commit makes those changes across the library and adds tests for most of the iteration APIs to make sure that they follow the above rules.

b8457baa

2012-07-24T07:57:58

portability: Improve x86/amd64 compatibility

521aedad

2012-06-05T14:48:51

odb: add git_odb_foreach() Go through each backend and list every objects that exists in them. This allows fsck-like uses.

c06e0003

2012-06-20T01:41:30

odb: don't leak when detecting id ambiguity If we find several objects with the same prefix, we need to free the memory where we stored the earlier object. Keep track of the raw.data pointer across read_prefix calls and free it if we find another object.

904b67e6

2012-05-18T01:48:50

errors: Rename error codes

e172cf08

2012-05-18T01:21:06

errors: Rename the generic return codes

24634c6f

2012-05-12T15:01:39

Handle duplicate objects from different backends in git_odb_read_prefix().

thodg/libgit2/src/odb.c

src/odb.c

Log