src/odb_pack.c


Log

Author Commit Date CI Message
lhchavez f847fa7b 2020-02-16T02:00:56 midx: Support multi-pack-index files in odb_pack.c This change adds support for reading multi-pack-index files from the packfile odb backend. This also makes git_pack_file objects open their backing failes lazily in more scenarios, since the multi-pack-index can avoid having to open them in some cases (yay!). This change also refreshes the documentation found in src/odb_pack.c to match the updated code. Part of: #5399
Edward Thomson a072a025 2020-04-05T17:34:12 odb_pack: use GIT_ASSERT
Patrick Steinhardt f0f1cd1d 2020-02-07T10:51:17 sha1_lookup: inline its only function into "pack.c" The file "sha1_lookup.c" contains a single function `sha1_position` only which is used only in the packfile implementation. As the function is comparatively small, to enable the compiler to optimize better and to remove symbol visibility, move it into "pack.c".
Patrick Steinhardt e54343a4 2019-06-29T09:17:32 fileops: rename to "futils.h" to match function signatures Our file utils functions all have a "futils" prefix, e.g. `git_futils_touch`. One would thus naturally guess that their definitions and implementation would live in files "futils.h" and "futils.c", respectively, but in fact they live in "fileops.h". Rename the files to match expectations.
Edward Thomson a1ef995d 2019-02-21T10:33:30 indexer: use git_indexer_progress throughout Update internal usage of `git_transfer_progress` to `git_indexer_progreses`.
Edward Thomson f673e232 2018-12-27T13:47:34 git_error: use new names in internal APIs and usage Move to the `git_error` name in the internal API for error-related functions.
Marijan Šuflaj f7416509 2019-01-20T20:15:31 Fix odb foreach to also close on positive error code In include/git2/odb.h it states that callback can also return positive value which should break looping. Implementations of git_odb_foreach() and pack_backend__foreach() did not respect that.
Edward Thomson 168fe39b 2018-11-28T14:26:57 object_type: use new enumeration names Use the new object_type enumeration names within the codebase.
bisho d22cd1f4 2018-09-05T11:49:13 Prevent heap-buffer-overflow When running repack while doing repo writes, `packfile_load__cb()` can see some temporary files in the directory that are bigger than the usual, and makes `memcmp` overflow on the `p->pack_name` string. ASAN detected this. This just uses `strncmp`, that should not have any performance impact and is safe for comparing strings of different sizes. ``` ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61200001a3f3 at pc 0x7f4a9e1976ec bp 0x7ffc1f80e100 sp 0x7ffc1f80d8b0 READ of size 89 at 0x61200001a3f3 thread T0 SCARINESS: 26 (multi-byte-read-heap-buffer-overflow) #0 0x7f4a9e1976eb in __interceptor_memcmp.part.78 (/build/cfgr-admin#link-tree/libtools_build_sanitizers_asan-ubsan-py.so+0xcf6eb) #1 0x7f4a518c5431 in packfile_load__cb /build/libgit2/0.27.0/src/libgit2-0.27.0/src/odb_pack.c:213 #2 0x7f4a518d9582 in git_path_direach /build/libgit2/0.27.0/src/libgit2-0.27.0/src/path.c:1134 #3 0x7f4a518c58ad in pack_backend__refresh /build/libgit2/0.27.0/src/libgit2-0.27.0/src/odb_pack.c:347 #4 0x7f4a518c1b12 in git_odb_refresh /build/libgit2/0.27.0/src/libgit2-0.27.0/src/odb.c:1511 #5 0x7f4a518bff5f in git_odb__freshen /build/libgit2/0.27.0/src/libgit2-0.27.0/src/odb.c:752 #6 0x7f4a518c17d4 in git_odb_stream_finalize_write /build/libgit2/0.27.0/src/libgit2-0.27.0/src/odb.c:1415 #7 0x7f4a51b9d015 in Repository_write /build/pygit2/0.27.0/src/pygit2-0.27.0/src/repository.c:509 ```
Patrick Steinhardt 261267e0 2018-06-22T13:19:47 odb_pack: fix passing partially initialized indexer options
Patrick Steinhardt c16556aa 2017-11-12T10:31:48 indexer: introduce options struct to `git_indexer_new` We strive to keep an options structure to many functions to be able to extend options in the future without breaking the API. `git_indexer_new` doesn't have one right now, but we want to be able to add an option for enabling strict packfile verification. Add a new `git_indexer_options` structure and adjust callers to use that.
Patrick Steinhardt ecf4f33a 2018-02-08T11:14:48 Convert usage of `git_buf_free` to new `git_buf_dispose`
Patrick Steinhardt 0c7f49dd 2017-06-30T13:39:01 Make sure to always include "common.h" first Next to including several files, our "common.h" header also declares various macros which are then used throughout the project. As such, we have to make sure to always include this file first in all implementation files. Otherwise, we might encounter problems or even silent behavioural differences due to macros or defines not being defined as they should be. So in fact, our header and implementation files should make sure to always include "common.h" first. This commit does so by establishing a common include pattern. Header files inside of "src" will now always include "common.h" as its first other file, separated by a newline from all the other includes to make it stand out as special. There are two cases for the implementation files. If they do have a matching header file, they will always include this one first, leading to "common.h" being transitively included as first file. If they do not have a matching header file, they instead include "common.h" as first file themselves. This fixes the outlined problems and will become our standard practice for header and source files inside of the "src/" from now on.
Patrick Steinhardt 8d452448 2017-03-20T09:34:59 odb_pack: initialize `git_rawobj` structure The `pack_entry_find_prefix` function receives a `git_rawobj` structure as argument. While the function first initializes the structure to a sensible state, Coverity is unable to correctly detect this, resulting in a warning. Fix this warning by initializing the object to all-zeroes before passing it to the function.
Edward Thomson 27051d4e 2016-07-22T13:34:19 odb: only freshen pack files every 2 seconds Since writing multiple objects may all already exist in a single packfile, avoid freshening that packfile repeatedly in a tight loop. Instead, only freshen pack files every 2 seconds.
Edward Thomson 8f09a98e 2016-07-14T16:23:24 odb: freshen existing objects when writing When writing an object, we calculate its OID and see if it exists in the object database. If it does, we need to freshen the file that contains it.
Edward Thomson 2381d9e4 2016-08-03T17:01:48 mwindow: init mwindow files in git_libgit2_init
Edward Thomson 6a2d2f8a 2015-06-17T06:42:20 delta: move delta application to delta.c Move the delta application functions into `delta.c`, next to the similar delta creation functions. Make the `git__delta_apply` functions adhere to other naming and parameter style within the library.
Edward Thomson e10144ae 2016-03-04T01:18:30 odb: improved not found error messages When looking up an abbreviated oid, show the actual (abbreviated) oid the caller passed instead of a full (but ambiguously truncated) oid.
Vicent Marti 43820f20 2015-10-14T19:24:07 odb: Be smarter when refreshing backends In the current implementation of ODB backends, each backend is tasked with refreshing itself after a failed lookup. This is standard Git behavior: we want to e.g. reload the packfiles on disk in case they have changed and that's the reason we can't find the object we're looking for. This behavior, however, becomes pathological in repositories where multiple alternates have been loaded. Given that each alternate counts as a separate backend, a miss in the main repository (which can potentially be very frequent in cases where object storage comes from the alternate) will result in refreshing all its packfiles before we move on to the alternate backend where the object will most likely be found. To fix this, the code in `odb.c` has been refactored as to perform the refresh of all the backends externally, once we've verified that the object is nowhere to be found. If the refresh is successful, we then perform the lookup sequentially through all the backends, skipping the ones that we know for sure weren't refreshed (because they have no refresh API). The on-disk pack backend has been adjusted accordingly: it no longer performs refreshes internally.
Carlos Martín Nieto 02980bdc 2015-06-09T16:53:07 Initialize a few variables Coverity complains about the git_rawobj ones because we use a loop in which we keep remembering the old version, and we end up copying our object as the base, so we want to have the data pointer be NULL.
Carlos Martín Nieto b3b66c57 2014-06-18T17:13:12 Share packs across repository instances Opening the same repository multiple times will currently open the same file multiple times, as well as map the same region of the file multiple times. This is not necessary, as the packfile data is immutable. Instead of opening and closing packfiles directly, introduce an indirection and allocate packfiles globally. This does mean locking on each packfile open, but we already use this lock for the global mwindow list so it doesn't introduce a new contention point.
Jacques Germishuys 48e60ae7 2014-04-21T11:23:29 Don't redefine the same callback types, their signatures may change
Russell Belfer f5753999 2014-03-04T15:34:23 Add exists_prefix to ODB backend and ODB API
Russell Belfer 96869a4e 2013-12-03T16:45:39 Improve GIT_EUSER handling This adds giterr_user_cancel to return GIT_EUSER and clear any error message that is sitting around. As a result of using that in places, we need to be more thorough with capturing errors that happen inside a callback when used internally. To help with that, this also adds giterr_capture and giterr_restore so that when we internally use a foreach-type function that clears errors and converts them to GIT_EUSER, it is easier to restore not just the return value, but the actual error message text.
Russell Belfer 25e0b157 2013-12-06T15:07:57 Remove converting user error to GIT_EUSER This changes the behavior of callbacks so that the callback error code is not converted into GIT_EUSER and instead we propagate the return value through to the caller. Instead of using the giterr_capture and giterr_restore functions, we now rely on all functions to pass back the return value from a callback. To avoid having a return value with no error message, the user can call the public giterr_set_str or some such function to set an error message. There is a new helper 'giterr_set_callback' that functions can invoke after making a callback which ensures that some error message was set in case the callback did not set one. In places where the sign of the callback return value is meaningful (e.g. positive to skip, negative to abort), only the negative values are returned back to the caller, obviously, since the other values allow for continuing the loop. The hardest parts of this were in the checkout code where positive return values were overloaded as meaningful values for checkout. I fixed this by adding an output parameter to many of the internal checkout functions and removing the overload. This added some code, but it is probably a better implementation. There is some funkiness in the network code where user provided callbacks could be returning a positive or a negative value and we want to rely on that to cancel the loop. There are still a couple places where an user error might get turned into GIT_EUSER there, I think, though none exercised by the tests.
Edward Thomson 1e60e5f4 2013-11-07T12:03:44 Allow callers to set mode on packfile creation
Carlos Martín Nieto a6154f21 2013-10-30T15:00:05 indexer: remove the stream infix It was there to keep it apart from the one which read in from a file on disk. This other indexer does not exist anymore, so there is no need for anything other than git_indexer to refer to it. While here, rename _add() function to _append() and _finalize() to _commit(). The former change is cosmetic, while the latter avoids talking about "finalizing", which OO languages use to mean something completely different.
Vicent Martí 5c50f22a 2013-10-28T09:25:44 Merge pull request #1891 from libgit2/cmn/fix-thin-packs Add support for thin packs
Carlos Martín Nieto 0b33fca0 2013-10-02T13:39:35 indexer: fix thin packs When given an ODB from which to read objects, the indexer will attempt to inject the missing bases at the end of the pack and update the header and trailer to reflect the new contents.
Russell Belfer 219d3457 2013-10-01T16:12:15 Initial iconv hookup for precomposed unicode This hooks up git_path_direach and git_path_dirload so that they will take a flag indicating if directory entry names should be tested and converted from decomposed unicode to precomposed form. This code will only come into play on the Apple platform and even then, only when certain types of filesystems are used. This involved adding a flag to these functions which involved changing a lot of places in the code. This was an opportunity to do a bit of code cleanup here and there, for example, getting rid of the git_futils_cleanupdir_r function in favor of a simple flag to git_futils_rmdir_r to not remove the top level entry. That ended up adding depth tracking during rmdir_r which led to a safety check for infinite directory recursion. Yay. This hasn't actually been tested on the Mac filesystems where the issue occurs. I still need to get test environment for that.
nulltoken d0cd6c42 2013-09-08T18:22:28 path: Make direach() return EUSER on callback error
nulltoken b1a6c316 2013-08-30T17:36:00 odb: Move the auto refresh logic to the pack backend Previously, `git_object_read()`, `git_object_read_prefix()` and `git_object_exists()` were implementing an auto refresh logic. When the expected object couldn't be found in any backend, a call to `git_odb_refresh()` was triggered and the lookup was once again performed against all backends. This commit removes this auto-refresh logic from the odb layer and pushes it down into the pack-backend (as it's the only one currently exposing a `refresh()` endpoint).
nulltoken 9b4ed214 2013-08-30T17:07:41 odb: Code beautification
Brodie Rao d19bcb33 2013-06-06T14:49:14 odb_pack: handle duplicate objects from different packs This is based on 24634c6fd02b2240e4a93fad70a08220f8fb793a. This also corrects an issue with error codes being mixed up with the number of found objects.
Russell Belfer 5d2d21e5 2013-04-16T15:00:43 Consolidate packfile allocation further Rename git_packfile_check to git_packfile_alloc since it is now being used more in that capacity. Fix the various places that use it. Consolidate some repeated code in odb_pack.c related to the allocation of a new pack_backend.
Russell Belfer 83cc70d9 2013-04-19T12:48:33 Move odb_backend implementors stuff into git2/sys This moves some of the odb_backend stuff that is related to the internals of an odb_backend implementation into include/git2/sys. Some of the stuff related to streaming I left in include/git2 because it seemed like it would be reasonably needed by a normal user who wanted to stream objects into and out of the ODB. Also, I added APIs for traversing the list of backends so that some of the tests would not need to access ODB internals.
Arkadiy Shapkin 10c06114 2013-03-17T04:46:46 Several warnings detected by static code analyzer fixed Implicit type conversion argument of function to size_t type Suspicious sequence of types castings: size_t -> int -> size_t Consider reviewing the expression of the 'A = B == C' kind. The expression is calculated as following: 'A = (B == C)' Unsigned type is never < 0
Vicent Marti a5f61384 2013-03-15T12:24:20 odb_pack: Unused functions
Carlos Martín Nieto 96c9b9f0 2013-01-12T18:38:19 indexer: properly free the packfile resources The indexer needs to call the packfile's free function so it takes care of freeing the caches. We still need to close the mwf descriptor manually so we can rename the packfile into its final name on Windows.
Carlos Martín Nieto 80d647ad 2013-01-11T20:15:06 Revert "pack: packfile_free -> git_packfile_free and use it in the indexers" This reverts commit f289f886cb81bb570bed747053d5ebf8aba6bef7, which makes the tests fail on Windows. Revert until we can figure out a solution.
Carlos Martín Nieto f289f886 2013-01-11T17:24:52 pack: packfile_free -> git_packfile_free and use it in the indexers It turns out the indexers have been ignoring the pack's free function and leaking data. Plug that.
Vicent Marti 891a4681 2013-01-04T17:42:41 dat errorcode
Vicent Marti 4a863c06 2013-01-03T20:36:26 Sane refresh logic All the ODB backends have a specific refresh interface. When reading an object, first we attempt every single backend: if the read fails, then we refresh all the backends and retry the read one more time to see if the object has appeared.
Edward Thomson 359fc2d2 2013-01-08T17:07:25 update copyrights
Vicent Martí e05ca13f 2012-12-05T11:47:19 Merge pull request #1115 from ben/struct-versions Version info for public structs
David Michael Barr bfb8bcc1 2012-12-03T10:36:32 odb-pack: resurrect pack_backend__read_header
Ben Straub 55f6f21b 2012-11-29T19:59:18 Deploy versioned git_odb_backend structure
Russell Belfer c3fb7d04 2012-11-27T15:00:49 Make git_odb_foreach_cb take const param This makes the first OID param of the ODB callback a const pointer and also propogates that change all the way to the backends.
Edward Thomson 09cc0b92 2012-11-05T11:33:10 create callback to handle packs from fetch, move the indexer to odb_pack
Ben Straub 63409451 2012-09-19T04:55:16 ODB pack: snapshot last_found to avoid race Also removed unnecessary refresh call and fixed some indentation.
Ben Straub 78216495 2012-09-13T20:08:05 Remove mtime checks from ODB packfile backend Now forcing refresh on a foreach, and on missed full-oid or short-oid lookups.
Ben Straub 5bb0dc93 2012-09-13T14:02:46 ODB: re-load packfiles on failed lookup The old method was avoiding re-loading of packfiles by watching the mtime of the pack directory. This causes the ODB to become stale if the directory and packfile are written within the same clock millisecond, as when cloning a fairly small repo. This method tries to find the object in the cached packs, and forces a refresh when that fails. This will cause extra stat'ing on a miss, but speeds up the success case and avoids this race condition.
Michael Schubert 3d7617e4 2012-09-14T21:33:50 odb_pack: fix race condition last_found is the last packfile a wanted object was found in. Since last_found is shared among all searching threads, it might changes while we're searching. As suggested by @arrbee, put a copy on the stack to fix the race condition.
David Michael Barr ab8a0402 2012-09-12T14:26:31 odb_pack: try lookup before refreshing packs This reduces the rate of syscalls for the common case of sequences of object reads from the same pack. Best of 5 timings for libgit2_clar before this patch: real 0m5.375s user 0m0.392s sys 0m3.564s After applying this patch: real 0m5.285s user 0m0.356s sys 0m3.544s 0.6% improvement in system time. 9.2% improvement in user time. 1.7% improvement in elapsed time. Confirmed a 0.6% reduction in number of system calls with strace. Expect greater improvement for graph-traversal with large packs.
Carlos Martín Nieto f9988d4e 2012-09-04T21:42:00 odb: pass the user's data pointer correctly in foreach
Vicent Marti 51e1d808 2012-08-06T12:41:08 Merge remote-tracking branch 'arrbee/tree-walk-fixes' into development Conflicts: src/notes.c src/transports/git.c src/transports/http.c src/transports/local.c tests-clar/odb/foreach.c
Russell Belfer 5dca2010 2012-08-03T17:08:01 Update iterators for consistency across library This updates all the `foreach()` type functions across the library that take callbacks from the user to have a consistent behavior. The rules are: * A callback terminates the loop by returning any non-zero value * Once the callback returns non-zero, it will not be called again (i.e. the loop stops all iteration regardless of state) * If the callback returns non-zero, the parent fn returns GIT_EUSER * Although the parent returns GIT_EUSER, no error will be set in the library and `giterr_last()` will return NULL if called. This commit makes those changes across the library and adds tests for most of the iteration APIs to make sure that they follow the above rules.
Vicent Marti e25dda51 2012-08-02T01:38:30 Merge remote-tracking branch 'nulltoken/topic/amd64-compat' into development Conflicts: src/netops.c src/netops.h src/oid.c
nulltoken b8457baa 2012-07-24T07:57:58 portability: Improve x86/amd64 compatibility
Carlos Martín Nieto 507523c3 2012-07-21T16:23:49 odb: allow creating an ODB backend from a packfile index git_odb_backend_one_packfile() allows us to create an ODB backend out of an .idx file.
Carlos Martín Nieto 521aedad 2012-06-05T14:48:51 odb: add git_odb_foreach() Go through each backend and list every objects that exists in them. This allows fsck-like uses.
Vicent Martí 904b67e6 2012-05-18T01:48:50 errors: Rename error codes
Vicent Martí e172cf08 2012-05-18T01:21:06 errors: Rename the generic return codes
Russell Belfer 282283ac 2012-05-04T16:46:46 Fix valgrind issues There are three changes here: - correctly propogate error code from failed object lookups - make zlib inflate use our allocators - add OID to notfound error in ODB lookups
nulltoken fa6420f7 2012-04-29T21:46:33 buf: deploy git_buf_len()
Russell Belfer 44ef8b1b 2012-04-13T13:00:10 Fix warnings on 64-bit windows builds This fixes all the warnings on win64 except those in deps, which come from the regex code.
Russell Belfer 0d0fa7c3 2012-03-16T15:56:01 Convert attr, ignore, mwindow, status to new errors Also cleaned up some previously converted code that still had little things to polish.
Russell Belfer e1de726c 2012-03-12T22:55:40 Migrate ODB files to new error handling This migrates odb.c, odb_loose.c, odb_pack.c and pack.c to the new style of error handling. Also got the unix and win32 versions of map.c. There are some minor changes to other files but no others were completely converted. This also contains an update to filebuf so that a zeroed out filebuf will not think that the fd (== 0) is actually open (and inadvertently call close() on fd 0 if cleaned up). Lastly, this was built and tested on win32 and contains a bunch of fixes for the win32 build which was pretty broken.
Vicent Martí 1a481123 2012-02-17T00:13:34 error-handling: References Yes, this is error handling solely for `refs.c`, but some of the abstractions leak all ofer the code base.
Russell Belfer 854eccbb 2012-02-29T12:04:59 Clean up GIT_UNUSED macros on all platforms It turns out that commit 31e9cfc4cbcaf1b38cdd3dbe3282a8f57e5366a5 did not fix the GIT_USUSED behavior on all platforms. This commit walks through and really cleans things up more thoroughly, getting rid of the unnecessary stuff. To remove the use of some GIT_UNUSED, I ended up adding a couple of new iterators for hashtables that allow you to iterator just over keys or just over values. In making this change, I found a bug in the clar tests (where we were doing *count++ but meant to do (*count)++ to increment the value). I fixed that but then found the test failing because it was not really using an empty repo. So, I took some of the code that I wrote for iterator testing and moved it to clar_helpers.c, then made use of that to make it easier to open fixtures on a per test basis even within a single test file.
Vicent Martí 0c3bae62 2012-02-15T16:56:56 zlib: Remove custom `git2/zlib.h` header This is legacy compat stuff for when `deflateBound` is not defined, but we're not embedding zlib and that function is always available. Kill that with fire.
schu 5e0de328 2012-02-13T17:10:24 Update Copyright header Signed-off-by: schu <schu-github@schulog.org>
Russell Belfer 1744fafe 2012-01-17T15:49:47 Move path related functions from fileops to path This takes all of the functions that look up simple data about paths (such as `git_futils_isdir`) and moves them over to path.h (becoming `git_path_isdir`). This leaves fileops.h just with functions that actually manipulate the filesystem or look at the file contents in some way. As part of this, the dir.h header which is really just for win32 support was moved into win32 (with some minor changes).
Russell Belfer 97769280 2011-11-30T11:27:15 Use git_buf for path storage instead of stack-based buffers This converts virtually all of the places that allocate GIT_PATH_MAX buffers on the stack for manipulating paths to use git_buf objects instead. The patch is pretty careful not to touch the public API for libgit2, so there are a few places that still use GIT_PATH_MAX. This extends and changes some details of the git_buf implementation to add a couple of extra functions and to make error handling easier. This includes serious alterations to all the path.c functions, and several of the fileops.c ones, too. Also, there are a number of new functions that parallel existing ones except that use a git_buf instead of a stack-based buffer (such as git_config_find_global_r that exists alongsize git_config_find_global). This also modifies the win32 version of p_realpath to allocate whatever buffer size is needed to accommodate the realpath instead of hardcoding a GIT_PATH_MAX limit, but that change needs to be tested still.
Vicent Marti 3286c408 2011-10-28T14:51:13 global: Properly use `git__` memory wrappers Ensure that all memory related functions (malloc, calloc, strdup, free, etc) are using their respective `git__` wrappers.
Sven Strickroth 599297fd 2011-10-03T23:12:43 ignore missing pack file as git does See http://code.google.com/p/tortoisegit/issues/detail?id=862 Signed-off-by: Sven Strickroth <email@cs-ware.de>
Vicent Martí 71a4c1f1 2011-09-18T20:07:59 Merge pull request #384 from kiryl/warnings Add more -W flags to CFLAGS
Vicent Marti 87d9869f 2011-09-19T03:34:49 Tabify everything There were quite a few places were spaces were being used instead of tabs. Try to catch them all. This should hopefully not break anything. Except for `git blame`. Oh well.
Vicent Marti bb742ede 2011-09-19T01:54:32 Cleanup legal data 1. The license header is technically not valid if it doesn't have a copyright signature. 2. The COPYING file has been updated with the different licenses used in the project. 3. The full GPLv2 header in each file annoys me.
Kirill A. Shutemov d568d585 2011-08-30T23:55:22 CMakefile: add -Wmissing-prototypes and fix warnings Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>
Kirill A. Shutemov 932669b8 2011-08-25T14:22:57 Drop STRLEN() macros There is no need in STRLEN macros. Compilers can do this trivial optimization on its own. Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>
Vicent Marti f6867e63 2011-08-08T16:56:28 Fix compilation in Windows
Carlos Martín Nieto 7d0cdf82 2011-07-09T02:25:01 Make packfile_unpack_header more generic On the way, store the fd and the size in the mwindow file. Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>
Carlos Martín Nieto b5b474dd 2011-07-28T11:45:46 Modify the given offset in git_packfile_unpack The callers immediately throw away the offset, so we don't need any logical changes in any of them. This will be useful for the indexer, as it does need to know where the compressed data ends. Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>
Carlos Martín Nieto a070f152 2011-07-29T01:08:02 Move pack functions to their own file
Carlos Martín Nieto 7bfdb3d2 2011-06-28T20:39:30 Factor out the mmap window code This code is useful for more things than just the packfile handling code. Factor it out so it can be reused. Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>
Carlos Martín Nieto c7c9e183 2011-07-07T10:17:40 Move the pack structs to an internal header
Kirill A. Shutemov 03cdbab4 2011-07-15T16:44:25 odb_pack: fix cast warnings /home/kas/git/public/libgit2/src/odb_pack.c: In function ‘packfile_sort__cb’: /home/kas/git/public/libgit2/src/odb_pack.c:702:24: warning: cast discards ‘__attribute__((const))’ qualifier from pointer target type [-Wcast-qual] /home/kas/git/public/libgit2/src/odb_pack.c:703:24: warning: cast discards ‘__attribute__((const))’ qualifier from pointer target type [-Wcast-qual] /home/kas/git/public/libgit2/src/odb_pack.c: In function ‘nth_packed_object_offset’: /home/kas/git/public/libgit2/src/odb_pack.c:944:10: warning: cast discards ‘__attribute__((const))’ qualifier from pointer target type [-Wcast-qual] /home/kas/git/public/libgit2/src/odb_pack.c:944:10: warning: cast discards ‘__attribute__((const))’ qualifier from pointer target type [-Wcast-qual] /home/kas/git/public/libgit2/src/odb_pack.c:944:10: warning: cast discards ‘__attribute__((const))’ qualifier from pointer target type [-Wcast-qual] /home/kas/git/public/libgit2/src/odb_pack.c:948:9: warning: cast discards ‘__attribute__((const))’ qualifier from pointer target type [-Wcast-qual] /home/kas/git/public/libgit2/src/odb_pack.c:948:9: warning: cast discards ‘__attribute__((const))’ qualifier from pointer target type [-Wcast-qual] /home/kas/git/public/libgit2/src/odb_pack.c:948:9: warning: cast discards ‘__attribute__((const))’ qualifier from pointer target type [-Wcast-qual] /home/kas/git/public/libgit2/src/odb_pack.c:952:22: warning: cast discards ‘__attribute__((const))’ qualifier from pointer target type [-Wcast-qual] /home/kas/git/public/libgit2/src/odb_pack.c:952:22: warning: cast discards ‘__attribute__((const))’ qualifier from pointer target type [-Wcast-qual] /home/kas/git/public/libgit2/src/odb_pack.c:952:22: warning: cast discards ‘__attribute__((const))’ qualifier from pointer target type [-Wcast-qual] /home/kas/git/public/libgit2/src/odb_pack.c:953:8: warning: cast discards ‘__attribute__((const))’ qualifier from pointer target type [-Wcast-qual] /home/kas/git/public/libgit2/src/odb_pack.c:953:8: warning: cast discards ‘__attribute__((const))’ qualifier from pointer target type [-Wcast-qual] /home/kas/git/public/libgit2/src/odb_pack.c:953:8: warning: cast discards ‘__attribute__((const))’ qualifier from pointer target type [-Wcast-qual] Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>
Vicent Marti de18f276 2011-07-07T01:46:20 vector: Timsort all of the things Drop the GLibc implementation of Merge Sort and replace it with Timsort. The algorithm has been tuned to work on arrays of pointers (void **), so there's no longer a need to abstract the byte-width of each element in the array. All the comparison callbacks now take pointers-to-elements, not pointers-to-pointers, so there's now one less level of dereferencing. E.g. int index_cmp(const void *a, const void *b) { - const git_index_entry *entry_a = *(const git_index_entry **)(a); + const git_index_entry *entry_a = (const git_index_entry *)(a); The result is up to a 40% speed-up when sorting vectors. Memory usage remains lineal. A new `bsearch` implementation has been added, whose callback also supplies pointer-to-elements, to uniform the Vector API again.
Carlos Martín Nieto 2ee318a7 2011-07-05T15:09:17 Small fixes in pack_window_open Check if the window structure has actually been allocated before trying to access it, and don't leak said structure if the map fails. Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Vicent Marti f79026b4 2011-07-04T11:43:34 fileops: Cleanup Cleaned up the structure of the whole OS-abstraction layer. fileops.c now contains a set of utility methods for file management used by the library. These are abstractions on top of the original POSIX calls. There's a new file called `posix.c` that contains emulations/reimplementations of all the POSIX calls the library uses. These are prefixed with `p_`. There's a specific posix file for each platform (win32 and unix). All the path-related methods have been moved from `utils.c` to `path.c` and have their own prefix.
Kirill A. Shutemov 932d1baf 2011-06-30T19:52:34 cleanup: remove trailing spaces Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>
Carlos Martín Nieto cdb6f9bf 2011-06-20T17:34:01 Allocate enough memory for the terminator in commit parsing Also allow space for the null-terminator when allocating the buffer in packfile_unpack_compressed. Up to now, the last newline had served as a terminator, but 858ef372 searches for a double-newline and exposes the problem. Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Vicent Marti f1d01851 2011-06-16T02:48:48 oid: Uniformize ncmp methods Drop redundant methods. The ncmp method is now public
Vicent Marti fa48608e 2011-06-16T02:36:21 oid: Rename methods Yeah. Finally. Fuck the old names, this ain't POSIX and they don't make any sense at all.
Vicent Martí 1b0d92b1 2011-06-06T18:28:11 Merge pull request #238 from pegonma/git_oid_ncmp Better name for git_oid_match
Marc Pegon c09093cc 2011-06-06T10:55:36 Renamed git_oid_match to git_oid_ncmp. As suggested by carlosmn, git_oid_ncmp would probably be a better name than git_oid_match, for it does the same as git_oid_cmp but only up to a certain amount of hex digits.
Johan 't Hart 393a9f9e 2011-06-06T00:33:23 Fix build errors on MSVC
Vicent Marti 7107b599 2011-06-02T01:03:52 odb-pack: More variable declarations