thodg/libgit2/src/odb.c

src/odb.c

Log

Author Commit Date CI Message
Russell Belfer 282283ac 2012-05-04T16:46:46 Fix valgrind issues There are three changes here: - correctly propogate error code from failed object lookups - make zlib inflate use our allocators - add OID to notfound error in ODB lookups
Russell Belfer 2bc8fa02 2012-04-17T10:14:24 Implement git_pool paged memory allocator This adds a `git_pool` object that can do simple paged memory allocation with free for the entire pool at once. Using this, you can replace many small allocations with large blocks that can then cheaply be doled out in small pieces. This is best used when you plan to free the small blocks all at once - for example, if they represent the parsed state from a file or data stream that are either all kept or all discarded. There are two real patterns of usage for `git_pools`: either for "string" allocation, where the item size is a single byte and you end up just packing the allocations in together, or for "fixed size" allocation where you are allocating a large object (e.g. a `git_oid`) and you generally just allocation single objects that can be tightly packed. Of course, you can use it for other things, but those two cases are the easiest.
Russell Belfer 4aa7de15 2012-03-19T17:49:46 Convert indexer, notes, sha1_lookup, and signature More files moved to new error handling style.
Russell Belfer deafee7b 2012-03-14T17:36:15 Continue error conversion This converts blob.c, fileops.c, and all of the win32 files. Also, various minor cleanups throughout the code. Plus, in testing the win32 build, I cleaned up a bunch (although not all) of the warnings with the 64-bit build.
Russell Belfer e1de726c 2012-03-12T22:55:40 Migrate ODB files to new error handling This migrates odb.c, odb_loose.c, odb_pack.c and pack.c to the new style of error handling. Also got the unix and win32 versions of map.c. There are some minor changes to other files but no others were completely converted. This also contains an update to filebuf so that a zeroed out filebuf will not think that the fd (== 0) is actually open (and inadvertently call close() on fd 0 if cleaned up). Lastly, this was built and tested on win32 and contains a bunch of fixes for the win32 build which was pretty broken.
Russell Belfer 998f7b3d 2012-03-07T10:52:17 Fix issues raised on pull request This resolves the comments on pull request #590
Russell Belfer ae9e29fd 2012-03-06T16:14:31 Migrating diff to new error handling Ended up migrating a bunch of upstream functions as well including vector, attr_file, and odb in order to get this to work right.
Vicent Martí 1a481123 2012-02-17T00:13:34 error-handling: References Yes, this is error handling solely for `refs.c`, but some of the abstractions leak all ofer the code base.
Vicent Martí 13224ea4 2012-02-27T04:28:31 buffer: Unify `git_fbuffer` and `git_buf` This makes so much sense that I can't believe it hasn't been done before. Kill the old `git_fbuffer` and read files straight into `git_buf` objects. Also: In order to fully support 4GB files in 32-bit systems, the `git_buf` implementation has been changed from using `ssize_t` for storage and storing negative values on allocation failure, to using `size_t` and changing the buffer pointer to a magical pointer on allocation failure. Hopefully this won't break anything.
Russell Belfer 1ec1de6d 2012-02-23T11:15:45 Fix warnings about type conversion on win32
Vicent Martí 0c3bae62 2012-02-15T16:56:56 zlib: Remove custom `git2/zlib.h` header This is legacy compat stuff for when `deflateBound` is not defined, but we're not embedding zlib and that function is always available. Kill that with fire.
schu 5e0de328 2012-02-13T17:10:24 Update Copyright header Signed-off-by: schu <schu-github@schulog.org>
Vicent Martí f19e3ca2 2012-02-10T20:16:42 odb: Proper symlink hashing
Vicent Martí 18e5b854 2012-02-10T19:47:02 odb: Add internal `git_odb__hashfd`
Russell Belfer 1744fafe 2012-01-17T15:49:47 Move path related functions from fileops to path This takes all of the functions that look up simple data about paths (such as `git_futils_isdir`) and moves them over to path.h (becoming `git_path_isdir`). This leaves fileops.h just with functions that actually manipulate the filesystem or look at the file contents in some way. As part of this, the dir.h header which is really just for win32 support was moved into win32 (with some minor changes).
Russell Belfer 97769280 2011-11-30T11:27:15 Use git_buf for path storage instead of stack-based buffers This converts virtually all of the places that allocate GIT_PATH_MAX buffers on the stack for manipulating paths to use git_buf objects instead. The patch is pretty careful not to touch the public API for libgit2, so there are a few places that still use GIT_PATH_MAX. This extends and changes some details of the git_buf implementation to add a couple of extra functions and to make error handling easier. This includes serious alterations to all the path.c functions, and several of the fileops.c ones, too. Also, there are a number of new functions that parallel existing ones except that use a git_buf instead of a stack-based buffer (such as git_config_find_global_r that exists alongsize git_config_find_global). This also modifies the win32 version of p_realpath to allocate whatever buffer size is needed to accommodate the realpath instead of hardcoding a GIT_PATH_MAX limit, but that change needs to be tested still.
Vicent Marti 45e79e37 2011-11-26T04:59:21 Rename all `_close` methods There's no difference between `_free` and `_close` semantics: keep everything with the same name to avoid confusions.
Vicent Marti 9462c471 2011-11-25T08:16:26 repository: Change ownership semantics The ownership semantics have been changed all over the library to be consistent. There are no more "borrowed" or duplicated references. Main changes: - `git_repository_open2` and `3` have been dropped. - Added setters and getters to hotswap all the repository owned objects: `git_repository_index` `git_repository_set_index` `git_repository_odb` `git_repository_set_odb` `git_repository_config` `git_repository_set_config` `git_repository_workdir` `git_repository_set_workdir` Now working directories/index files/ODBs and so on can be hot-swapped after creating a repository and between operations. - All these objects now have proper ownership semantics with refcounting: they all require freeing after they are no longer needed (the repository always keeps its internal reference). - Repository open and initialization has been updated to keep in mind the configuration files. Bare repositories are now always detected, and a default config file is created on init. - All the tests affected by these changes have been dropped from the old test suite and ported to the new one.
Vicent Marti 3286c408 2011-10-28T14:51:13 global: Properly use `git__` memory wrappers Ensure that all memory related functions (malloc, calloc, strdup, free, etc) are using their respective `git__` wrappers.
Vicent Marti 8af4d074 2011-09-29T15:34:17 odb: Let users decide compression level for the loose ODB
Vicent Marti 87d9869f 2011-09-19T03:34:49 Tabify everything There were quite a few places were spaces were being used instead of tabs. Try to catch them all. This should hopefully not break anything. Except for `git blame`. Oh well.
Vicent Marti bb742ede 2011-09-19T01:54:32 Cleanup legal data 1. The license header is technically not valid if it doesn't have a copyright signature. 2. The COPYING file has been updated with the different licenses used in the project. 3. The full GPLv2 header in each file annoys me.
Vicent Marti 84dd3820 2011-08-18T02:13:51 posix: Properly handle `snprintf` in all platforms
Vicent Marti c85e08b1 2011-08-16T13:05:05 odb: Do not pass around a header when hashing
nulltoken b21fb849 2011-07-09T06:36:18 Fix MSVC compilation warning
Vicent Marti c52736fa 2011-07-09T15:05:14 status: Cleanup The `hashfile` function has been moved to ODB, next to `git_odb_hash`. Global state has been removed from the dirent call in `status.c`, because global state is killing the rainforest and causing global warming.
Vicent Marti de18f276 2011-07-07T01:46:20 vector: Timsort all of the things Drop the GLibc implementation of Merge Sort and replace it with Timsort. The algorithm has been tuned to work on arrays of pointers (void **), so there's no longer a need to abstract the byte-width of each element in the array. All the comparison callbacks now take pointers-to-elements, not pointers-to-pointers, so there's now one less level of dereferencing. E.g. int index_cmp(const void *a, const void *b) { - const git_index_entry *entry_a = *(const git_index_entry **)(a); + const git_index_entry *entry_a = (const git_index_entry *)(a); The result is up to a 40% speed-up when sorting vectors. Memory usage remains lineal. A new `bsearch` implementation has been added, whose callback also supplies pointer-to-elements, to uniform the Vector API again.
Vicent Marti f79026b4 2011-07-04T11:43:34 fileops: Cleanup Cleaned up the structure of the whole OS-abstraction layer. fileops.c now contains a set of utility methods for file management used by the library. These are abstractions on top of the original POSIX calls. There's a new file called `posix.c` that contains emulations/reimplementations of all the POSIX calls the library uses. These are prefixed with `p_`. There's a specific posix file for each platform (win32 and unix). All the path-related methods have been moved from `utils.c` to `path.c` and have their own prefix.
Kirill A. Shutemov 932d1baf 2011-06-30T19:52:34 cleanup: remove trailing spaces Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>
Vicent Marti 984ed6b6 2011-06-19T12:48:16 odb: Add GIT_EPASSTHROUGH Allows a custom user backend to passthrough one of the callbacks. Used for e.g. caching backends.
Vicent Marti 0291b5b7 2011-06-03T19:59:16 odb: Fix loading ODB alternates Fixed an issue with the `strtokz implementation and added support for comments and relative paths in the alternates file.
Vicent Martí 1e9b7a09 2011-06-02T15:12:37 Merge pull request #144 from nordsturm/fix_fakewstream Fix fake wstream write
Marc Pegon 6c8ca697 2011-05-29T17:57:25 Fixed some error messages related to searching objects from a short oid. Fixed forgot to check that prefix length is greater than minimum prefix length in read_unique_short_oid method from pack backend.
Vicent Marti d0323a5f 2011-06-01T21:25:56 short-oid: Cleanup
Marc Pegon dd453c4d 2011-05-27T22:46:41 Added git.git sha1 lookup method to replace simple binary search in pack backend. Implemented find_unique_short_oid for pack backend, based on git sha1 lookup method; finding an object given its full oid is just a particular case of searching the unique object matching an oid prefix (short oid). Added git_odb_read_unique_short_oid, which iterates over all the backends to find and read the unique object matching the given oid prefix. Added a git_object_lookup_short_oid method to find the unique object in the repository matching a given oid prefix : it generalizes git_object_lookup which now does nothing but calls git_object_lookup_short_oid.
Vicent Marti 1e85d1aa 2011-05-23T21:09:07 odb: Reword errors
Jakob Pfender d3d5d86d 2011-05-18T12:35:08 odb.c: Move to new error handling mechanism
schu 12de98c1 2011-05-18T18:00:34 Move odb.c to the new error handling Add missing free in git_odb_new(). Signed-off-by: schu <schu-github@schulog.org>
Vicent Marti 7cadd1f6 2011-05-15T23:46:22 Check error code from `git_cache_init`
Vicent Martí 3fe2e770 2011-05-15T13:34:43 Merge pull request #184 from nulltoken/repo-error-handling Updated fileops.c and repository.c to new error handling mechanism
Shuhei Tanuma 71747bca 2011-05-15T20:07:54 fix git_otype typo when calling `git_odb_read_header`.
nulltoken 81201a4c 2011-05-15T06:57:34 Move cache.c to the new error handling
Carlos Martín Nieto 39e1032c 2011-05-06T12:43:37 odb backend_sort_cmp should be static Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Sergey Nikishin 411823a3 2011-04-24T16:02:57 Fix whole buffer writing in fake wstream
Vicent Marti d69d0185 2011-04-04T13:05:20 Add a fake wstream to the ODB Streaming writes will no longer fail when writing to a backend that doesn't support streaming writes but supports direct ones. Now we create a fake stream on memory and then write it as a single block using the backend `write` callback.
Vicent Marti f6f72d7e 2011-03-23T18:44:53 Improve the ODB writing backend Temporary files when doing streaming writes are now stored inside the Objects folder, to prevent issues when moving files between disks/partitions. Add support for block writes to the ODB again (for those backends that cannot implement streaming).
Vicent Marti 1881f078 2011-03-21T20:28:02 Add getters for `git_odb_object`
Vicent Marti 72a3fe42 2011-03-18T19:38:49 I broke your bindings Hey. Apologies in advance -- I broke your bindings. This is a major commit that includes a long-overdue redesign of the whole object-database structure. This is expected to be the last major external API redesign of the library until the first non-alpha release. Please get your bindings up to date with these changes. They will be included in the next minor release. Sorry again! Major features include: - Real caching and refcounting on parsed objects - Real caching and refcounting on objects read from the ODB - Streaming writes & reads from the ODB - Single-method writes for all object types - The external API is now partially thread-safe The speed increases are significant in all aspects, specially when reading an object several times from the ODB (revwalking) and when writing big objects to the ODB. Here's a full changelog for the external API: blob.h ------ - Remove `git_blob_new` - Remove `git_blob_set_rawcontent` - Remove `git_blob_set_rawcontent_fromfile` - Rename `git_blob_writefile` -> `git_blob_create_fromfile` - Change `git_blob_create_fromfile`: The `path` argument is now relative to the repository's working dir - Add `git_blob_create_frombuffer` commit.h -------- - Remove `git_commit_new` - Remove `git_commit_add_parent` - Remove `git_commit_set_message` - Remove `git_commit_set_committer` - Remove `git_commit_set_author` - Remove `git_commit_set_tree` - Add `git_commit_create` - Add `git_commit_create_v` - Add `git_commit_create_o` - Add `git_commit_create_ov` tag.h ----- - Remove `git_tag_new` - Remove `git_tag_set_target` - Remove `git_tag_set_name` - Remove `git_tag_set_tagger` - Remove `git_tag_set_message` - Add `git_tag_create` - Add `git_tag_create_o` tree.h ------ - Change `git_tree_entry_2object`: New signature is `(git_object **object_out, git_repository *repo, git_tree_entry *entry)` - Remove `git_tree_new` - Remove `git_tree_add_entry` - Remove `git_tree_remove_entry_byindex` - Remove `git_tree_remove_entry_byname` - Remove `git_tree_clearentries` - Remove `git_tree_entry_set_id` - Remove `git_tree_entry_set_name` - Remove `git_tree_entry_set_attributes` object.h ------------ - Remove `git_object_new - Remove `git_object_write` - Change `git_object_close`: This method is now *mandatory*. Not closing an object causes a memory leak. odb.h ----- - Remove type `git_rawobj` - Remove `git_rawobj_close` - Rename `git_rawobj_hash` -> `git_odb_hash` - Change `git_odb_hash`: New signature is `(git_oid *id, const void *data, size_t len, git_otype type)` - Add type `git_odb_object` - Add `git_odb_object_close` - Change `git_odb_read`: New signature is `(git_odb_object **out, git_odb *db, const git_oid *id)` - Change `git_odb_read_header`: New signature is `(size_t *len_p, git_otype *type_p, git_odb *db, const git_oid *id)` - Remove `git_odb_write` - Add `git_odb_open_wstream` - Add `git_odb_open_rstream` odb_backend.h ------------- - Change type `git_odb_backend`: New internal signatures are as follows int (* read)(void **, size_t *, git_otype *, struct git_odb_backend *, const git_oid *) int (* read_header)(size_t *, git_otype *, struct git_odb_backend *, const git_oid *) int (* writestream)(struct git_odb_stream **, struct git_odb_backend *, size_t, git_otype) int (* readstream)( struct git_odb_stream **, struct git_odb_backend *, const git_oid *) - Add type `git_odb_stream` - Add enum `git_odb_streammode` Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti 86d7e1ca 2011-02-28T12:46:13 Fix searching in git_vector We now store only one sorting callback that does entry comparison. This is used when sorting the entries using a quicksort, and when looking for a specific entry with the new search methods. The following search methods now exist: git_vector_search(vector, entry) git_vector_search2(vector, custom_search_callback, key) git_vector_bsearch(vector, entry) git_vector_bsearch2(vector, custom_search_callback, key) The sorting state of the vector is now stored internally. Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti d4b5a4e2 2011-02-09T19:49:02 Internal changes on the backend system The priority value for different backends has been removed from the public `git_odb_backend` struct. We handle that internally. The priority value is specified on the `git_odb_add_alternate`. This is convenient because it allows us to poll a backend twice with different priorities without having to instantiate it twice. We also differentiate between main backends and alternates; alternates have lower priority and cannot be written to. These changes come with some unit tests to make sure that the backend sorting is consistent. The libgit2 version has been bumped to 0.4.0. This commit changes the external API: CHANGED: struct git_odb_backend No longer has a `priority` attribute; priority for the backend in managed internally by the library. git_odb_add_backend(git_odb *odb, git_odb_backend *backend, int priority) Now takes an additional priority parameter, the priority that will be given to the backend. ADDED: git_odb_add_alternate(git_odb *odb, git_odb_backend *backend, int priority) Add a backend as an alternate. Alternate backends have always lower priority than main backends, and writing is disabled on them. Signed-off-by: Vicent Marti <tanoku@gmail.com> Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti 5a800efc 2011-02-09T12:46:54 Honor alternate entries in the ODB The alternates file is now parsed, and the alternate ODB folders are added as separate backends. This allows the library to efficiently query the alternate folders. Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti ec3c7a16 2011-01-13T04:54:14 Add new Repository initialization method Lets the user specify the ODB that will be used by the repository manually. Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti 1f080e2d 2010-12-13T03:43:56 Fix initialization & freeing of inexistent repos Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti 44908fe7 2010-12-06T23:03:16 Change the library include file Libgit2 is now officially include as #include "<git2.h>" or indidividual files may be included as #include <git2/index.h> Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti 7d7cd885 2010-12-03T18:01:30 Decouple storage from ODB logic Comes with two default backends: loose object and packfiles. Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti d12299fe 2010-12-03T22:22:10 Change include structure for the project The maze with include dependencies has been fixed. There is now a global include: #include <git.h> The git_odb_backend API has been exposed. Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti 86bfec39 2010-12-06T01:13:09 Revert changes in odb.c to eventually rebase The odb.c will disappear after the decoupling changes. Signed-off-by: Vicent Marti <tanoku@gmail.com>
nulltoken 6f02c3ba 2010-12-05T20:18:56 Small source code readability improvements. Replaced magic number "0" with GIT_SUCCESS constant wherever it made sense.
Vicent Marti 824f4acd 2010-11-23T22:33:28 Fix redeclaration warnings in MSVC Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti 8b0e448f 2010-11-23T16:51:18 Removed unreachable code (MSVC warning)
Vicent Marti 7a3924fc 2010-11-14T22:07:48 Fix overcomplicated return statement Note to self: don't be stupid Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti 1795f879 2010-11-05T03:20:17 Improve error handling All initialization functions now return error codes instead of pointers. Error codes are now properly propagated on most functions. Several new and more specific error codes have been added in common.h Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti adc0327a 2010-10-29T00:44:15 Add git_odb_read_header method to the ODB As requested, git_odb_read_header looks up an object on the ODB, but loads only the header information (type & size) without loading any of the actual file contents in memory. It is significantly faster than doing a git_odb_read if you only need an object's information and not its contents. Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti 0c3596f1 2010-09-20T01:57:53 Add setter methods & write support for git_commit All the required git_commit_set_XXX methods have been implemented; all the attributes of a commit object can now be modified in-memory. The new method git_object_write() automatically writes back the in-memory changes of any object to the repository. So far it only supports git_commit objects. Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti f49a2e49 2010-09-19T03:21:06 Give object structures more descriptive names The 'git_obj' structure is now called 'git_rawobj', since it represents a raw object read from the ODB. The 'git_repository_object' structure is now called 'git_object', since it's the base object class for all objects. Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti 7e4f56a5 2010-08-06T18:37:59 Add packfile reading Packed objects inside packfiles are now properly unpacked when calling the git_odb__read_packed() method; delta'ed objects are also properly generated when needed. A new unit test 0204-readpack tries to read a couple hundred packed objects from a standard packed repository. Signed-off-by: Vicent Marti <tanoku@gmail.com>
Ramsay Jones 89217d8f 2010-04-28T20:20:00 Add functions to open a '*.pack' file and perform some basic validation Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones 3cc60635 2010-03-05T19:06:28 Add some more (macro) file operation wrappers Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones 54b9460f 2010-04-07T20:13:56 Fix the memory leak caused by failing to free the 'offset index' Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones 2cdc4544 2010-02-26T22:53:10 Add a pack index 'virtual function' to fetch an index entry Given an index entry number, the idx_get() function returns an (version agnostic) index_entry structure containing all of the information required to unpack the corresponding object from the '.pack' file. Since the v1 and v2 file formats differ in the layout of the object records, we provide two implementations of the get function and initialise the function pointer appropriately. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones 608d33fa 2010-02-26T19:59:06 Add a pack index 'virtual function' to search by file offset In addition to searching the index by oid, we need to search by '.pack' file offset, particularly when processing OBJ_OFS_DELTA objects. Since the v1 and v2 file formats differ in the layout of the object records, we provide two implementations of the search function and initialise the (virtual) function pointer appropriately. Note that, as part of the creation of the 'offset index', we also add a check that the offset data in the index is within the bounds of the '.pack' file. Having sorted the file offsets, while creating the index, we only need to check the smallest and largest values. The offset index consists of the im_off_idx array, which contains the index entry numbers sorted into file offset order, and the im_off_next mapping array. The im_off_next array maps an index entry number to the 'next' index entry in file offset order. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones abf5681c 2010-02-26T19:29:54 Change the interface of the pack index search function In particular, on a successful search, we now return the index entry number of the object rather than the '.pack' file offset. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones 238e54bc 2010-02-21T15:34:53 Add an 64-bit offset table index bounds check for v2 pack index Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones 255a0dab 2010-02-19T22:36:31 Add a minimum size check when opening an v2 pack index file Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones 1e5dd572 2010-02-12T16:50:33 Fix some coding style issues Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones e8a95256 2010-01-04T18:57:13 msvc: Fix some -W4 warnings Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones 1a7bae4d 2010-01-11T22:51:42 Fix some "unused parameter" warnings with -Wextra Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones 0ef9d2aa 2010-01-03T22:56:54 Fix some "signed v unsigned comparison" warnings with -Wextra Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones a4f863af 2010-01-03T19:37:04 Fix an always false comparison "unsigned < 0" warning Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones 8a086f87 2009-06-14T22:12:20 win32: Add support for the MS Visual C/C++ compiler Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones 2bf93fa1 2009-06-14T22:10:23 Correct the spelling of the FLEX_ARRAY macro When setting the default value, the macro name was specified as GIT_FLEX_ARRAY, which is inconsistent with it's earlier usage in the file. This caused a compilation error, using the MS Visual C/C++ compiler, when compiling the git_packlist struct definition in src/odb.c. In addition to changing the spelling of the FLEX_ARRAY macro to GIT_FLEX_ARRAY, including it's use in src/odb.c, we also rename the TYPEOF macro to GIT_TYPEOF. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones e17a3f56 2009-06-04T16:47:59 Implement git_odb_write() Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Andreas Ericsson <ae@op5.se>
Ramsay Jones ac04bdf6 2009-06-04T16:44:54 Fix a usage error in a call to the object_file_name() function In 82324ac, the new static function exists_loose() called object_file_name() and, in order to detect an error return, tested for a negative value. This usage is incorrect, as the error return is indicated by a positive return value. (A successful call is indicated by a zero return value) The only error return from object_file_name() relates to insufficient buffer space and the return value gives the required minimum buffer size (which will always be >0). If the caller requires a dynamically allocated buffer, this allows something like the following call sequence: size_t len = object_file_name(NULL, 0, db->object_dir, id); char *buf = git__malloc(len); if (!buf) error(...); object_file_name(buf, len, db->object_dir,id); ... No current callers take advantage of this capability. Fix up the call site and change the return type of the function, from int to size_t, which more accurately reflects the implementation. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Andreas Ericsson <ae@op5.se>
Ramsay Jones 79ca2edc 2009-03-20T19:51:48 win32: Add routines to abstract memory-mapped file functions In particular, the git__mmap() and git__munmap() routines provide the interface to platform specific memory-mapped file facilities. We provide implementations for unix and win32, which can be found in their own sub-directories. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones 840fb8b7 2009-02-18T18:53:48 win32: fixup some headers to improve win32 compilation Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones a9984a4e 2009-02-18T18:52:13 Fix some (digital-mars) compiler warnings In particular, conditional expressions which contain an assignment statement, where the expression type is not explicitly made to be boolean, elicits the following message: warning 2: possible unintended assignment Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones 82324ac1 2009-02-10T21:49:48 Add the git_odb_exists() object query function This function determines if the given object can be found in the object database. At present, only the local object database is searched. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones e9f5e877 2009-02-03T18:25:13 Rearrange some code to improve clarity In particular, the test for z-stream input completion (zs.avail_in != 0) logically belongs with the test for the Z_STREAM_END stream status. This is also consistent with the identical check in finish_inflate(). Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones 236e7579 2009-02-03T18:23:50 Check for error returns from inflateInit() At present, it is sufficient to ensure that an error return from inflateInit() is not ignored. Most error returns, like Z_VERSION_ERROR and Z_STREAM_ERROR, indicate programming or build errors. These errors could, perhaps, be handled with simple asserts. However, for a Z_MEM_ERROR, we may want to perform some further error handling in the future. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones c94eb4aa 2009-02-03T18:22:09 Fix a potential memory leak In particular, neglecting to call inflateEnd() along various codepaths in the inflate_tail() routine, would result in the failure to release zlib internal state. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce bed3229b 2009-01-03T03:34:09 Precompute the fanout decoding and the oid offset in a pack-*.idx The fanout table is fairly commonly accessed, we need to read it twice for each object we lookup in any given pack file. Most of the processors running Git are running in little-endian mode, as they are variants of the x86 platform, so reading the fanout is a costly operation as we need to convert from network byte order to local byte order. By decoding the fanout table into a malloc obtained buffer we can save these 2 decode operations per lookup and make search go more quickly. This also cleans up the initialization of the search functions by cutting out a few instructions, saving a small amount of time. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce a7c60cfc 2009-01-03T02:41:26 Add basic support to read pack-*.idx v1 and v2 files The index data is mapped into memory and then scanned using a binary search algorithm to locate the matching entry for the supplied git_oid. The standard fanout hash trick is applied to reduce the search space by 8 iterations. Since the v1 and v2 file formats differ in their search function, due to the different layouts used for the object records, we use two different search implementations and a virtual function pointer to jump to the correct version of code for the current pack index. The single function jump per-pack should be faster then computing a branch point inside the inner loop of a common binary search. To improve concurrency during read operations the pack lock is only held while verifying the index is actually open, or while opening the index for the first time. This permits multiple concurrent readers to scan through the same index. If an invalid index file is opened we close it and mark the git_pack's invalid bit to true. The git_pack structure is kept around in its parent git_packlist, but the invalid bit will cause all future readers to skip over the pack entirely. Pruning the invalid entries is relatively unimportant because they shouldn't be very common, a $GIT_DIRECTORY/objects/pack directory tends to only have valid pack files. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce 098ac57a 2009-01-03T00:02:25 Refactor pack memory management and locking to be safer Using an atomic reference counter is difficult to make cross-platform, as the reference count implementations are generally processor specific. Its also hard to do a proper multi-read/single-write implementation. We now use a simple mutex around the reference count for the list of packs. Readers grab the mutex and either build the list, or increment the existing one's reference count. When the reader is done with the list, the reference count is decremented. In this way parallel readers are able to operate on the list without worrying about it being deallocated out from under them. Individual pack structures are held by reference counts, but we only care about the list the pack structure is held in. There is no need to increment/decrement the pack reference counts as we scan through them during a read operation, the caller holds the git_packlist and that is sufficient to hold the packs it references. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones 3a33c7b3 2009-01-02T20:51:47 Fix snprintf compiler warning on cygwin As far as gcc is concerned, the "z size specifier" is available as an extension to the language, which is available with or without any -std= switch. (I think you have to go back to 2.95 for a version of gcc which doesn't work.) Many other compilers have this as an extension as well (ie without the equivalent of -std=c99). Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce b438016e 2008-12-31T16:20:05 Find pack files in $GIT_DIR/objects/pack directory on git_odb_open Currently we only catalog the available pack files into a table, storing their path names relative to the pack directory. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce 5614dc18 2008-12-31T13:27:51 Add basic locking to the git_odb structure We grab the lock while accessing the alternates list, ensuring that we only initialize it once for the given git_odb. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce d44cfd46 2008-12-31T13:16:31 Cleanup our header inclusion order to ensure pthread.h is early If we are using threads we need to make sure pthread.h comes in before just about anything else. Some platforms enable macros that alter what other headers define. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce 64a47c01 2008-12-30T23:21:36 Wrap malloc and friends and report out of memory as GIT_ENOMEM We now forbid direct use of malloc, strdup or calloc within the library and instead use wrapper functions git__malloc, etc. to invoke the underlying library malloc and set git_errno to a no memory error code if the allocation fails. In the future once we have pack objects in memory we are likely to enhance these routines with garbage collection logic to purge cached pack data when allocations fail. Because the size of the function will grow somewhat large, we don't want to mark them for inline as gcc tends to aggressively inline, creating larger than expected executables. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce ffb55c53 2008-12-30T22:29:04 Rename the path of the objects directory to be more specific We're likely to add additional path data, like the path of the refs or the path to the config file into the git_odb structure, as it may grow into the repository wrapper. Changing the name of the objects directory reference makes it more clear should we later add something else. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce 4c67e2e9 2008-12-30T22:25:30 Change git_odb__read_packed to return ENOTFOUND until implemented We didn't search for the object, so we cannot possibly promise it to the caller of git_odb_read(). Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
kmx.io     mail     discord kmxgit v0.4.0