lib/object.c


Log

Author Commit Date CI Message
Stefan Sperling fae7e038 2022-05-07T11:50:56 run the search for deltas to reuse in got-read-pack This significantly speeds up the deltification step of packing by avoiding imsg traffic. gotadmin no longer requests individual raw deltas from got-read-pack to check whether it can reuse them. Instead, got-read-pack obtains a list of objects we want to pack, and hands back the list of all deltas in its pack file which can be reused. Messages are now batched such that imsg buffers are filled as much as possible. Another advantage is that deltas we are not going to reuse will no longer be written to the delta cache file, saving disk space. Before this patch, any raw delta candidate was written to the delta cache file by got-read-pack, and the decision whether to reuse the delta happened afterwards in the gotadmin process. Code for reading individual raw deltas is now unused and could be removed at some point. ok op@
Stefan Sperling 2d9e6abf 2022-05-04T13:43:24 store deltas in compressed form while packing, both in memory and cache file This reduces memory and disk space consumption during packing. with tweaks + memleak on error fix from op@ ok op@
Stefan Sperling d7b5a0e8 2022-04-20T14:00:12 inline struct got_object_id in struct got_object_qid Saves us from doing a malloc/free call for every item on the list. ok op@
Stefan Sperling a44927cc 2022-04-07T12:57:31 stop relying on commit cache for good performance of got_object_id_by_path() Instead of internally opening and closing the same commit object over and over again, require callers to pass an open commit object in. Avoids an inherent dependency on the commit object cache for reasonable performance. ok op@
Stefan Sperling 67fd6849 2022-02-13T00:10:25 reuse existing deltas when creating pack files tested by thomas, naddy, and myself
Stefan Sperling 64a8571e 2022-01-07T23:32:27 map raw object files into memory while packing if possible
Stefan Sperling db696021 2022-01-04T16:11:52 avoid the creation of new temporary files whenever a packed object is read This speeds up the creation of pack files by about 30%.
Stefan Sperling 8bd0cdad 2021-12-31T09:11:21 add O_CLOEXEC (close-on-exec) flag to open(2) calls suggested by millert ok thomas_adam
Stefan Sperling 94dac27c 2021-10-15T09:24:56 raw object blocksize and read buffer were unused; remove them
Stefan Sperling 284e7663 2021-10-15T09:21:46 remove unused internal raw object API functions
Stefan Sperling d3c116bf 2021-10-15T09:10:14 cache raw objects in order to speed up gotadmin pack
Stefan Sperling cc7a354a 2021-10-15T07:15:00 reuse temporary files which were not used by got_object_raw_open()
Stefan Sperling e65c7410 2021-10-14T20:50:01 add missing error checks in got_object_raw_open()
Stefan Sperling f8b19efd 2021-10-13T11:09:15 use RB_TREE instead of STAILQ to manage packindex bloom filters; much faster
Stefan Sperling 67b631c9 2021-10-10T15:22:14 implement 'got diff -c' for diffing commits with optional filtering by path Need for filtering by path sugggested by kn@ ok naddy@
Stefan Sperling d5c81d44 2021-07-08T11:03:29 verify object ID checksums while loose objects are being accessed
Stefan Sperling aea75d87 2021-07-06T07:57:00 make got_packidx_get_packfile_path() usable for callers who only have a path
Stefan Sperling b3d68e7f 2021-07-03T19:49:37 implement 'gotadmin cleanup'
Christian Weisgerber dbdddfee 2021-06-23T20:48:35 switch from SIMPLEQ to equivalent STAILQ macros The singly-linked tail queue macros were added to OpenBSD 6.9 and are more widely available on other systems. ok stsp
Stefan Sperling a8591711 2021-06-18T13:52:26 raw object size should not include the length of the object's header This way, the size of a raw object is the same regardless of whether the object was found in a loose object file or in a pack file.
Stefan Sperling 9ca9aafb 2021-06-18T13:36:51 introduce got_object_id_queue_copy() This will be required by a future 'gotadmin pack' command.
Stefan Sperling 762d73f4 2021-04-10T13:13:56 introduce got_object_open_loose_fd() for library-internal use
Stefan Sperling 509c9973 2021-04-10T13:11:17 add got_packidx_get_packfile_path() for library-internal use
Stefan Sperling 7495ec13 2021-04-04T14:24:39 fix open file descriptor leak in error path of read_object_header_privsep()
Stefan Sperling 59d1e4a0 2021-03-10T22:49:22 implement raw object data access; this will be required for packing
Stefan Sperling 08578a35 2021-01-22T11:05:05 make close(2) failure checks consistent; check 'close() == -1' everywhere ok millert, naddy
Stefan Sperling 56b63ca4 2021-01-22T11:04:47 make fclose(3) failure checks consistent; check 'fclose() == EOF' everywhere ok millert, naddy
Stefan Sperling e8bfb8f3 2020-12-18T15:52:25 fix path existence check in got_object_tree_path_changed() A symptom of this was a segfault in 'got blame distrib/miniroot/install.sub' with the OpenBSD src.git from github, reported by jrick. The problem was that the commit graph traversed one commit too far. This could be more easily reproduced with 'got log': got log -c 05f568 -P distrib/miniroot/install.sub Which listed two commits instead of just the first one: 05f568ecc6aadefa1aff9064a29e798874a71409 <-- install.sub first created here 7c0d87f00e480cdf004324dad6f3e6f4418f8f42 "distrib/miniroot" exists in 7c0d87f00e480cdf004324... but not in the parent of this commit. Ensure that we traverse the full path in tree1 even if an intermediate tree2 cannot be opened. Instead of reporting success and a file change if we can't traverse further through tree2, we now hit this failing attempt to open the file 'install.sub' in tree object 180aa33df8d1 (tree1): te1 = find_entry_by_name(tree1, seg, seglen); if (te1 == NULL) { err = got_error(GOT_ERR_NO_OBJ); goto done; } fix tested and ok jrick
Stefan Sperling be659d10 2020-11-18T17:18:23 fix type of filesize output parameter of got_object_blob_dump_to_file()
Stefan Sperling c33ebc60 2020-11-18T17:05:28 reallocate line_offsets array less often in got_object_blob_dump_to_file()
Stefan Sperling f1cbc3bc 2020-11-18T13:48:23 decouple line_offsets/nlines output params of got_object_blob_dump_to_file()
Stefan Sperling dbdd6209 2020-10-19T23:14:59 handle non-const basename in resolve_symlink() ok millert
Christian Weisgerber 23c57b28 2020-09-11T19:44:18 Stop including <sys/syslimits.h> directly. POSIX says the limits defined there are available from <limits.h>, which almost all affected source files already included anyway. ok millert stsp
Christian Weisgerber 81a12da5 2020-09-09T19:35:32 do not rely on <zlib.h> to pull in <unistd.h> ok stsp
Stefan Sperling b66cd6f3 2020-07-31T10:33:34 display more context info in "no such entry found in tree" error messages
Stefan Sperling 559d127c 2020-07-23T15:07:52 errant tab found by tracey
Stefan Sperling aa092692 2020-07-23T14:22:40 initialize output parameter of got_object_blob_read_to_str(); found by ofree
Stefan Sperling af57b12a 2020-07-23T14:21:30 add symlink support to 'got cherrypick'; test non-conflict cases only for now
Stefan Sperling 40dde666 2020-07-23T14:21:29 add support for symlinks in the repository to 'got diff'
Stefan Sperling 32596e16 2020-07-23T14:21:28 fix got_tree_entry_get_symlink_target() in case of multiple loop iterations
Stefan Sperling f8f7c882 2020-07-23T14:21:28 fix buffer overflow in got_tree_entry_get_symlink_target()
Stefan Sperling e40622f4 2020-07-23T14:21:28 add got_object_tree_entry_is_symlink() and got_object_resolve_symlinks()
Stefan Sperling 659dc16e 2020-07-23T14:21:28 skip the blob header in got_tree_entry_get_symlink_target()
Stefan Sperling 8ba819a3 2020-07-23T14:21:27 let 'got checkout' create symlinks in a work tree
Stefan Sperling 0d6c6ee3 2020-05-20T17:37:29 make 'got/tog tree' show symlink targets like 'ls -lF' does: link@ -> target
Stefan Sperling ac5f2b26 2020-05-05T15:26:11 normalize file mode bits to avoid false positive tree entry differences
Stefan Sperling 82bde4e0 2020-03-18T16:10:31 remove unused header include
Stefan Sperling 93658fb9 2020-03-18T16:10:30 Add initial support for network protocol. Ported from git9 by Ori Bernstein.
Stefan Sperling 61a7d79f 2020-02-29T08:05:11 remove GOT_ERR_NOT_ABSPATH and fix existing users to deal with absolute paths
Stefan Sperling 63f810e6 2020-02-29T07:54:55 when "bad path" errors occur, always show the path in question
Tracey Emery 992eb9d8 2020-02-07T17:02:17 spelling
Tracey Emery 845785d4 2020-02-02T17:08:02 typo
Stefan Sperling e1a68182 2020-01-07T17:49:17 fix pack index cache; don't open/close pack index files needlessly
Stefan Sperling ca6e02ac 2020-01-07T11:14:52 add support for first-parent history traversal to got-read-pack
Stefan Sperling 56e0773d 2019-11-28T02:32:58 convert tree entries from SIMPLEQ to an array
Stefan Sperling 5f25cc85 2019-11-26T22:43:12 remove obj->path_packfile which is unused nowadays; saves a few free() calls
Stefan Sperling 6dd1ece6 2019-11-10T11:32:03 when an object id string is bad, include the bad string in the error message
Stefan Sperling ddc7b220 2019-09-08T18:29:14 do not leak ibuf in error paths of read_*_privsep()
Stefan Sperling aba9c984 2019-09-08T17:14:34 obtain repo format version and commit author name/email from .git/config
Stefan Sperling 63c5ca5d 2019-08-24T20:56:15 detect and ignore Git submodules
Stefan Sperling 01073a5d 2019-08-22T14:17:38 initial 'got cat' implementation
Stefan Sperling b02560ec 2019-08-19T09:54:52 fix number of lines accounted for during blame
Stefan Sperling f595d9bd 2019-08-14T22:51:13 fix line offset calculations in got_object_blob_dump_to_file()
Stefan Sperling 5d844a1e 2019-08-13T15:41:46 don't attempt to parse Git's "leightweight" tags as real tag objects
Stefan Sperling 8d2c5ea3 2019-08-13T10:49:23 don't leak memory for pack file path when opening objects
Stefan Sperling 78695fb7 2019-08-12T12:38:08 fix blame with single-commit history and with files without \n
Stefan Sperling d24820bf 2019-08-11T21:19:14 add support for tag objects to 'got diff'
Stefan Sperling 41d2888b 2019-08-11T20:03:36 move an assignment closer to error check in got_object_get_path()
Stefan Sperling e82b1d81 2019-07-27T19:12:22 switch back to searching packed objects first; it is indeed faster
Stefan Sperling 34f480ff 2019-07-27T18:36:14 open local objects before searching packs; faster on large repos
Stefan Sperling e09a504c 2019-06-28T20:59:14 introduce support for abbreviated object IDs
Stefan Sperling 2246482e 2019-06-25T12:27:42 make search actually work in 'tog blame'
Stefan Sperling 6c4c42e0 2019-06-24T22:28:11 implement search for 'tog blame'
Stefan Sperling 5e54fb30 2019-05-31T14:57:57 test and fix 'got revert' with a path in a sub-directory Relax input path requirements of got_object_id_by_path() to make things easier to callers. Allows the revert code to pass a path from the file index in lieu of a repository path.
Stefan Sperling da506691 2019-05-22T13:00:28 set_max_datasize() should be a static function
Stefan Sperling b48e2ddb 2019-05-22T12:31:30 set datasize limit of got-read-pack helpers to maximum
Stefan Sperling 638f9024 2019-05-13T12:40:57 rename got_error_prefix_errno() to got_error_from_errno()
joshua stein 230a42bd 2019-05-11T13:04:38 got_error_from_errno -> got_error_prefix_errno also add got_error_prefix_errno2 and got_error_prefix_errno3 which should hopefully all be merged into a single function with variadic args (but can't alloc mem)
joshua stein 48b8b0eb 2019-05-11T10:40:57 error: introduce got_error_prefix_errno for better errno messages
Stefan Sperling 324d37e7 2019-05-11T09:14:30 make got path APIs available to library consumers
Stefan Sperling 8c4eabf2 2019-05-10T18:40:10 redo: fix tree_entry_dup error path
Stefan Sperling 83a7ae6d 2019-05-10T18:38:59 revert previous
Stefan Sperling 3ffe981f 2019-05-10T18:38:23 fix tree_entry_dup error path
Stefan Sperling ed175427 2019-05-07T19:53:44 more progress on generating new tree entries
Stefan Sperling 90bdb554 2019-04-11T14:30:40 expose a static helper function as got_object_get_path()
Stefan Sperling a5b57ccf 2019-04-11T13:37:50 omit unused mode argument to open(2)
Stefan Sperling b7011c59 2019-04-06T06:05:44 fix double-free in error path of open_blob()
Stefan Sperling a129376b 2019-03-29T00:04:52 add initial implementation of 'got revert'
Stefan Sperling 7baf5860 2019-03-19T12:37:32 avoid code duplication in error case of open_blob()
Stefan Sperling 41496140 2019-02-21T15:17:36 prevent double-close(2) of file descriptors passed in imsg
Stefan Sperling 3a6ce05a 2019-02-11T12:32:53 check for errors from close(2)
Stefan Sperling fb43ecf1 2019-02-11T11:59:09 check for errors from fclose()
Stefan Sperling cbe7f848 2019-02-11T11:35:55 check for fflush() errors
Stefan Sperling 0bd18d37 2019-02-01T22:16:47 add potentially useful helpers for tag objects and refs
Stefan Sperling 5d56da81 2019-01-13T17:15:53 happy new copyright year
Stefan Sperling ac544f8c 2019-01-13T15:58:39 pass smallish blobs directly via imsg buffer
Stefan Sperling de060dff 2018-12-24T17:39:23 handle zero-length files in open_blob() correctly
Stefan Sperling 454a6b59 2018-12-24T17:32:40 fix fwrite() call in got_object_blob_dump_to_file()
Stefan Sperling b6752625 2018-12-24T17:28:46 fix an unchecked fwrite() call in got_object_blob_dump_to_file()
Stefan Sperling aea5f015 2018-12-24T14:29:50 drop dead code from got_privsep_send_obj_req()