thodg/got/lib/object.c

---

lib/object.c

Branch - branch -mainorigin/HEADorigin/gotwebdorigin/main

---

Log

Author	Commit	Date	CI	Message
	f9d37699	2022-06-28T20:33:41		move got_opentempfd() out of lib/diff.c again ok tracey
	7a13e6e5	2022-06-28T18:19:03		pass the correct file to fdopen(); ok tracey
	ce7556f2	2022-06-28T18:04:05		fix missed dup in open_blob per stsp@
	eb81bc23	2022-06-28T13:27:12		move got_opentempfd out of open_blob. ok stsp@
	3d589bee	2022-06-25T14:54:41		allow start_pack_privsep_child() to be called from outside lib/object.c
	336075a4	2022-06-25T09:22:02		build with -Wmissing-prototypes ok stsp@
	db9b9b1c	2022-06-14T20:26:15		let got-read-pack be explicit about whether it could enumerate all objects This allows the main process to avoid looping over all object IDs again in case the pack file used for enumeration is complete. ok op@
	0ab4c957	2022-06-13T17:13:59		Bring back object enumeration inside got-read-pack as a fast path. The problem that was found in the earlier version has been fixed. ok op@
	e44d9391	2022-06-07T19:20:01		revert object enumeration in got-read-pack for now; needs more work This implementation marked commits and trees as enumerated before all trees which they depend on were enumerated. This behaviour leads to incomplete pack files when a tree is only partially packed and got-read-pack hits a missing tree entry as a result. The algorithm must be reworked such that packed leave nodes are marked enumerated first, then bubble-up. Found by op@
	cee6a7ea	2022-06-07T15:56:46		implement object enumeration support in got-read-pack ok op@
	57160834	2022-05-31T11:00:09		open temporary files needed for delta application in got_repo_open() This prepares for callers of got_repo_open() that cannot afford to open files in /tmp, such as gotwebd. In a follow-up change, we could ask such callers to pass in the required amount of open temporary files. One consequence is that got_repo_open() now requires the "cpath" pledge promise. Add the "cpath" promise to affected callers and remove it once the repository has been opened. ok tracey
	fae7e038	2022-05-07T11:50:56		run the search for deltas to reuse in got-read-pack This significantly speeds up the deltification step of packing by avoiding imsg traffic. gotadmin no longer requests individual raw deltas from got-read-pack to check whether it can reuse them. Instead, got-read-pack obtains a list of objects we want to pack, and hands back the list of all deltas in its pack file which can be reused. Messages are now batched such that imsg buffers are filled as much as possible. Another advantage is that deltas we are not going to reuse will no longer be written to the delta cache file, saving disk space. Before this patch, any raw delta candidate was written to the delta cache file by got-read-pack, and the decision whether to reuse the delta happened afterwards in the gotadmin process. Code for reading individual raw deltas is now unused and could be removed at some point. ok op@
	2d9e6abf	2022-05-04T13:43:24		store deltas in compressed form while packing, both in memory and cache file This reduces memory and disk space consumption during packing. with tweaks + memleak on error fix from op@ ok op@
	d7b5a0e8	2022-04-20T14:00:12		inline struct got_object_id in struct got_object_qid Saves us from doing a malloc/free call for every item on the list. ok op@
	a44927cc	2022-04-07T12:57:31		stop relying on commit cache for good performance of got_object_id_by_path() Instead of internally opening and closing the same commit object over and over again, require callers to pass an open commit object in. Avoids an inherent dependency on the commit object cache for reasonable performance. ok op@
	67fd6849	2022-02-13T00:10:25		reuse existing deltas when creating pack files tested by thomas, naddy, and myself
	64a8571e	2022-01-07T23:32:27		map raw object files into memory while packing if possible
	db696021	2022-01-04T16:11:52		avoid the creation of new temporary files whenever a packed object is read This speeds up the creation of pack files by about 30%.
	8bd0cdad	2021-12-31T09:11:21		add O_CLOEXEC (close-on-exec) flag to open(2) calls suggested by millert ok thomas_adam
	94dac27c	2021-10-15T09:24:56		raw object blocksize and read buffer were unused; remove them
	284e7663	2021-10-15T09:21:46		remove unused internal raw object API functions
	d3c116bf	2021-10-15T09:10:14		cache raw objects in order to speed up gotadmin pack
	cc7a354a	2021-10-15T07:15:00		reuse temporary files which were not used by got_object_raw_open()
	e65c7410	2021-10-14T20:50:01		add missing error checks in got_object_raw_open()
	f8b19efd	2021-10-13T11:09:15		use RB_TREE instead of STAILQ to manage packindex bloom filters; much faster
	67b631c9	2021-10-10T15:22:14		implement 'got diff -c' for diffing commits with optional filtering by path Need for filtering by path sugggested by kn@ ok naddy@
	d5c81d44	2021-07-08T11:03:29		verify object ID checksums while loose objects are being accessed
	aea75d87	2021-07-06T07:57:00		make got_packidx_get_packfile_path() usable for callers who only have a path
	b3d68e7f	2021-07-03T19:49:37		implement 'gotadmin cleanup'
	dbdddfee	2021-06-23T20:48:35		switch from SIMPLEQ to equivalent STAILQ macros The singly-linked tail queue macros were added to OpenBSD 6.9 and are more widely available on other systems. ok stsp
	a8591711	2021-06-18T13:52:26		raw object size should not include the length of the object's header This way, the size of a raw object is the same regardless of whether the object was found in a loose object file or in a pack file.
	9ca9aafb	2021-06-18T13:36:51		introduce got_object_id_queue_copy() This will be required by a future 'gotadmin pack' command.
	762d73f4	2021-04-10T13:13:56		introduce got_object_open_loose_fd() for library-internal use
	509c9973	2021-04-10T13:11:17		add got_packidx_get_packfile_path() for library-internal use
	7495ec13	2021-04-04T14:24:39		fix open file descriptor leak in error path of read_object_header_privsep()
	59d1e4a0	2021-03-10T22:49:22		implement raw object data access; this will be required for packing
	08578a35	2021-01-22T11:05:05		make close(2) failure checks consistent; check 'close() == -1' everywhere ok millert, naddy
	56b63ca4	2021-01-22T11:04:47		make fclose(3) failure checks consistent; check 'fclose() == EOF' everywhere ok millert, naddy
	e8bfb8f3	2020-12-18T15:52:25		fix path existence check in got_object_tree_path_changed() A symptom of this was a segfault in 'got blame distrib/miniroot/install.sub' with the OpenBSD src.git from github, reported by jrick. The problem was that the commit graph traversed one commit too far. This could be more easily reproduced with 'got log': got log -c 05f568 -P distrib/miniroot/install.sub Which listed two commits instead of just the first one: 05f568ecc6aadefa1aff9064a29e798874a71409 <-- install.sub first created here 7c0d87f00e480cdf004324dad6f3e6f4418f8f42 "distrib/miniroot" exists in 7c0d87f00e480cdf004324... but not in the parent of this commit. Ensure that we traverse the full path in tree1 even if an intermediate tree2 cannot be opened. Instead of reporting success and a file change if we can't traverse further through tree2, we now hit this failing attempt to open the file 'install.sub' in tree object 180aa33df8d1 (tree1): te1 = find_entry_by_name(tree1, seg, seglen); if (te1 == NULL) { err = got_error(GOT_ERR_NO_OBJ); goto done; } fix tested and ok jrick
	be659d10	2020-11-18T17:18:23		fix type of filesize output parameter of got_object_blob_dump_to_file()
	c33ebc60	2020-11-18T17:05:28		reallocate line_offsets array less often in got_object_blob_dump_to_file()
	f1cbc3bc	2020-11-18T13:48:23		decouple line_offsets/nlines output params of got_object_blob_dump_to_file()
	dbdd6209	2020-10-19T23:14:59		handle non-const basename in resolve_symlink() ok millert
	23c57b28	2020-09-11T19:44:18		Stop including <sys/syslimits.h> directly. POSIX says the limits defined there are available from <limits.h>, which almost all affected source files already included anyway. ok millert stsp
	81a12da5	2020-09-09T19:35:32		do not rely on <zlib.h> to pull in <unistd.h> ok stsp
	b66cd6f3	2020-07-31T10:33:34		display more context info in "no such entry found in tree" error messages
	559d127c	2020-07-23T15:07:52		errant tab found by tracey
	aa092692	2020-07-23T14:22:40		initialize output parameter of got_object_blob_read_to_str(); found by ofree
	af57b12a	2020-07-23T14:21:30		add symlink support to 'got cherrypick'; test non-conflict cases only for now
	40dde666	2020-07-23T14:21:29		add support for symlinks in the repository to 'got diff'
	32596e16	2020-07-23T14:21:28		fix got_tree_entry_get_symlink_target() in case of multiple loop iterations
	f8f7c882	2020-07-23T14:21:28		fix buffer overflow in got_tree_entry_get_symlink_target()
	e40622f4	2020-07-23T14:21:28		add got_object_tree_entry_is_symlink() and got_object_resolve_symlinks()
	659dc16e	2020-07-23T14:21:28		skip the blob header in got_tree_entry_get_symlink_target()
	8ba819a3	2020-07-23T14:21:27		let 'got checkout' create symlinks in a work tree
	0d6c6ee3	2020-05-20T17:37:29		make 'got/tog tree' show symlink targets like 'ls -lF' does: link@ -> target
	ac5f2b26	2020-05-05T15:26:11		normalize file mode bits to avoid false positive tree entry differences
	82bde4e0	2020-03-18T16:10:31		remove unused header include
	93658fb9	2020-03-18T16:10:30		Add initial support for network protocol. Ported from git9 by Ori Bernstein.
	61a7d79f	2020-02-29T08:05:11		remove GOT_ERR_NOT_ABSPATH and fix existing users to deal with absolute paths
	63f810e6	2020-02-29T07:54:55		when "bad path" errors occur, always show the path in question
	992eb9d8	2020-02-07T17:02:17		spelling
	845785d4	2020-02-02T17:08:02		typo
	e1a68182	2020-01-07T17:49:17		fix pack index cache; don't open/close pack index files needlessly
	ca6e02ac	2020-01-07T11:14:52		add support for first-parent history traversal to got-read-pack
	56e0773d	2019-11-28T02:32:58		convert tree entries from SIMPLEQ to an array
	5f25cc85	2019-11-26T22:43:12		remove obj->path_packfile which is unused nowadays; saves a few free() calls
	6dd1ece6	2019-11-10T11:32:03		when an object id string is bad, include the bad string in the error message
	ddc7b220	2019-09-08T18:29:14		do not leak ibuf in error paths of read_*_privsep()
	aba9c984	2019-09-08T17:14:34		obtain repo format version and commit author name/email from .git/config
	63c5ca5d	2019-08-24T20:56:15		detect and ignore Git submodules
	01073a5d	2019-08-22T14:17:38		initial 'got cat' implementation
	b02560ec	2019-08-19T09:54:52		fix number of lines accounted for during blame
	f595d9bd	2019-08-14T22:51:13		fix line offset calculations in got_object_blob_dump_to_file()
	5d844a1e	2019-08-13T15:41:46		don't attempt to parse Git's "leightweight" tags as real tag objects
	8d2c5ea3	2019-08-13T10:49:23		don't leak memory for pack file path when opening objects
	78695fb7	2019-08-12T12:38:08		fix blame with single-commit history and with files without \n
	d24820bf	2019-08-11T21:19:14		add support for tag objects to 'got diff'
	41d2888b	2019-08-11T20:03:36		move an assignment closer to error check in got_object_get_path()
	e82b1d81	2019-07-27T19:12:22		switch back to searching packed objects first; it is indeed faster
	34f480ff	2019-07-27T18:36:14		open local objects before searching packs; faster on large repos
	e09a504c	2019-06-28T20:59:14		introduce support for abbreviated object IDs
	2246482e	2019-06-25T12:27:42		make search actually work in 'tog blame'
	6c4c42e0	2019-06-24T22:28:11		implement search for 'tog blame'
	5e54fb30	2019-05-31T14:57:57		test and fix 'got revert' with a path in a sub-directory Relax input path requirements of got_object_id_by_path() to make things easier to callers. Allows the revert code to pass a path from the file index in lieu of a repository path.
	da506691	2019-05-22T13:00:28		set_max_datasize() should be a static function
	b48e2ddb	2019-05-22T12:31:30		set datasize limit of got-read-pack helpers to maximum
	638f9024	2019-05-13T12:40:57		rename got_error_prefix_errno() to got_error_from_errno()
	230a42bd	2019-05-11T13:04:38		got_error_from_errno -> got_error_prefix_errno also add got_error_prefix_errno2 and got_error_prefix_errno3 which should hopefully all be merged into a single function with variadic args (but can't alloc mem)
	48b8b0eb	2019-05-11T10:40:57		error: introduce got_error_prefix_errno for better errno messages
	324d37e7	2019-05-11T09:14:30		make got path APIs available to library consumers
	8c4eabf2	2019-05-10T18:40:10		redo: fix tree_entry_dup error path
	83a7ae6d	2019-05-10T18:38:59		revert previous
	3ffe981f	2019-05-10T18:38:23		fix tree_entry_dup error path
	ed175427	2019-05-07T19:53:44		more progress on generating new tree entries
	90bdb554	2019-04-11T14:30:40		expose a static helper function as got_object_get_path()
	a5b57ccf	2019-04-11T13:37:50		omit unused mode argument to open(2)
	b7011c59	2019-04-06T06:05:44		fix double-free in error path of open_blob()
	a129376b	2019-03-29T00:04:52		add initial implementation of 'got revert'
	7baf5860	2019-03-19T12:37:32		avoid code duplication in error case of open_blob()