thodg/got/libexec/got-index-pack

---

libexec/got-index-pack

Branch - branch -mainorigin/HEADorigin/gotwebdorigin/main

---

Log

Author	Commit	Date	CI	Message
	dac5c75e	2022-06-04T13:58:48		convert delta cache to a hash table This approach uses more memory but is much faster. To offset the additional memory usage somewhat the cache now stores very small deltas only. However, overall memory usage goes up. Hopefully we will find a way to reduce this later. ok op@
	c0591751	2022-03-24T12:17:45		link libexec helpers against _p libraries when building with 'make PROFILE=1'
	d0f1e2f1	2022-02-23T12:13:03		apply time-based rate-limiting to got-index-pack progress output
	8347f8e7	2021-10-15T17:17:52		plug memory leak in an error path of read_packed_object()
	07965a0d	2021-10-15T17:08:54		fix wrong function names in error messages from got-index-pack.c
	d5c81d44	2021-07-08T11:03:29		verify object ID checksums while loose objects are being accessed
	dbdddfee	2021-06-23T20:48:35		switch from SIMPLEQ to equivalent STAILQ macros The singly-linked tail queue macros were added to OpenBSD 6.9 and are more widely available on other systems. ok stsp
	85f4e1e7	2020-12-17T16:17:51		fix the type of variable 'idx' in update_packidx()
	6059809a	2020-12-17T15:54:47		use size_t for loop indices to avoid signedness warnings; by emaste@freebsd Same change as 16aeacf7088d, for subdirectories other than lib/
	1367695b	2020-09-26T20:53:13		fix potential type mismatches between format specifiers and arguments Cast printf arguments of type time_t and off_t to long long to match the %lld format specifier on platforms where this might not be the case. In parse.y, switch the number variable to long long because all its interactions are with that type anyway. ok millert stsp
	23c57b28	2020-09-11T19:44:18		Stop including <sys/syslimits.h> directly. POSIX says the limits defined there are available from <limits.h>, which almost all affected source files already included anyway. ok millert stsp
	3168e5da	2020-09-10T00:10:55		zap trailing tabs
	78fb0967	2020-09-09T21:41:22		use the POSIX standard endian functions and explicitly include <endian.h> ok stsp
	81a12da5	2020-09-09T19:35:32		do not rely on <zlib.h> to pull in <unistd.h> ok stsp
	3abebdc2	2020-03-24T15:24:08		make got-index-pack check the expected pack checksum passed from main process
	6ad68bce	2020-03-24T15:15:51		make got-index-pack compute and verify the pack file's SHA1 checksum
	e70bf110	2020-03-22T17:20:18		move functions from privsep.c to the helpers which use them where possible
	d582f26c	2020-03-18T16:13:48		write large objects to disk when resolving deltas; raise in-mem delta threshold
	18d4da03	2020-03-18T16:13:46		tweak delta cache size in got-index-pack
	93bba072	2020-03-18T16:13:46		use qsort(3) instead of mergesort(3) to avoid a large spike in memory usage
	97684601	2020-03-18T16:13:46		write CRC info in one chunk rather than calling write(2) once per object
	021b0c6f	2020-03-18T16:13:46		don't forget to send a final progress notification from got-index-pack
	5672d305	2020-03-18T16:13:46		waste less time on sending progress messages over the privsep pipe
	160bbe2e	2020-03-18T16:13:46		jump right to the index of the first delta
	e6c1173d	2020-03-18T16:13:46		document why deltas will usually be resolved within a single loop iteration
	4788f1ce	2020-03-18T16:13:46		extract large objects to a temporary file in got-index-pack
	ec92f929	2020-03-18T16:13:45		another indentation fix
	4c06f612	2020-03-18T16:13:45		remove now pointless error check
	b3e1118b	2020-03-18T16:13:45		fix indentation
	73ab1060	2020-03-18T16:13:45		use a dedicated imsg for sending the pack index output file descriptor
	964d3c5b	2020-03-18T16:13:44		increase delta cache size for got-index-pack
	b102634c	2020-03-18T16:13:44		avoid magic numbers when writing pack index header in got-index-pack
	2e5a6fad	2020-03-18T16:13:44		add mmap support to got-index-pack
	f63dc90d	2020-03-18T16:13:44		remove unnecessary & in read_packed_object()
	861f3006	2020-03-18T16:13:44		pledge got-index-pack
	4d0fef1d	2020-03-18T16:13:44		plug a memory leak in resolve_deltified_object()
	37ebab0a	2020-03-18T16:13:44		free objects when indexing is done
	262c582a	2020-03-18T16:13:44		allocate one large indexed object array upfront, not an array of pointers
	7cd14ea0	2020-03-18T16:13:43		plug memory leak in error path of read_packed_object()
	836f2c92	2020-03-18T16:13:43		use a union for delta-specific data in struct got_indexed_object
	cbc66309	2020-03-18T16:13:43		wrap overlong lines
	0fd91daf	2020-03-18T16:13:43		populate an in-progress pack index only if ref deltas are present
	950de2cd	2020-03-18T16:13:43		avoid unnecessary memmove calls during the first indexing pass
	55fdd257	2020-03-18T16:13:43		do not buffer more data than necessary in got_inflate_to_mem_fd()
	1e87a3c3	2020-03-18T16:13:42		avoid re-reading the entirety of an object's data to calculate the CRC
	7bad1537	2020-03-18T16:13:42		make use of the large object counter inside the packidx structure
	b4152680	2020-03-18T16:13:42		remove duplicated code under #if 0
	64ab7238	2020-03-18T16:13:42		remove pointless lseek() call in index_pack()
	8bb2b40c	2020-03-18T16:13:42		fix CRC values in generated pack index
	c5621f1c	2020-03-18T16:13:42		enable object CRC calculation in got-index-pack
	668a20f6	2020-03-18T16:13:41		rewritten got-index-pack; sorry about the monster commit
	baa9fea0	2020-03-18T16:11:33		send indexing progress via an imsg
	6fb3a497	2020-03-18T16:11:29		add optional 'consumed' output parameter to got_inflate_to_mem()
	27937592	2020-03-18T16:11:28		add more missing \n in fprintfs from got-index-pack
	fd86ad6e	2020-03-18T16:11:28		pass SEEK_SET instead of zero for fseek() whence parameter
	57138e05	2020-03-18T16:11:28		add missing \n in error print
	a98e36f8	2020-03-18T16:11:28		fix inverted error check in readodelta()
	279090e1	2020-03-18T16:10:31		pass an object_id * to got_privsep_send_index_pack_req()
	a251e40e	2020-03-18T16:10:31		GOT_IMSG_ACK isn't needed; just fill the pipe, reading responses will block
	93658fb9	2020-03-18T16:10:30		Add initial support for network protocol. Ported from git9 by Ori Bernstein.