src/zstream.c


Log

Author Commit Date CI Message
Edward Thomson 6155e06b 2017-12-17T18:44:02 zstream: introduce a single chunk reader Introduce `get_output_chunk` that will inflate/deflate all the available input buffer into the output buffer. `get_output` will call `get_output_chunk` in a loop, while other consumers can use it to inflate only a piece of the data.
Edward Thomson 8642feba 2017-12-10T17:23:44 zstream: use UINT_MAX sized chunks Instead of paging to zlib in INT_MAX sized chunks, we can give it as many as UINT_MAX bytes at a time. zlib doesn't care how big a buffer we give it, this simply results in fewer calls into zlib.
Edward Thomson b7d36ef4 2017-12-12T12:24:11 zstream: treat `Z_BUF_ERROR` as non-fatal zlib will return `Z_BUF_ERROR` whenever there is more input to inflate or deflate than there is output to store the result. This is normal for us as we iterate through the input, particularly with very large input buffers.
Patrick Steinhardt 0c7f49dd 2017-06-30T13:39:01 Make sure to always include "common.h" first Next to including several files, our "common.h" header also declares various macros which are then used throughout the project. As such, we have to make sure to always include this file first in all implementation files. Otherwise, we might encounter problems or even silent behavioural differences due to macros or defines not being defined as they should be. So in fact, our header and implementation files should make sure to always include "common.h" first. This commit does so by establishing a common include pattern. Header files inside of "src" will now always include "common.h" as its first other file, separated by a newline from all the other includes to make it stand out as special. There are two cases for the implementation files. If they do have a matching header file, they will always include this one first, leading to "common.h" being transitively included as first file. If they do not have a matching header file, they instead include "common.h" as first file themselves. This fixes the outlined problems and will become our standard practice for header and source files inside of the "src/" from now on.
Edward Thomson 909d5494 2016-12-29T12:25:15 giterr_set: consistent error messages Error messages should be sentence fragments, and therefore: 1. Should not begin with a capital letter, 2. Should not conclude with punctuation, and 3. Should not end a sentence and begin a new one
Pranit Bauva 65b78ea3 2016-11-17T01:08:49 use `giterr_set_str()` wherever possible `giterr_set()` is used when it is required to format a string, and since we don't really require it for this case, it is better to stick to `giterr_set_str()`. This also suppresses a warning(-Wformat-security) raised by the compiler. Signed-off-by: Pranit Bauva <pranit.bauva@gmail.com>
Edward Thomson b8dc2fdb 2015-07-09T18:36:53 zstream: fail when asked to inflate garbage When we are provided some input buffer (with a length) to inflate, and it contains more data than simply the deflated data, fail. zlib will helpfully tell us when it is done reading (via Z_STREAM_END), so if there is data leftover in the input buffer, fail lest we continually try to inflate it.
Edward Thomson b88f1713 2015-06-17T08:07:34 zstream: offer inflating, `git_zstream_inflatebuf` Introduce `git_zstream_inflatebuf` for simple uses.
Edward Thomson 4aa664ae 2015-02-10T23:55:07 git_buf_grow_by: increase buf asize incrementally Introduce `git_buf_grow_by` to incrementally increase the size of a `git_buf`, performing an overflow calculation on the growth.
Edward Thomson 392702ee 2015-02-09T23:41:13 allocations: test for overflow of requested size Introduce some helper macros to test integer overflow from arithmetic and set error message appropriately.
Carlos Martín Nieto c7f86efb 2014-04-26T18:04:43 zstream: grow based on used memory rather than allocated When deflating data, we might need to grow the buffer. Currently we add a guess on top of the currently-allocated buffer size. When we re-use the buffer, it already has some memory allocated; adding to that means that we always grow the buffer regardless of how much we need to use. Instead, increase on top of the currently-used size. This still leaves us with the allocated size of the largest object we compress, but it's a minor pain compared to unbounded growth. This fixes #2285.
Russell Belfer 19459b1e 2014-01-30T10:23:35 Defer zstream NUL termination to end And don't terminate if there isn't space for it (since it's binary data, it's not worth a reallocation).
Russell Belfer 8606f33b 2014-01-30T09:59:15 Expand zstream tests and fix off-by-one error
Russell Belfer d9b04d78 2014-01-29T15:02:35 Reorganize zstream API and fix wrap problems There were some confusing issues mixing up the number of bytes written to the zstream output buffer with the number of bytes consumed from the zstream input. This reorganizes the zstream API and makes it easier to deflate an arbitrarily large input while still using a fixed size output.
XTao 1cb5a811 2014-01-26T17:07:39 Fix write_object.
Edward Thomson 52a8a130 2014-01-06T16:41:12 Packbuilder contains its own zstream
Edward Thomson c6f26b48 2013-12-13T18:26:46 Refactor zlib for easier deflate streaming