src/buffer.c


Log

Author Commit Date CI Message
Edward Thomson e5e2fac8 2019-01-21T00:57:39 buffer: explicitly cast Quiet down a warning from MSVC about how we're potentially losing data. This is safe since we've explicitly tested it.
Edward Thomson f673e232 2018-12-27T13:47:34 git_error: use new names in internal APIs and usage Move to the `git_error` name in the internal API for error-related functions.
Patrick Steinhardt ecf4f33a 2018-02-08T11:14:48 Convert usage of `git_buf_free` to new `git_buf_dispose`
Patrick Steinhardt 56ffdfc6 2018-02-08T11:14:30 buffer: deprecate `git_buf_free` in favor of `git_buf_dispose`
Edward Thomson 8070a357 2018-03-03T18:47:35 Introduce `git_buf_decode_percent` Introduce a function to take a percent-encoded string (URI encoded, described by RFC 1738) and decode it into a `git_buf`.
Patrick Steinhardt 4796c916 2017-06-07T09:56:31 buffer: return errors for `git_buf_init` and `git_buf_attach` Both the `git_buf_init` and `git_buf_attach` functions may call `git_buf_grow` in case they were given an allocation length as parameter. As such, it is possible for these functions to fail when we run out of memory. While it won't probably be used anytime soon, it does indeed make sense to also record this fact by returning an error code from both functions. As they belong to the internal API only, this change does not break our interface.
Patrick Steinhardt 9a8386a2 2017-06-07T09:50:54 buffer: consistently use `ENSURE_SIZE` to grow buffers on-demand The `ENSURE_SIZE` macro can be used to grow a buffer if its currently allocated size does not suffice a required target size. While most of the code already uses this macro, the `git_buf_join` and `git_buf_join3` functions do not yet use it. Due to the macro first checking whether we have to grow the buffer at all, this has the benefit of saving a function call when it is not needed. While this is nice to have, it will probably not matter at all performance-wise -- instead, this only serves for consistency across the code.
Patrick Steinhardt e82dd813 2017-06-08T11:52:32 buffer: fix `ENSURE_SIZE` macro referencing wrong variable While the `ENSURE_SIZE` macro gets a reference to both the buffer that is to be resized and a new size, we were not consistently referencing the passed buffer, but instead a variable `buf`, which is not passed in. Funnily enough, we never noticed because our buffers seem to always be named `buf` whenever the macro was being used. Fix the macro by always using the passed-in buffer. While at it, add braces around all mentions of passed-in variables as should be done with macros to avoid subtle errors. Found-by: Edward Thompson
Edward Thomson 909d5494 2016-12-29T12:25:15 giterr_set: consistent error messages Error messages should be sentence fragments, and therefore: 1. Should not begin with a capital letter, 2. Should not conclude with punctuation, and 3. Should not end a sentence and begin a new one
Edward Thomson 8cb27223 2015-09-25T10:48:19 git_buf_quote/unquote: handle > \177 Parse values up to and including `\377` (`0xff`) when unquoting. Print octal values as an unsigned char when quoting, lest `printf` think we're talking about negatives.
Edward Thomson d3d95d5a 2015-09-23T16:30:48 git_buf_quote: quote ugly characters
Edward Thomson 5b78dbdb 2015-07-09T13:04:10 git_buf: decode base85 inputs
Edward Thomson d34f6826 2014-04-08T17:18:47 Patch parsing from patch files
Carlos Martín Nieto a6599235 2015-06-24T19:32:56 buffer: make use of EINVALID for growing a borrowed buffer This explains more closely what happens. While here, set an error message.
Carlos Martín Nieto caab22c0 2015-06-23T15:41:58 buffer: don't allow growing borrowed buffers When we don't own a buffer (asize=0) we currently allow the usage of grow to copy the memory into a buffer we do own. This muddles the meaning of grow, and lets us be a bit cavalier with ownership semantics. Don't allow this any more. Usage of grow should be restricted to buffers which we know own their own memory. If unsure, we must not attempt to modify it.
Edward Thomson d4cf1675 2015-02-19T10:05:33 buffer: introduce git_buf_attach_notowned Provide a convenience function that creates a buffer that can be provided to callers but will not be freed via `git_buf_free`, so the buffer creator maintains the allocation lifecycle of the buffer's contents.
Edward Thomson f1453c59 2015-02-12T12:19:37 Make our overflow check look more like gcc/clang's Make our overflow checking look more like gcc and clang's, so that we can substitute it out with the compiler instrinsics on platforms that support it. This means dropping the ability to pass `NULL` as an out parameter. As a result, the macros also get updated to reflect this as well.
Edward Thomson 2884cc42 2015-02-11T09:39:38 overflow checking: don't make callers set oom Have the ALLOC_OVERFLOW testing macros also simply set_oom in the case where a computation would overflow, so that callers don't need to.
Edward Thomson 4aa664ae 2015-02-10T23:55:07 git_buf_grow_by: increase buf asize incrementally Introduce `git_buf_grow_by` to incrementally increase the size of a `git_buf`, performing an overflow calculation on the growth.
Edward Thomson 392702ee 2015-02-09T23:41:13 allocations: test for overflow of requested size Introduce some helper macros to test integer overflow from arithmetic and set error message appropriately.
Jeff Hostetler 4b1018d2 2015-01-08T17:24:12 Fix crash in free() when git_buf_grow() fails.
Vicent Marti 92e0b679 2014-11-21T13:31:30 buffer: Do not `put` anything if len is 0
Edward Thomson e003f83a 2014-07-31T15:14:56 Introduce git_buf_decode_base64 Decode base64-encoded text into a git_buf
joshaber b3af2d80 2014-07-16T13:34:25 Just put it all in buffer.
Russell Belfer 1e4976cb 2014-05-08T10:17:14 Be more careful with user-supplied buffers This adds in missing calls to `git_buf_sanitize` and fixes a number of places where `git_buf` APIs could inadvertently write NUL terminator bytes into invalid buffers. This also changes the behavior of `git_buf_sanitize` to NUL terminate a buffer if it can and of `git_buf_shorten` to do nothing if it can. Adds tests of filtering code with zeroed (i.e. unsanitized) buffer which was previously triggering a segfault.
Edward Thomson e349ed50 2014-04-22T14:58:33 patch: emit binary patches (optionally)
Jacques Germishuys b3b36a68 2014-04-10T12:43:16 Introduce git_buf_putcn Allows for inserting the same character n amount of times
Russell Belfer 18234b14 2014-02-21T09:14:16 Add efficient git_buf join3 API There are a few places where we need to join three strings to assemble a path. This adds a simple join3 function to avoid the comparatively expensive join_n (which calls strlen on each string twice).
Edward Thomson 4f46a98b 2014-02-24T23:32:25 Remove now-duplicated stdarg.h include
Patrick Reynolds 7cbc6241 2014-01-20T11:41:21 fix corner cases and an undefined behavior
Edward Thomson 6adcaab7 2014-01-08T10:07:30 Handle git_buf's from users more liberally
Russell Belfer 00e85927 2013-09-23T21:52:42 Clean up unnecessary git_buf_printf calls This replaces some git_buf_printf calls with simple calls to git_buf_put instead. Also, it fixes a missing va_end inside the git_buf_vprintf implementation.
Vicent Martí 92d19d16 2013-09-21T09:34:03 Merge pull request #1840 from linquize/warning Fix warning
Linquize 66566516 2013-09-08T17:15:42 Fix warning
Russell Belfer 40cb40fa 2013-09-11T14:23:39 Add functions to manipulate filter lists Extend the git2/sys/filter API with functions to look up a filter and add it manually to a filter list. This requires some trickery because the regular attribute lookups and checks are bypassed when this happens, but in the right hands, it will allow a user to have granular control over applying filters.
Russell Belfer a9f51e43 2013-09-11T22:00:36 Merge git_buf and git_buffer This makes the git_buf struct that was used internally into an externally available structure and eliminates the git_buffer. As part of that, some of the special cases that arose with the externally used git_buffer were blended into the git_buf, such as being careful about git_buf objects that may have a NULL ptr and allowing for bufs with a valid ptr and size but zero asize as a way of referring to externally owned data.
Russell Belfer 2a7d224f 2013-09-10T16:33:32 Extend public filter api with filter lists This moves the git_filter_list into the public API so that users can create, apply, and dispose of filter lists. This allows more granular application of filters to user data outside of libgit2 internals. This also converts all the internal usage of filters to the public APIs along with a few small tweaks to make it easier to use the public git_buffer stuff alongside the internal git_buf.
Russell Belfer 0cf77103 2013-08-26T23:17:07 Start of filter API + git_blob_filtered_content This begins the process of exposing git_filter objects to the public API. This includes: * new public type and API for `git_buffer` through which an allocated buffer can be passed to the user * new API `git_blob_filtered_content` * make the git_filter type and GIT_FILTER_TO_... constants public
Russell Belfer 278ce746 2013-07-01T10:20:38 Add helpful buffer shorten function
Edward Thomson 359fc2d2 2013-01-08T17:07:25 update copyrights
Russell Belfer 7bf87ab6 2012-11-28T09:58:48 Consolidate text buffer functions There are many scattered functions that look into the contents of buffers to do various text manipulations (such as escaping or unescaping data, calculating text stats, guessing if content is binary, etc). This groups all those functions together into a new file and converts the code to use that. This has two enhancements to existing functionality. The old text stats function is significantly rewritten and the BOM detection code was extended (although largely we can't deal with anything other than a UTF8 BOM).
nulltoken 3a14d3e2 2012-10-01T11:58:15 buf: introduce git_buf_splice()
Philip Kelley 9d9288f4 2012-10-14T12:29:05 Fix buffer overrun in git_buf_put_base64
Russell Belfer 2d3579be 2012-10-10T14:54:31 Add git_buf_put_base64 to buffer API
Russell Belfer 0c8858de 2012-08-03T14:28:07 Fix valgrind issues and leaks This fixes up a number of problems flagged by valgrind and also cleans up the internal `git_submodule` allocation handling overall with a simpler model.
yorah 02a0d651 2012-07-12T16:31:59 Add git_buf_unescape and git__unescape to unescape all characters in a string (in-place)
Russell Belfer 54e29b93 2012-07-10T23:51:32 Fix missing NUL termination of buffer
Russell Belfer 039fc406 2012-07-10T15:10:14 Add a couple of useful git_buf utilities * `git_buf_rfind` (with tests and tests for `git_buf_rfind_next`) * `git_buf_puts_escaped` and `git_buf_puts_escaped_regex` (with tests) to copy strings into a buffer while injecting an escape sequence (e.g. '\') in front of particular characters.
Vicent Martí 3f035860 2012-06-07T22:43:03 misc: Fix warnings from PVS Studio trial
Russell Belfer e3557172 2012-05-17T14:44:17 No point in keeping commented out fn
Russell Belfer a0d95962 2012-05-17T13:14:17 Other optimization and warning fixes This fixes a warning left by the earlier optimization and addresses one of the other hotspots identified by GProf.
Russell Belfer b59c73d3 2012-05-17T13:06:20 Optimize away git_text_gather_stats in diff GProf shows `git_text_gather_stats` as the most expensive call in large diffs. The function calculates a lot of information that is not actually used and does not do so in a optimal order. This introduces a tuned `git_buf_is_binary` function that executes the same algorithm in a fraction of the time.
Russell Belfer 2c833917 2012-05-15T16:33:05 Document git_buf_common_prefix This function fills in a git_buf with the common prefix of an array of strings, but let's make that a little more clear.
Russell Belfer 41a82592 2012-05-15T14:17:39 Ranged iterators and rewritten git_status_file The goal of this work is to rewrite git_status_file to use the same underlying code as git_status_foreach. This is done in 3 phases: 1. Extend iterators to allow ranged iteration with start and end prefixes for the range of file names to be covered. 2. Improve diff so that when there is a pathspec and there is a common non-wildcard prefix of the pathspec, it will use ranged iterators to minimize excess iteration. 3. Rewrite git_status_file to call git_status_foreach_ext with a pathspec that covers just the one file being checked. Since ranged iterators underlie the status & diff implementation, this is actually fairly efficient. The workdir iterator does end up loading the contents of all the directories down to the single file, which should ideally be avoided, but it is pretty good.
Vicent Martí 0f49200c 2012-05-09T04:37:02 msvc: Do not use `isspace` Locale-aware bullshit bitting my ass again yo
Vicent Martí baaf1c47 2012-05-02T23:44:22 buffer: Add `git_buf_vprintf`
Russell Belfer 2bc8fa02 2012-04-17T10:14:24 Implement git_pool paged memory allocator This adds a `git_pool` object that can do simple paged memory allocation with free for the entire pool at once. Using this, you can replace many small allocations with large blocks that can then cheaply be doled out in small pieces. This is best used when you plan to free the small blocks all at once - for example, if they represent the parsed state from a file or data stream that are either all kept or all discarded. There are two real patterns of usage for `git_pools`: either for "string" allocation, where the item size is a single byte and you end up just packing the allocations in together, or for "fixed size" allocation where you are allocating a large object (e.g. a `git_oid`) and you generally just allocation single objects that can be tightly packed. Of course, you can use it for other things, but those two cases are the easiest.
Russell Belfer 952f94c8 2012-03-30T14:42:23 Fix bug when join_n refers to original buffer There was a bug in git_buf_join_n when the contents of the original buffer were joined into itself and the realloc moved the pointer to the original buffer.
Russell Belfer 0d0fa7c3 2012-03-16T15:56:01 Convert attr, ignore, mwindow, status to new errors Also cleaned up some previously converted code that still had little things to polish.
Russell Belfer deafee7b 2012-03-14T17:36:15 Continue error conversion This converts blob.c, fileops.c, and all of the win32 files. Also, various minor cleanups throughout the code. Plus, in testing the win32 build, I cleaned up a bunch (although not all) of the warnings with the 64-bit build.
Vicent Martí cb8a7961 2012-03-07T00:02:55 error-handling: Repository This also includes droping `git_buf_lasterror` because it makes no sense in the new system. Note that in most of the places were it has been dropped, the code needs cleanup. I.e. GIT_ENOMEM is going away, so instead it should return a generic `-1` and obviously not throw anything.
Russell Belfer ce49c7a8 2012-03-02T15:09:40 Add filter tests and fix some bugs This adds some initial unit tests for file filtering and fixes some simple bugs in filter application.
Vicent Martí eb8f90e5 2012-02-27T17:22:51 buffer: Null terminate on rtrim
Vicent Martí 44b1ff4c 2012-02-27T04:31:05 filter: Apply filters before writing a file to the ODB Initial implementation. The relevant code is in `blob.c`: the blob write function has been split into smaller functions. - Directly write a file to the ODB in streaming mode - Directly write a symlink to the ODB in direct mode - Apply a filter, and write a file to the ODB in direct mode When trying to write a file, we first call `git_filter__load_for_file`, which populates a filters array with the required filters based on the filename. If no filters are resolved to the filename, we can write to the ODB in streaming mode straight from disk. Otherwise, we load the whole file in memory and use double-buffering to apply the filter chain. We finish by writing the file as a whole to the ODB.
Vicent Martí 13224ea4 2012-02-27T04:28:31 buffer: Unify `git_fbuffer` and `git_buf` This makes so much sense that I can't believe it hasn't been done before. Kill the old `git_fbuffer` and read files straight into `git_buf` objects. Also: In order to fully support 4GB files in 32-bit systems, the `git_buf` implementation has been changed from using `ssize_t` for storage and storing negative values on allocation failure, to using `size_t` and changing the buffer pointer to a magical pointer on allocation failure. Hopefully this won't break anything.
Russell Belfer b6c93aef 2012-02-21T14:46:24 Uniform iterators for trees, index, and workdir This create a new git_iterator type of object that provides a uniform interface for iterating over the index, an arbitrary tree, or the working directory of a repository. As part of this, git ignore support was extended to support push and pop of directory-based ignore files as the working directory is being traversed (so the array of ignores does not have to be recreated at each directory during traveral). There are a number of other small utility functions in buffer, path, vector, and fileops that are included in this patch that made the iterator implementation cleaner.
schu 5e0de328 2012-02-13T17:10:24 Update Copyright header Signed-off-by: schu <schu-github@schulog.org>
Russell Belfer 1dbcc9fc 2012-01-11T21:07:16 Fix several memory issues This contains fixes for several issues discovered by MSVC and by valgrind, including some bad data access, some memory leakage (in where certain files were not being successfully added to the cache), and some code simplification.
Russell Belfer df743c7d 2012-01-09T15:37:19 Initial implementation of gitignore support Adds support for .gitignore files to git_status_foreach() and git_status_file(). This includes refactoring the gitattributes code to share logic where possible. The GIT_STATUS_IGNORED flag will now be passed in for files that are ignored (provided they are not already in the index or the head of repo).
Russell Belfer b5daae68 2011-12-14T12:34:43 Allow git_buf_joinpath to accept self-joins It was not safe for git_buf_joinpath to be used with a pointer into the buf itself because a reallocation could invalidate the input parameter that pointed into the buffer. This patch makes it safe to self join, at least for the leading input to the join, which is the common "append" case for self joins. Also added unit tests to explicitly cover this case. This should actually fix #511
Vicent Marti bf6d2717 2011-12-14T03:27:53 buffer: inline `git_buf_cstr`
Russell Belfer 97769280 2011-11-30T11:27:15 Use git_buf for path storage instead of stack-based buffers This converts virtually all of the places that allocate GIT_PATH_MAX buffers on the stack for manipulating paths to use git_buf objects instead. The patch is pretty careful not to touch the public API for libgit2, so there are a few places that still use GIT_PATH_MAX. This extends and changes some details of the git_buf implementation to add a couple of extra functions and to make error handling easier. This includes serious alterations to all the path.c functions, and several of the fileops.c ones, too. Also, there are a number of new functions that parallel existing ones except that use a git_buf instead of a stack-based buffer (such as git_config_find_global_r that exists alongsize git_config_find_global). This also modifies the win32 version of p_realpath to allocate whatever buffer size is needed to accommodate the realpath instead of hardcoding a GIT_PATH_MAX limit, but that change needs to be tested still.
Russell Belfer 969d588d 2011-11-30T13:10:47 Optimized of git_buf_join. This streamlines git_buf_join and removes the join-append behavior, opting instead for a very compact join-replace of the git_buf contents. The unit tests had to be updated to remove the join-append tests and have a bunch more exhaustive tests added.
Russell Belfer 309113c9 2011-11-29T23:45:17 Make initial value of git_buf ptr always be a valid empty string. Taking a page from core git's strbuf, this introduces git_buf_initbuf which is an empty string that is used to initialize the git_buf ptr value even for new buffers. Now the git_buf ptr will always point to a valid NUL-terminated string. This change required jumping through a few hoops for git_buf_grow and git_buf_free to distinguish between a actual allocated buffer and the global initial value. Also, this moves the allocation related functions to be next to each other near the top of buffer.c.
Russell Belfer c63728cd 2011-11-29T16:39:49 Make git_buf functions always maintain a valid cstr. At a tiny cost of 1 extra byte per allocation, this makes git_buf_cstr into basically a noop, which simplifies error checking when trying to convert things to use dynamic allocation. This patch also adds a new function (git_buf_copy_cstr) for copying the cstr data directly into an external buffer.
Russell Belfer 679b69c4 2011-11-28T13:05:25 Resolve remaining feedback * replace some ints with size_ts * update NULL checks in various places
Russell Belfer 3aa294fd 2011-11-28T10:42:57 Add two string git_buf_join and tweak input error checking. This commit addresses two of the comments: * renamed existing n-input git_buf_join to git_buf_join_n * added new git_buf_join that always takes two inputs * moved some parameter error checking to asserts * extended unit tests to cover new version of git_buf_join
Russell Belfer 8c74d22e 2011-11-27T21:47:58 Extend git_buf with new utility functions and unit tests. Add new functions to git_buf for: * initializing a buffer from a string * joining one or more strings onto a buffer with separators * swapping two buffers in place * extracting data from a git_buf (leaving it empty) Also, make git_buf_free leave a git_buf back in its initted state, and slightly tweak buffer allocation sizes and thresholds. Finally, port unit tests to clay and extend with lots of new tests for the various git_buf functions.
Vicent Marti 3286c408 2011-10-28T14:51:13 global: Properly use `git__` memory wrappers Ensure that all memory related functions (malloc, calloc, strdup, free, etc) are using their respective `git__` wrappers.
Vicent Martí 8114ee4c 2011-09-22T10:17:43 Merge pull request #405 from carlosmn/http-ls Implement ls-remote over HTTP
Vicent Marti bb742ede 2011-09-19T01:54:32 Cleanup legal data 1. The license header is technically not valid if it doesn't have a copyright signature. 2. The COPYING file has been updated with the different licenses used in the project. 3. The full GPLv2 header in each file annoys me.
Carlos Martín Nieto b87600cb 2011-09-05T02:33:02 buffer: add git_buf_clear Set the size to zero so the memory that has already been allocated can be reused Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>
Carlos Martín Nieto c7c30513 2011-09-05T21:38:56 buffer: add git_buf_consume Moves the content after 'end' to the beginning of the buffer Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>
Vicent Marti afeecf4f 2011-07-09T02:10:46 odb: Direct writes are back DIRECT WRITES ARE BACK AND FASTER THAN EVER. The streaming writer to the ODB was an overkill for the smaller objects like Commit and Tags; most of the streaming logic was taking too long. This commit makes Commits, Tags and Trees to be built-up in memory, and then written to disk in 2 pushes (header + data), instead of streaming everything. This is *always* faster, even for big files (since the git_filebuf class still does streaming writes when the memory cache overflows). This is also a gazillion lines of code smaller, because we don't have to precompute the final size of the object before starting the stream (this was kind of defeating the point of streaming, anyway). Blobs are still written with full streaming instead of loading them in memory, since this is still the fastest way. A new `git_buf` class has been added. It's missing some features, but it'll get there.