src/config_file.c


Log

Author Commit Date CI Message
Patrick Steinhardt 1b329089 2017-05-31T22:27:19 config_file: refuse modifying included variables Modifying variables pulled in by an included file currently succeeds, but it doesn't actually do what one would expect, as refreshing the configuration will cause the values to reappear. As we are currently not really able to support this use case, we will instead just return an error for deleting and setting variables which were included via an include.
Patrick Steinhardt 3a7f7a6e 2017-05-31T14:43:46 config_file: pass reader directly to callbacks Previously, the callbacks passed to `config_parse` got the reader via a pointer to a pointer. This allowed the callbacks to update the callers `reader` variable when the array holding it has been reallocated. As the array is no longer present, we can simply the code by making the reader a simple pointer.
Patrick Steinhardt 73df75d8 2017-05-31T14:34:48 config_file: refactor include handling Current code for configuration files uses the `reader` structure to parse configuration files and store additional metadata like the file's path and checksum. These structures are stored within an array in the backend itself, which causes multiple problems. First, it does not make sense to keep around the file's contents with the backend itself. While this data is usually free'd before being added to the backend, this brings along somewhat intricate lifecycle problems. A better solution would be to store only the file paths as well as the checksum of the currently parsed content only. The second problem is that the `reader` structures are stored inside an array. When re-parsing configuration files due to changed contents, we may cause this array to be reallocated, requiring us to update pointers hold by callers. Furthermore, we do not keep track of includes which are already associated to a reader inside of this array. This causes us to add readers multiple times to the backend, e.g. in the scenario of refreshing configurations. This commit fixes these shortcomings. We introduce a split between the parsing data and the configuration file's metadata. The `reader` will now only hold the file's contents and the parser state and the new `config_file` structure holds the file's path and checksum. Furthermore, the new structure is a recursive structure in that it will also hold references to the files it directly includes. The diskfile is changed to only store the top-level configuration file. These changes allow us further refactorings and greatly simplify understanding the code.
Patrick Steinhardt 28c2cc3d 2017-05-31T16:41:44 config_file: move reader into `config_read` only Right now, we have multiple call sites which initialize a `reader` structure. As the structure is only actually used inside of `config_read`, we can instead just move the reader inside of the `config_read` function. Instead, we can just pass in the configuration file into `config_read`, which eases code readability.
Patrick Steinhardt 83bcd3a1 2017-05-31T22:45:25 config_file: refresh all files if includes were modified Currently, we only re-parse the top-level configuration file when it has changed itself. This can cause problems when an include is changed, as we were not updating all values correctly. Instead of conditionally reparsing only refreshed files, the logic becomes much clearer and easier to follow if we always re-parse the top-level configuration file when either the file itself or one of its included configuration files has changed on disk. This commit implements this logic. Note that this might impact performance in some cases, as we need to re-read all configuration files whenever any of the included files changed. It could increase performance to just re-parse include files which have actually changed, but this would compromise maintainability of the code without much gain. The only case where we will gain anything is when we actually use includes and when only these includes are updated, which will probably be quite an unusual scenario to actually be worthwhile to optimize.
Patrick Steinhardt 56a7a264 2017-05-31T14:50:40 config_file: remove unused backend field from parse data The backend passed to `config_read` is never actually used anymore, so we can remove it from the function and the `parse_data` structure.
Patrick Steinhardt a693b873 2017-06-07T10:20:44 buffer: use `git_buf_init` with length The `git_buf_init` function has an optional length parameter, which will cause the buffer to be initialized and allocated in one step. This can be used instead of static initialization with `GIT_BUF_INIT` followed by a `git_buf_grow`. This patch does so for two functions where it is applicable.
Carlos Martín Nieto a1023a43 2017-05-20T17:18:07 Merge pull request #4179 from libgit2/ethomson/expand_tilde Introduce home directory expansion function for config files, attribute files
Patrick Steinhardt 4467aeac 2017-03-28T09:00:48 config_file: handle errors other than OOM while parsing section headers The current code in `parse_section_header_ext` is only prepared to properly handle out-of-memory conditions for the `git_buf` structure. While very unlikely and probably caused by a programming error, it is also possible to run into error conditions other than out-of-memory previous to reaching the actual parsing loop. In these cases, we will run into undefined behavior as the `rpos` variable is only initialized after these triggerable errors, but we use it in the cleanup-routine. Fix the issue by unifying the function's cleanup code with an `end_error` section, which will not use the `rpos` variable.
Edward Thomson 29aef948 2017-03-23T11:59:06 config, attrcache: don't fallback to dirs literally named `~` The config and attrcache file reading code would attempt to load a file in a home directory by expanding the `~` and looking for the file, using `git_sysdir_find_global_file`. If the file was not found, the error handling would look for the literal path, eg `~/filename.txt`. Use the new `git_config_expand_global_file` instead, which allows us to get the path to the file separately, when the path is prefixed with `~/`, and fail with a not found error without falling back to looking for the literal path.
Sim Domingo 301dc26a 2016-06-20T13:15:35 fix error when including a missing config file relative to the home directory
Patrick Steinhardt 2cf48e13 2017-03-20T09:34:41 config_file: check if section header buffer runs out of memory While parsing section headers, we use a buffer to store the actual section name. We do not check though if the buffer runs out of memory at any stage. Do so.
Patrick Steinhardt 13c3bc9a 2017-01-27T14:32:23 strmap: remove GIT__USE_STRMAP macro
Patrick Steinhardt 73028af8 2017-01-27T14:20:24 khash: avoid using macro magic to get return address
Edward Thomson 909d5494 2016-12-29T12:25:15 giterr_set: consistent error messages Error messages should be sentence fragments, and therefore: 1. Should not begin with a capital letter, 2. Should not conclude with punctuation, and 3. Should not end a sentence and begin a new one
Arthur Schreiber ab96ca55 2016-10-06T13:15:31 Make sure we use the `C` locale for `regcomp` on macOS.
Carlos Martín Nieto 2f0450f4 2016-03-29T03:26:43 Merge pull request #3712 from ethomson/config_duplicate_section config: don't write duplicate section
Edward Thomson e25e1ca1 2016-03-28T11:13:51 config: don't write section header if we're in it If we hit the EOF while trying to write a new value, it may be that we're already in the section that we were looking for. If so, do not write a (duplicate) section header, just write the value.
Carlos Martín Nieto 6f09911c 2016-03-21T21:10:26 config: don't special-case multivars that don't exist yet This special-casing ignores that we might have a locked file, so the hashtable does not represent the contents of the file we want to write. This causes multivar writes to overwrite entries instead of add to them when under lock. There is no need for this as the normal code-path will write to the file just fine, so simply get rid of it.
Carlos Martín Nieto 77394a27 2016-03-14T19:15:20 Merge pull request #3677 from pks-t/pks/coverity-fixes-round7 Coverity fixes round 7
Patrick Steinhardt 836447e5 2016-03-10T16:52:09 config_file: handle error when trying to lock strmap Accessing the current values map is handled through the `refcounder_strmap_take` function, which first acquires a mutex before accessing its values. While this assures everybody is trying to access the values with the mutex only we do not check if the locking actually succeeds. Fix the issue by checking if acquiring the lock succeeds and returning `NULL` if we encounter an error. Adjust callers.
Patrick Steinhardt e126bc95 2016-03-01T14:40:17 config_file: handle missing quotation marks in section header When parsing a section header we expect something along the format of '[section "subsection"]'. When a section is mal-formated and is entirely missing its quotation marks we catch this case by observing that `strchr(line, '"') - strrchr(line, '"') = NULL - NULL = 0` and error out. Unfortunately, the error message is misleading though, as we state that we are missing the closing quotation mark while we in fact miss both quotation marks. Improve the error message by explicitly checking if the first quotation mark could be found and, if not, stating that quotation marks are completely missing.
Carlos Martín Nieto eb597799 2015-10-29T21:12:37 filebuf: use a checksum to detect file changes Instead of relying on the size and timestamp, which can hide changes performed in the same second, hash the file content's when we care about detecting changes.
Carlos Martín Nieto cd677b8f 2015-09-18T12:28:05 config: buffer comments to match git's variable-adding When there is a comment at the end of a section, git keeps it there, while we write the new variable right at the end. Keep comments buffered and dump them when we're going to output a variable or section, or reach EOF. This puts us in line with the config files which git produces.
Carlos Martín Nieto b1667039 2015-06-01T19:17:03 config: implement basic transactional support When a configuration file is locked, any updates made to it will be done to the in-memory copy of the file. This allows for multiple updates to happen while we hold the lock, preventing races during complex config-file manipulation.
Carlos Martín Nieto 3ce9e4d2 2015-06-01T08:45:15 config: write the modified file to memory Instead of writing into the filebuf directly, make the functions to write the modified config file write into a buffer which can then be dumped into the lockfile for committing. This allows us to re-use the same code for modifying a locked configuration, as we can simply skip the last step of dumping the data to disk.
Edward Thomson 75a4636f 2015-05-29T16:56:38 git__tolower: a tolower() that isn't dumb Some brain damaged tolower() implementations appear to want to take the locale into account, and this may require taking some insanely aggressive lock on the locale and slowing down what should be the most trivial of trivial calls for people who just want to downcase ASCII.
Carlos Martín Nieto b162d97a 2015-05-05T09:47:16 config: plug a couple of leaks
Edward Thomson 63c0cc65 2015-04-27T16:29:00 config: cleanup some now-unused variables
Edward Thomson 9c26de0f 2015-04-27T15:38:44 config: lock the file for write before reading When writing a configuration file, we want to take a lock on the new file (eg, `config.lock`) before opening the configuration file (`config`) for reading so that we can prevent somebody from changing the contents underneath us.
Edward Thomson 2a950c94 2015-04-27T10:43:50 config: write existing lines as-is when rewriting When updating a configuration file, we want to copy the old data from the file to preserve comments and funny whitespace, instead of writing it in some "canonical" format. Thus, we keep a pointer to the start of the line and the line length to preserve these things we don't care to rewrite.
Edward Thomson bf99390e 2015-04-23T16:54:36 config: examine whole file when writing Previously we would try to be clever when writing the configuration file and try to stop parsing (and simply copy the rest of the old file) when we either found the value we were trying to write, or when we left the section that value was in, the assumption being that there was no more work to do. Regrettably, you can have another section with the same name later in the file, and we must cope with that gracefully, thus we read the whole file in order to write a new file. Now, writing a file looks even more than reading. Pull the config parsing out into its own function that can be used by both reading and writing the configuration.
Edward Thomson d369d71f 2015-04-21T17:18:35 config: peek returns '\n' on EOF; handle in write
Edward Thomson 2c8c00c6 2015-04-21T12:52:29 config: validate config keys
Ryan Roden-Corrent 9a810c5e 2015-04-16T15:32:16 git_config_delete: search until last section. If git_config_delete is to work properly in the presence of duplicate section headers, it cannot stop searching at the end of the first matching section, as there may be another matching section later. When config_write is used for deletion (value = NULL), it may only terminate when the desired key is found or there are no sections left to parse.
Edward Thomson e009a705 2015-04-20T00:22:20 config_file: comment char can be invalid escape Don't assume that comment chars are comment chars, they may be (an attempt to be escaped). If so, \; is not a valid escape sequence, complain.
Edward Thomson 7f2e61f3 2015-04-19T23:55:02 config_file: parse multilines generously Combine unquoting and multiline detection to avoid ambiguity when parsing.
Patrick Steinhardt 129022ee 2015-04-10T09:36:38 Fix checking of return value for regcomp. The regcomp function returns a non-zero value if compilation of a regular expression fails. In most places we only check for negative values, but positive values indicate an error, as well. Fix this tree-wide, fixing a segmentation fault when calling git_config_iterator_glob_new with an invalid regexp.
Carlos Martín Nieto 9a97f49e 2014-12-21T15:31:03 config: borrow refcounted references This changes the get_entry() method to return a refcounted version of the config entry, which you have to free when you're done. This allows us to avoid freeing the memory in which the entry is stored on a refresh, which may happen at any time for a live config. For this reason, get_string() has been forbidden on live configs and a new function get_string_buf() has been added, which stores the string in a git_buf which the user then owns. The functions which parse the string value takea advantage of the borrowing to parse safely and then release the entry.
Stefan Widgren c8e02b87 2015-02-15T21:07:05 Remove extra semicolon outside of a function Without this change, compiling with gcc and pedantic generates warning: ISO C does not allow extra ‘;’ outside of a function.
Carlos Martín Nieto a7fa970f 2015-02-15T05:13:50 Merge pull request #2895 from ethomson/alloc_overflow allocations: test for overflow of requested size
Edward Thomson f1453c59 2015-02-12T12:19:37 Make our overflow check look more like gcc/clang's Make our overflow checking look more like gcc and clang's, so that we can substitute it out with the compiler instrinsics on platforms that support it. This means dropping the ability to pass `NULL` as an out parameter. As a result, the macros also get updated to reflect this as well.
Edward Thomson 2884cc42 2015-02-11T09:39:38 overflow checking: don't make callers set oom Have the ALLOC_OVERFLOW testing macros also simply set_oom in the case where a computation would overflow, so that callers don't need to.
Edward Thomson 392702ee 2015-02-09T23:41:13 allocations: test for overflow of requested size Introduce some helper macros to test integer overflow from arithmetic and set error message appropriately.
Yury G. Kudryashov 17136538 2015-02-05T23:39:59 Reinit `reader` pointer after reading included config file Fixes #2869. If included file includes more files, it may reallocate cfg_file->readers, hence invalidate not only `r` pointer, but `result` pointer as well.
Jacques Germishuys 6f73e026 2014-12-24T11:42:50 Plug some leaks
Will Stamper b874629b 2014-12-04T21:06:59 Spelling fixes
John Fultz ebc13b2b 2014-11-02T19:16:49 Clean up issues include.path issues found during code review. * Error-handling is cleaned up to only let a file-not-found error through, not other sorts of errors. And when a file-not-found error happens, we clean up the error. * Test now checks that file-not-found introduces no error. And other minor cleanups.
John Fultz 727ae380 2014-11-01T11:21:45 Make config reading continue after hitting a missing include file. For example, if you have [include] path = foo and foo didn't exist, git_config_open_ondisk() would just give up on the rest of the file. Now it ignores the unresolved include without error and continues reading the rest of the file.
Carlos Martín Nieto 55cb4999 2014-10-23T19:05:02 config: remove the refresh function and backend field We have been refreshing on read and write for a while now, so git_config_refresh() is at best a no-op, and might just end up wasting cycles.
Alan Rogers ad5adacb 2014-10-21T09:29:45 Patch from @carlosmn to refresh the parent config before snapshotting.
Linquize 0a641647 2014-10-04T23:27:06 config: Fix multiple trailing spaces before comments not completely trimmed
Carlos Martín Nieto 9dac1f95 2014-08-09T10:56:50 config: a multiline var can start immediately In the check for multiline, we traverse the backslashes from the end backwards and int the end assert that we haven't gone past the beginning of the line. We make sure of this in the loop condition, but we also check in the return value. However, for certain configurations, a line in a multiline variable might be empty to aid formatting. In that case, 'end' == 'start', since we ended up looking at the first char which made it a multiline. There is no need for the (end > start) check in the return, since the loop guarantees we won't go further back than the first char in the line, and we do accept the first char to be the final backslash. This fixes #2483.
Linquize 991dab2d 2014-07-16T21:09:53 Make sure \n is at the end of config file before a new section is written
Philip Kelley 4af0ef96 2014-05-15T11:09:49 Fix mutex init/free in config_file.c
Russell Belfer a37aa82e 2014-05-13T15:54:23 Some coverity inspired cleanups
Russell Belfer b1914c36 2014-05-12T10:24:46 Minor fixes for warnings and error propagation
Carlos Martín Nieto 2280b388 2014-04-09T13:17:51 config: share the strmap on snapshot Now that our strmap is no longer modified but replaced, we can use the same strmap for the snapshot's values and it will be freed when we don't need it anymore.
Carlos Martín Nieto 4b99b8f5 2014-04-09T13:08:01 config: refcount the values map This is mostly groundwork to let us re-use the map in the snapshots.
Carlos Martín Nieto 8c1f4ab4 2014-04-09T12:26:53 config: refresh on delete When we delete an entry, we also want to refresh the configuration to catch any changes that happened externally. This allows us to simplify the logic, as we no longer need to delete these variables internally. The whole state will be refreshed and the deleted entries won't be there.
Carlos Martín Nieto 523032cd 2014-03-31T09:58:44 config: refresh before reading a value With the isolation of complex reads, we can now try to refresh the on-disk file before reading a value from it. This changes the semantics a bit, as before we could be sure that a string we got from the configuration was valid until we wrote or refreshed. This is no longer the case, as a read can also invalidate the pointer.
Carlos Martín Nieto eaf37034 2014-03-31T08:53:56 config: refresh the values on write When writing out, parse the resulting file instead of adding or replacing the value locally. This has the effect of reading external changes as well.
Carlos Martín Nieto 0500a1ef 2014-03-31T08:32:45 config: use a snapshot for the iterator
Carlos Martín Nieto bd95f836 2014-03-31T08:08:17 config: split out the refresh step This will be used by the writing commands in a later step.
Carlos Martín Nieto c047317e 2014-03-18T17:54:32 config: make refresh atomic Current code sets the active map to a new one and builds it whilst it's active. This is a race condition with someone else trying to access the same config. Instead, let's build up our new map and swap the active and new one.
Carlos Martín Nieto 55ebd7d3 2014-03-13T17:11:34 config: implement config snapshotting In order to have consistent views of the config files for remotes, submodules et al. and a configuration that represents what is currently stored on-disk, we need a way to provide a view of the configuration that does not change. The goal here is to provide the snapshotting part by creating a read-only copy of the state of the configuration at a particular point in time, which does not change when a repository's main config changes.
Russell Belfer 40ed4990 2014-02-11T14:45:37 Add diff threading tests and attr file cache locks This adds a basic test of doing simultaneous diffs on multiple threads and adds basic locking for the attr file cache because that was the immediate problem that arose from these tests.
Edward Thomson 83634d38 2014-02-24T17:43:10 Move system directory cache out of utils
Arthur Schreiber a8e4cb11 2014-01-13T22:12:17 Fix a memory leak in `config_parse`.
Robert Konrad 6014b7b5 2013-12-30T18:08:04 Fixed a compile error in VS2013.
Russell Belfer 9f77b3f6 2013-11-25T14:21:34 Add config read fns with controlled error behavior This adds `git_config__lookup_entry` which will look up a key in a config and return either the entry or NULL if the key was not present. Optionally, it can either suppress all errors or can return them (although not finding the key is not an error for this function). Unlike other accessors, this does not normalize the config key string, so it must only be used when the key is known to be in normalized form (i.e. all lower-case before the first dot and after the last dot, with no invalid characters). This also adds three high-level helper functions to look up config values with no errors and a fallback value. The three functions are for string, bool, and int values, and will resort to the fallback value for any error that arises. They are: * `git_config__get_string_force` * `git_config__get_bool_force` * `git_config__get_int_force` None of them normalize the config `key` either, so they can only be used for internal cases where the key is known to be in normal format.
Vicent Marti a1d35ede 2013-11-10T16:41:41 config_file: style
Vicent Martí b9cb72c2 2013-11-10T07:33:11 Merge pull request #1950 from csware/quote-config-values Correctly quote config values while saving
Sven Strickroth 590c5efb 2013-11-07T17:51:43 Rename method Signed-off-by: Sven Strickroth <email@cs-ware.de>
Sven Strickroth fde93250 2013-11-07T13:17:36 Correctly quote config values while saving If the value contains a command (; or #) char or starts or ends with space it needs to be quoted. Signed-off-by: Sven Strickroth <email@cs-ware.de>
nulltoken 61080a95 2013-11-05T15:10:02 Fix leaks
nulltoken e8162fd0 2013-11-05T12:01:14 Propagate ELOCKED error when updating the config
Edward Thomson 1d3a8aeb 2013-11-04T18:28:57 move mode_t to filebuf_open instead of _commit
Carlos Martín Nieto a7a64d2c 2013-11-02T18:36:34 remote: don't write too much when dealing with multivars We used to move `data_start` forward, which is wrong as that needs to point to the beginning of the buffer in order to perform size calculations. Introduce a `write_start` variable which indicates where we should start writing from, which is what the `data_start` was being wrongly reused to be.
Vicent Marti b22593fb 2013-11-01T17:28:59 config_file: Style fixes
Daniel Rodríguez Troitiño a71331eb 2013-10-31T23:41:48 Fix memory leaks.
Daniel Rodríguez Troitiño 3793fa9b 2013-10-31T01:08:50 Fix saving remotes with several fetch/push ref specs. At some moment git_config_delete_entry lost the ability to delete one entry of a multivar configuration. The moment you had more than one fetch or push ref spec for a remote you will not be able to save that remote anymore. The changes in network::remote::remotes::save show that problem. I needed to create a new git_config_delete_multivar because I was not able to remove one or several entries of a multivar config with the current API. Several tries modifying how git_config_set_multivar(..., NULL) behaved were not successful. git_config_delete_multivar is very similar to git_config_set_multivar, and delegates into config_delete_multivar of config_file. This function search for the cvar_t that will be deleted, storing them in a temporal array, and rebuilding the linked list. After calling config_write to delete the entries, the cvar_t stored in the temporal array are freed. There is a little fix in config_write, it avoids an infinite loop when using a regular expression (case for the multivars). This error was found by the test network::remote::remotes::tagopt.
Linquize 566dd8ce 2013-09-30T23:38:22 Config subsection name should allow to have ']' and '\\' should allow to escape any characters
Vicent Martí 92d19d16 2013-09-21T09:34:03 Merge pull request #1840 from linquize/warning Fix warning
Linquize 66566516 2013-09-08T17:15:42 Fix warning
Russell Belfer a9f51e43 2013-09-11T22:00:36 Merge git_buf and git_buffer This makes the git_buf struct that was used internally into an externally available structure and eliminates the git_buffer. As part of that, some of the special cases that arose with the externally used git_buffer were blended into the git_buf, such as being careful about git_buf objects that may have a NULL ptr and allowing for bufs with a valid ptr and size but zero asize as a way of referring to externally owned data.
Carlos Martín Nieto 53ea0513 2013-09-07T19:07:39 config: handle realloc issues from larger depths As the include depth increases, the chance of a realloc increases. This means that whenever we run git_array_alloc() or call config_parse(), we need to remember what our reader's index is so we can look it up again.
Carlos Martín Nieto 69789922 2013-09-07T18:50:35 config: return an error when reaching the maximum include depth
Carlos Martín Nieto 73fc5e01 2013-09-06T21:12:26 config: fix variable overriding When two or more variables of the same name exist and the user asks for a scalar, we must return the latest value assign to it.
Carlos Martín Nieto a9fb7989 2013-09-06T20:51:35 config: refresh included files We need to refresh the variables from the included files if they are changed, so loop over all included files and re-parse the files if any of them has changed.
Carlos Martín Nieto 19be0692 2013-09-06T20:34:02 config: keep a list of included files When refreshing we need to refresh if any of the files have been touched, so we need to keep the list.
Carlos Martín Nieto d8d25acb 2013-09-05T19:24:20 config: add support for include directives Relative, absolute and home-relative paths are supported. The recursion limit it set at 10, just like in git.
Carlos Martín Nieto d209cc47 2013-09-05T18:06:12 config: decouple the backend from the reader at the low level In order to support config includes, we must differentiate between the backend's main file and the file we are currently parsing. This lays the groundwork for includes, keeping the current behaviours.
Vicent Martí b8b22d77 2013-08-28T06:04:51 Merge pull request #1772 from libgit2/config-iter Configuration iterators redux
Carlos Martín Nieto f4be8209 2013-08-14T00:45:05 config: don't special-case the multivar iterator Build it on top of the normal iterator instead, which lets use re-use a lot of code.
Carlos Martín Nieto 43e5dda7 2013-08-12T11:21:57 config: get rid of a useless asignment
Carlos Martín Nieto 5880962d 2013-08-09T09:05:19 config: introduce _iterator_new() As the name suggests, it iterates over all the entries
Russell Belfer 7f7ebe13 2013-08-08T12:57:13 Merge pull request #1771 from nvloff/write_empty_config_value config: allow setting empty string as value
Nikolai Vladimirov c57f6682 2013-08-08T21:17:32 config: allow empty string as value `git_config_set_string(config, "config.section", "")` fails when escaping the value. The buffer in `escape_value` is allocated without NULL-termination. And in case of empty string 0 is passed for buffer size in `git_buf_grow`. `git_buf_detach` returns NULL when the allocated size is 0 and that leads to an error return in `GITERR_CHECK_ALLOC` called after `escape_value` The change in `config_file.c` was suggested by Russell Belfer <rb@github.com>
Carlos Martín Nieto 99dfb538 2013-08-08T17:57:59 config: working multivar iterator Implement the foreach version as a wrapper around the iterator.