src/transports/smart_protocol.c


Log

Author Commit Date CI Message
Edward Thomson 6fc6eeb6 2021-12-24T15:14:38 remote: introduce `git_remote_connect_options` The existing mechanism for providing options to remote fetch/push calls, and subsequently to transports, is unsatisfactory. It requires an options structure to avoid breaking the API and callback signatures. 1. Introduce `git_remote_connect_options` to satisfy those needs. 2. Add a new remote connection API, `git_remote_connect_ext` that will take this new options structure. Existing `git_remote_connect` calls will proxy to that. `git_remote_fetch` and `git_remote_push` will proxy their fetch/push options to that as well. 3. Define the interaction between `git_remote_connect` and fetch/push. Connect _may_ be called before fetch/push, but _need not_ be. The semantics of which options would be used for these operations was not specified if you specify options for both connect _and_ fetch. Now these are defined that the fetch or push options will be used _if_ they were specified. Otherwise, the connect options will be used if they were specified. Otherwise, the library's defaults will be used. 4. Update the transports to understand `git_remote_connect_options`. This is a breaking change to the systems API.
Edward Thomson f0e693b1 2021-09-07T17:53:49 str: introduce `git_str` for internal, `git_buf` is external libgit2 has two distinct requirements that were previously solved by `git_buf`. We require: 1. A general purpose string class that provides a number of utility APIs for manipulating data (eg, concatenating, truncating, etc). 2. A structure that we can use to return strings to callers that they can take ownership of. By using a single class (`git_buf`) for both of these purposes, we have confused the API to the point that refactorings are difficult and reasoning about correctness is also difficult. Move the utility class `git_buf` to be called `git_str`: this represents its general purpose, as an internal string buffer class. The name also is an homage to Junio Hamano ("gitstr"). The public API remains `git_buf`, and has a much smaller footprint. It is generally only used as an "out" param with strict requirements that follow the documentation. (Exceptions exist for some legacy APIs to avoid breaking callers unnecessarily.) Utility functions exist to convert a user-specified `git_buf` to a `git_str` so that we can call internal functions, then converting it back again.
Peter Pettersson e4e173e8 2021-07-15T21:00:02 Allow compilation on systems without CLOCK_MONOTONIC Makes usage of CLOCK_MONOTONIC conditional and makes functions that uses git__timer handle clock resynchronization. Call gettimeofday with tzp set to NULL as required by https://pubs.opengroup.org/onlinepubs/9699919799/functions/gettimeofday.html
Edward Thomson 37763d38 2020-12-05T15:26:59 threads: rename git_atomic to git_atomic32 Clarify the `git_atomic` type and functions now that we have a 64 bit version as well (`git_atomic64`).
Edward Thomson 05816a98 2020-04-05T17:20:08 netops: use GIT_ASSERT
Edward Thomson 7e0f5a6a 2019-10-22T22:37:14 smart protocol: correct case in error messages
Etienne Samson 53f51c60 2019-08-21T19:48:05 smart: implement by-date insertion when revwalking
Etienne Samson 39d18fe6 2019-07-31T08:37:10 smart: use push_glob instead of manual filtering The code worked under the assumption that anything under `refs/tags` are tag objects, and all the rest would be peelable to a commit. As it is completely valid to have tags to blobs under a non `refs/tags` ref, this would cause failures when trying to peel a tag to a commit. Fix the broken filtering by switching to `git_revwalk_push_glob`, which already handles this case.
Edward Thomson 8c925ef8 2019-05-21T14:30:28 smart protocol: validate progress message length Ensure that the server has not sent us overly-large sideband messages (ensure that they are no more than `INT_MAX` bytes), then cast to `int`.
Edward Thomson 5d92e547 2019-06-08T17:28:35 oid: `is_zero` instead of `iszero` The only function that is named `issomething` (without underscore) was `git_oid_iszero`. Rename it to `git_oid_is_zero` for consistency with the rest of the library.
Edward Thomson 59001e83 2019-02-21T11:41:19 remote: rename git_push_transfer_progress callback The `git_push_transfer_progress` is a callback and as such should be suffixed with `_cb` for consistency. Rename `git_push_transfer_progress` to `git_push_transfer_progress_cb`.
Edward Thomson a1ef995d 2019-02-21T10:33:30 indexer: use git_indexer_progress throughout Update internal usage of `git_transfer_progress` to `git_indexer_progreses`.
Edward Thomson f673e232 2018-12-27T13:47:34 git_error: use new names in internal APIs and usage Move to the `git_error` name in the internal API for error-related functions.
Edward Thomson ed8cfbf0 2019-01-17T00:32:31 references: use new names in internal usage Update internal usage to use the `git_reference` names for constants.
Patrick Steinhardt 0b3dfbf4 2018-08-09T11:13:59 smart_pkt: reorder and rename parameters of `git_pkt_parse_line` The parameters of the `git_pkt_parse_line` function are quite confusing. First, there is no real indicator what the `out` parameter is actually all about, and it's not really clear what the `bufflen` parameter refers to. Reorder and rename the parameters to make this more obvious.
Patrick Steinhardt 0652abaa 2018-07-20T12:56:49 Merge pull request #4702 from tiennou/fix/coverity Assorted Coverity fixes
Patrick Steinhardt fa401a32 2018-07-19T08:20:04 Merge pull request #4704 from nelhage/no-pkt-pack Remove GIT_PKT_PACK entirely
Etienne Samson 6ae6491e 2018-07-06T22:24:16 smart: don't dereference a NULL pkt pointer By clarifying what detect_caps returns on empty/missing packet, we can be sure there are actually refs to process. The old code could blindly dereference `first`, which might have been NULL. Reported by Coverity, CID 1393614
Nelson Elhage a73b7c2f 2018-06-29T16:54:06 This error case is now unneeded
Patrick Steinhardt af3088e4 2018-06-29T11:45:15 refspec: rename `git_refspec__free` to `git_refspec__dispose` Since commit 630a67366 (refspec: add public parsing api, 2018-02-07), we now have two functions `git_refspec_free` and `git_refspec__free`. The difference is that the first one will free the structure itself, while the second one will only free the structure's contents. Use our new `dispose` naming pattern for the latter function to help avoid confusion.
Edward Thomson 12232a5e 2018-06-27T17:19:37 Merge pull request #4698 from nelhage/fix-leaks Fix assorted leaks found via fuzzing
Edward Thomson 9286e413 2018-06-26T09:56:06 smart protocol: correct error message capitalization
Nelson Elhage 3a547417 2018-06-25T15:38:29 git_pkt_free: Allow freeing NULL
Nelson Elhage e31c450b 2018-06-24T23:46:36 Fix another missing git_pkt_free
Nelson Elhage bf4c2c57 2018-06-24T21:56:51 wait_while_ack: use git_pkt_free git__free is insufficient if the packet is a git_pkt_ref or another type that requires freeing referenced structures.
Nelson Elhage 3eec73ae 2018-06-24T20:54:41 PACK packets are illegal while downloading refs
Patrick Steinhardt ecf4f33a 2018-02-08T11:14:48 Convert usage of `git_buf_free` to new `git_buf_dispose`
Etienne Samson 2cf9b84c 2018-04-11T19:13:42 smart: free the pkt when we fail to store it
Etienne Samson 32586d5e 2018-04-11T19:03:57 smart: separate error handling from pkt handling
Etienne Samson 01381149 2018-02-26T21:27:10 smart: make out arguments explicit on recv_pkt
Etienne Samson 08961c9d 2017-08-22T16:29:07 smart: typedef git_pkt_type and clarify recv_pkt return type
Patrick Steinhardt 7cb705cb 2017-10-06T12:05:26 transports: smart: fix memory leak when skipping symbolic refs When we setup the revision walk for negotiating references with a remote, we iterate over all references, ignoring tags and symbolic references. While skipping over symbolic references, we forget to free the looked up reference, resulting in a memory leak when the next iteration simply overwrites the variable. Fix that issue by freeing the reference at the beginning of each iteration and collapsing return paths for error and success.
Patrick Steinhardt 0c7f49dd 2017-06-30T13:39:01 Make sure to always include "common.h" first Next to including several files, our "common.h" header also declares various macros which are then used throughout the project. As such, we have to make sure to always include this file first in all implementation files. Otherwise, we might encounter problems or even silent behavioural differences due to macros or defines not being defined as they should be. So in fact, our header and implementation files should make sure to always include "common.h" first. This commit does so by establishing a common include pattern. Header files inside of "src" will now always include "common.h" as its first other file, separated by a newline from all the other includes to make it stand out as special. There are two cases for the implementation files. If they do have a matching header file, they will always include this one first, leading to "common.h" being transitively included as first file. If they do not have a matching header file, they instead include "common.h" as first file themselves. This fixes the outlined problems and will become our standard practice for header and source files inside of the "src/" from now on.
Roger Gee e141f079 2017-06-10T11:46:09 smart_protocol: fix parsing of server ACK responses Fix ACK parsing in wait_while_ack() internal function. This patch handles the case where multi_ack_detailed mode sends 'ready' ACKs. The existing functionality would bail out too early, thus causing the processing of the ensuing packfile to fail if/when 'ready' ACKs were sent.
Gaurav Saral 61acc9fa 2017-02-08T16:22:44 Changes to provide option to turn off/on ofs_delta This change provides an option in git_libgit2_opt_t which can be used in git_libgit2_opts to turn off/on ofs_delta capability in libGit2
Carlos Martín Nieto a6d833a2 2017-01-13T17:05:58 Merge pull request #4049 from libgit2/ethomson/error_msgs giterr_set: consistent error messages
Patrick Steinhardt 2fdef641 2016-11-15T11:44:51 smart_pkt: treat empty packet lines as error The Git protocol does not specify what should happen in the case of an empty packet line (that is a packet line "0004"). We currently indicate success, but do not return a packet in the case where we hit an empty line. The smart protocol was not prepared to handle such packets in all cases, though, resulting in a `NULL` pointer dereference. Fix the issue by returning an error instead. As such kind of packets is not even specified by upstream, this is the right thing to do.
Edward Thomson 909d5494 2016-12-29T12:25:15 giterr_set: consistent error messages Error messages should be sentence fragments, and therefore: 1. Should not begin with a capital letter, 2. Should not conclude with punctuation, and 3. Should not end a sentence and begin a new one
Patrick Steinhardt 34b32053 2016-11-25T15:02:34 Fix potential use of uninitialized values
Patrick Steinhardt 482d1748 2016-11-25T15:01:35 transports: smart: do not redeclare loop counters
Patrick Steinhardt 62494bf2 2016-11-02T09:38:40 transports: smart: abort receiving packets on end of stream When trying to receive packets from the remote, we loop until either an error distinct to `GIT_EBUFS` occurs or until we successfully parsed the packet. This does not honor the case where we are looping over an already closed socket which has no more data, leaving us in an infinite loop if we got a bogus packet size or if the remote hang up. Fix the issue by returning `GIT_EEOF` when we cannot read data from the socket anymore.
Patrick Steinhardt 61530c49 2016-11-01T16:56:07 transports: smart: abort ref announcement on early end of stream When reading a server's reference announcements via the smart protocol, we expect the server to send multiple flushes before the protocol is finished. If we fail to receive new data from the socket, we will only return an end of stream error if we have not seen any flush yet. This logic is flawed in that we may run into an infinite loop when receiving a server's reference announcement with a bogus flush packet. E.g. assume the last flushing package is changed to not be '0000' but instead any other value. In this case, we will still await one more flush package and ignore the fact that we are not receiving any data from the socket, causing an infinite loop. Fix the issue by always returning `GIT_EEOF` if the socket indicates an end of stream.
Patrick Steinhardt 7d02019a 2016-06-06T12:59:17 transports: smart: fix potential invalid memory dereferences When we receive a packet of exactly four bytes encoding its length as those four bytes it can be treated as an empty line. While it is not really specified how those empty lines should be treated, we currently ignore them and do not return an error when trying to parse it but simply advance the data pointer. Callers invoking `git_pkt_parse_line` are currently not prepared to handle this case as they do not explicitly check this case. While they could always reset the passed out-pointer to `NULL` before calling `git_pkt_parse_line` and determine if the pointer has been set afterwards, it makes more sense to update `git_pkt_parse_line` to set the out-pointer to `NULL` itself when it encounters such an empty packet. Like this it is guaranteed that there will be no invalid memory references to free'd pointers. As such, the issue has been fixed such that `git_pkt_parse_line` always sets the packet out pointer to `NULL` when an empty packet has been received and callers check for this condition, skipping such packets.
Carlos Martín Nieto d4763c98 2016-03-24T06:56:25 Merge pull request #3574 from chescock/buffer-sideband-pack-data Buffer sideband packet data
Chris Hescock 9028a8a2 2016-03-08T10:16:37 Only buffer if necessary.
Patrick Steinhardt 704554cd 2016-02-15T11:37:48 transports: smart: fix memory leak on OOM path
Chris Hescock a7d9d93d 2016-01-11T17:09:32 Buffer sideband packet data The inner packet may be split across multiple sideband packets.
Dan Leehr b0b2c722 2015-08-13T22:52:52 Fix bug in git_smart__push: push_transfer_progress cb is never called The conditional checked cbs->transfer_progress then used the value in cbs->push_transfer_progress. In both cases it should be push_transfer_progress
Carlos Martín Nieto 1396c381 2015-05-18T16:04:55 errors: add GIT_EEOF to indicate early EOF This can be used by tools to show mesages about failing to communicate with the server. The error message in this case will often contain the server's error message, as far as it managed to send anything.
Marius Ungureanu 7e9a240e 2015-05-14T21:33:55 Make "Early EOF" message start with lowercase
Carlos Martín Nieto 8f0104ec 2015-04-21T22:10:36 Remove the callbacks struct from the remote Having the setting be different from calling its actions was not a great idea and made for the sake of the wrong convenience. Instead of that, accept either fetch options, push options or the callbacks when dealing with the remote. The fetch options are currently only the callbacks, but more options will be moved from setters and getters on the remote to the options. This does mean passing the same struct along the different functions but the typical use-case will only call git_remote_fetch() or git_remote_push() and so won't notice much difference.
Carlos Martín Nieto 05259114 2015-04-21T20:16:48 push: remove own copy of callbacks The push object knows which remote it's associated with, and therefore does not need to keep its own copy of the callbacks stored in the remote. Remove the copy and simply access the callbacks struct within the remote.
Edward Thomson 392702ee 2015-02-09T23:41:13 allocations: test for overflow of requested size Introduce some helper macros to test integer overflow from arithmetic and set error message appropriately.
Jacques Germishuys 6f73e026 2014-12-24T11:42:50 Plug some leaks
Carlos Martín Nieto aad638f3 2014-11-07T15:00:11 push: use the common refspec parser There is one well-known and well-tested parser which we should use, instead of implementing parsing a second time. The common parser is also augmented to copy the LHS into the RHS if the latter is empty. The expressions test had to change a bit, as we now catch a bad RHS of a refspec locally.
Ciro Santilli 3b2cb2c9 2014-09-16T11:49:25 Factor 40 and 41 constants from source.
Arkady Shapkin 5cd81bb3 2014-09-03T01:01:25 Several CppCat warnings fixed
Philip Kelley bc8a0886 2014-06-27T11:51:35 Fix assert when receiving uncommon sideband packet
Carlos Martín Nieto 4c4408c3 2014-05-22T12:28:39 Plug leaks and fix a C99-ism We have too many places where we repeat free code, so when adding the new free to the generic code, it didn't take for the local transport. While there, fix a C99-ism that sneaked through.
Carlos Martín Nieto 8156835d 2014-05-20T09:29:39 smart: store reported symrefs The protocol has a capability which allows the server to tell us which refs are symrefs, so we can e.g. know which is the default branch. This capability is different from the ones we already support, as it's not setting a flag to true, but requires us to store a list of refspec-formatted mappings. This commit does not yet expose the information in the reference listing.
Jacques Germishuys 8b686b31 2014-04-21T15:25:19 Correct argument order of git__calloc()
Jacques Germishuys 48e60ae7 2014-04-21T11:23:29 Don't redefine the same callback types, their signatures may change
Jacques Germishuys 4f62163e 2014-04-20T22:06:05 Check the return codes of remote callbacks. The user may have requested that the operation be cancelled.
Jacques Germishuys 9effa2fb 2014-04-20T19:19:13 Fire progress callbacks also for pushes. It's not very useful to only know that a pre-receive hook has declined a push, you probably want to know why.
Miha 300f4412 2014-02-25T11:56:11 - BUGFIX #2133 (@fourplusone) in smart_protocol.c - added MSVC cmake definitions to disable warnings - general.c is rewritten so it is ansi-c compatible and compiles ok on microsoft windows - some MSVC reported warning fixes
Russell Belfer 25e0b157 2013-12-06T15:07:57 Remove converting user error to GIT_EUSER This changes the behavior of callbacks so that the callback error code is not converted into GIT_EUSER and instead we propagate the return value through to the caller. Instead of using the giterr_capture and giterr_restore functions, we now rely on all functions to pass back the return value from a callback. To avoid having a return value with no error message, the user can call the public giterr_set_str or some such function to set an error message. There is a new helper 'giterr_set_callback' that functions can invoke after making a callback which ensures that some error message was set in case the callback did not set one. In places where the sign of the callback return value is meaningful (e.g. positive to skip, negative to abort), only the negative values are returned back to the caller, obviously, since the other values allow for continuing the loop. The hardest parts of this were in the checkout code where positive return values were overloaded as meaningful values for checkout. I fixed this by adding an output parameter to many of the internal checkout functions and removing the overload. This added some code, but it is probably a better implementation. There is some funkiness in the network code where user provided callbacks could be returning a positive or a negative value and we want to rely on that to cancel the loop. There are still a couple places where an user error might get turned into GIT_EUSER there, I think, though none exercised by the tests.
Jameson Miller db4cbfe5 2013-12-02T14:09:12 Updates to cancellation logic during download and indexing of packfile.
Edward Thomson 80fc7d6b 2013-11-13T16:46:45 Propagate auth error codes as GIT_EUSER in winhttp
Vicent Martí 6414fd33 2013-11-11T06:47:15 Merge pull request #1956 from libgit2/cmn/fetch-default-head Remote revamp (director's cut)
Carlos Martín Nieto a6192d7c 2013-11-11T15:32:13 remote: update head list on push A previous commit forgot to update the head list after push as well, leading to wrong output of git_remote_ls().
Carlos Martín Nieto 359dce72 2013-11-02T00:05:32 remote: make _ls return the list directly The callback-based method of listing remote references dates back to the beginning of the network code's lifetime, when we didn't know any better. We need to keep the list around for update_tips() after disconnect() so let's make use of this to simply give the user a pointer to the array so they can write straightforward code instead of having to go through a callback.
Edward Thomson 7616b8d3 2013-11-05T17:34:45 don't double free pkt
Carlos Martín Nieto af613ecd 2013-10-28T23:30:45 remote: store dwimed refspecs separately This allows us to add e.g. "HEAD" as a refspec when none are given without overwriting the user's data.
Carlos Martín Nieto a7382aa2 2013-10-26T16:52:54 remote: give up after 256 failures to find a common object This avoids sending our whole history bit by bit to the remote in cases where there is no common history, just to give up in the end. The number comes from the canonical implementation.
Vicent Martí ac5e507c 2013-11-01T09:31:52 Merge pull request #1918 from libgit2/cmn/indexer-naming indexer: remove the stream infix
Carlos Martín Nieto 2f8c481c 2013-10-08T16:22:21 protocol: basic support for multi_ack_detailed This tells the server that we speak it, but we don't make use of its extra information to determine if there's a better place to stop negotiating. In a somewhat-related change, reorder the capabilities so we ask for them in the same order as git does. Also take this opportunity to factor out a fairly-indented portion of the negotiation logic.
Carlos Martín Nieto a6154f21 2013-10-30T15:00:05 indexer: remove the stream infix It was there to keep it apart from the one which read in from a file on disk. This other indexer does not exist anymore, so there is no need for anything other than git_indexer to refer to it. While here, rename _add() function to _append() and _finalize() to _commit(). The former change is cosmetic, while the latter avoids talking about "finalizing", which OO languages use to mean something completely different.
Vicent Martí 5c50f22a 2013-10-28T09:25:44 Merge pull request #1891 from libgit2/cmn/fix-thin-packs Add support for thin packs
Carlos Martín Nieto 5cb13670 2013-10-23T15:45:29 transport: let the progress output return an error There are any number of issues that can come up in the progress callback, and we should let the user cancel at that point as well.
Carlos Martín Nieto b4342b11 2013-10-04T10:27:45 net: advertise our support for fixing thin packs
Jameson Miller 7baa7631 2013-10-02T15:39:18 Style tweaks and changes for code review feedback
Jameson Miller 5b188225 2013-10-02T13:45:32 Support cancellation in push operation This commit adds cancellation for the push operation. This work consists of: 1) Support cancellation during push operation - During object counting phase - During network transfer phase - Propagate GIT_EUSER error code out to caller 2) Improve cancellation support during fetch - Handle cancellation request during network transfer phase - Clear error string when cancelled during indexing 3) Fix error handling in git_smart__download_pack Cancellation during push is still only handled in the pack building and network transfer stages of push (and not during packbuilding).
Jameson Miller b176eded 2013-09-19T14:52:57 Initial Implementation of progress reports during push This adds the basics of progress reporting during push. While progress for all aspects of a push operation are not reported with this change, it lays the foundation to add these later. Push progress reporting can be improved in the future - and consumers of the API should just get more accurate information at that point. The main areas where this is lacking are: 1) packbuilding progress: does not report progress during deltafication, as this involves coordinating progress from multiple threads. 2) network progress: reports progress as objects and bytes are going to be written to the subtransport (instead of as client gets confirmation that they have been received by the server) and leaves out some of the bytes that are transfered as part of the push protocol. Basically, this reports the pack bytes that are written to the subtransport. It does not report the bytes sent on the wire that are received by the server. This should be a good estimate of progress (and an improvement over no progress).
Russell Belfer 33c8c6f0 2013-07-10T10:48:32 trivial whitespace fixup
Linquize 7026ad89 2013-05-16T21:08:55 calloc() to initialize memory
Linquize e583334c 2013-05-10T21:42:22 Fix broken build when MSVC SDL checks is enabled
Carlos Martín Nieto 2b562c3a 2013-05-04T16:32:58 refs: remove the OID/SYMBOLIC filtering Nobody should ever be using anything other than ALL at this level, so remove the option altogether. As part of this, git_reference_foreach_glob is now implemented in the frontend using an iterator. Backends will later regain the ability of doing the glob filtering in the backend.
Edward Thomson d8041638 2013-05-02T17:22:13 fix some leaks
Carlos Martín Nieto 51e4da6d 2013-04-29T01:49:40 push: don't send a packfile when only issuing delete commands For update and create commands where all the objects are known to exist in the remote, we must send an empty packfile. However, if all we issue are delete commands, no packfile must be sent. Take this into consideration for push.
Russell Belfer 83cc70d9 2013-04-19T12:48:33 Move odb_backend implementors stuff into git2/sys This moves some of the odb_backend stuff that is related to the internals of an odb_backend implementation into include/git2/sys. Some of the stuff related to streaming I left in include/git2 because it seemed like it would be reasonably needed by a normal user who wanted to stream objects into and out of the ODB. Also, I added APIs for traversing the list of backends so that some of the tests would not need to access ODB internals.
Arkadiy Shapkin 10c06114 2013-03-17T04:46:46 Several warnings detected by static code analyzer fixed Implicit type conversion argument of function to size_t type Suspicious sequence of types castings: size_t -> int -> size_t Consider reviewing the expression of the 'A = B == C' kind. The expression is calculated as following: 'A = (B == C)' Unsigned type is never < 0
Philip Kelley f5898324 2013-03-12T15:31:14 Style: Reverse lhs and rhs of == comparisons
Philip Kelley b8c32580 2013-03-12T15:19:32 Advertise and support side-band-64k when calling receive-pack
Ben Straub a9e1339c 2013-02-14T08:12:05 Fix a leak when canceling a network operation
Russell Belfer 9c258af0 2013-02-12T10:13:56 Merge pull request #1316 from ben/clone-cancel Allow network operations to cancel
Philip Kelley a150cc87 2013-02-10T18:16:10 Fix a bug introduced in df93a681 'Merge the push...'
Michael Schubert a9d081e5 2013-02-10T19:36:39 Fix -Wmaybe-uninitialized warning
Philip Kelley df93a681 2013-02-08T15:00:08 Merge the push report into the refs to avoid a 3rd network call
Ben Straub ea57f66b 2013-02-06T11:02:29 Expect standard error code from internal calls
Ben Straub def60ea4 2013-02-05T13:14:48 Allow all non-zero returns to cancel transfers