src/transports/httpclient.c


Log

Author Commit Date CI Message
Edward Thomson 4e8840fd 2021-08-30T18:20:35 Merge pull request #6022 from lollipopman/connect-proxy-host-header Set Host Header to match CONNECT authority target
Jesse Hathaway fc5d0e80 2021-08-30T21:24:54 Set Host Header to match CONNECT authority target Prior to this change, for CONNECT requests, the Host header was set to the host and port of the target http proxy. However, per the rfc7230 for HTTP/1.1 this is incorrect as the Host header should match the target of the CONNECT request, as detailed in section 5.3.3 & 5.4. 5.3.3. authority-form The authority-form of request-target is only used for CONNECT requests (Section 4.3.6 of [RFC7231]). authority-form = authority When making a CONNECT request to establish a tunnel through one or more proxies, a client MUST send only the target URI's authority component (excluding any userinfo and its "@" delimiter) as the request-target. For example, CONNECT www.example.com:80 HTTP/1.1 5.4. Host <snip> A client MUST send a Host header field in all HTTP/1.1 request messages. If the target URI includes an authority component, then a client MUST send a field-value for Host that is identical to that authority component, excluding any userinfo subcomponent and its "@" delimiter (Section 2.7.1). If the authority component is missing or undefined for the target URI, then a client MUST send a Host header field with an empty field-value. This issue was noticed when proxying requests through HAProxy 2.2 which rejects these invalid http requests.
Edward Thomson 5eb2b0b3 2021-08-30T08:27:27 httpclient: actually return `GIT_EAUTH`
Edward Thomson 9937967e 2021-08-29T21:29:14 Merge branch 'main' into http-use-eauth
Tyler Ang-Wanek 3473a088 2021-05-12T11:48:23 httpclient: no proxy creds in requests if proxy is CONNECT type
Tyler Ang-Wanek 049618ce 2021-04-30T15:11:54 httpclient: git_http_client_skip_body should drain socket of body
Edward Thomson 2807de5c 2020-12-17T15:18:31 http: handle ipv6 addresses
Edward Thomson 4f5f1127 2020-11-22T00:01:09 transports: use GIT_ASSERT
Edward Thomson e316b0d3 2020-05-15T11:47:09 runtime: move init/shutdown into the "runtime" Provide a mechanism for system components to register for initialization and shutdown of the libgit2 runtime.
Edward Thomson 2ab99c6d 2020-10-04T18:30:10 Merge pull request #5576 from lollipopman/double-auth httpclient: only free challenges for current_server type
Sven Strickroth 2dea3eb4 2020-09-08T13:03:07 Don't fail if a HTTP server announces he supports a protocol upgrade cf. RFC7230 section 6.7, an Upgrade header in a normal response merely informs the client that the server supports upgrading to other protocols, and the client can ask for such an upgrade in a later request. The server requiring an upgrade is via the 426 Upgrade Required response code, not the mere presence of the Upgrade response header. (closes issue #5573) Signed-off-by: Sven Strickroth <email@cs-ware.de>
Jesse Hathaway bd346313 2020-07-10T15:37:08 httpclient: only free challenges for current_server type Prior to this commit we freed both the server and proxy auth challenges in git_http_client_read_response. This works when the proxy needs auth or when the server needs auth, but it does not work when both the proxy and the server need auth as we erroneously remove the server auth challenge before we have added them as server credentials. Instead only remove the challenges for the current_server type. Co-authored-by: Stephen Gelman <ssgelm@gmail.com>
Patrick Steinhardt a6c9e0b3 2020-06-08T12:40:47 tree-wide: mark local functions as static We've accumulated quite some functions which are never used outside of their respective code unit, but which are lacking the `static` keyword. Add it to reduce their linkage scope and allow the compiler to optimize better.
Edward Thomson 04c7bdb4 2020-06-01T22:44:14 httpclient: clear the read_buf on new requests The httpclient implementation keeps a `read_buf` that holds the data in the body of the response after the headers have been written. We store that data for subsequent calls to `git_http_client_read_body`. If we want to stop reading body data and send another request, we need to clear that cached data. Clear the cached body data on new requests, just like we read any outstanding data from the socket.
Edward Thomson aa8b2c0f 2020-06-01T23:53:55 httpclient: don't read more than the client wants When `git_http_client_read_body` is invoked, it provides the size of the buffer that can be read into. This will be set as the parser context's `output_size` member. Use this as an upper limit on our reads, and ensure that we do not read more than the client requests.
Edward Thomson 570f0340 2020-06-01T19:10:38 httpclient: read_body should return 0 at EOF When users call `git_http_client_read_body`, it should return 0 at the end of a message. When the `on_message_complete` callback is called, this will set `client->state` to `DONE`. In our read loop, we look for this condition and exit. Without this, when there is no data left except the end of message chunk (`0\r\n`) in the http stream, we would block by reading the three bytes off the stream but not making progress in any `on_body` callbacks. Listening to the `on_message_complete` callback allows us to stop trying to read from the socket when we've read the end of message chunk.
Utkarsh Gupta e7a1fd88 2020-03-26T11:42:47 Fix spelling error Signed-off-by: Utkarsh Gupta <utkarsh@debian.org>
Edward Thomson 502e5d51 2020-03-01T12:44:39 httpclient: use a 16kb read buffer for macOS Use a 16kb read buffer for compatibility with macOS SecureTransport. SecureTransport `SSLRead` has the following behavior: 1. It will return _at most_ one TLS packet's worth of data, and 2. It will try to give you as much data as you asked for This means that if you call `SSLRead` with a buffer size that is smaller than what _it_ reads (in other words, the maximum size of a TLS packet), then it will buffer that data for subsequent calls. However, it will also attempt to give you as much data as you requested in your SSLRead call. This means that it will guarantee a network read in the event that it has buffered data. Consider our 8kb buffer and a server sending us 12kb of data on an HTTP Keep-Alive session. Our first `SSLRead` will read the TLS packet off the network. It will return us the 8kb that we requested and buffer the remaining 4kb. Our second `SSLRead` call will see the 4kb that's buffered and decide that it could give us an additional 4kb. So it will do a network read. But there's nothing left to read; that was the end of the data. The HTTP server is waiting for us to provide a new request. The server will eventually time out, our `read` system call will return, `SSLRead` can return back to us and we can make progress. While technically correct, this is wildly ineffecient. (Thanks, Tim Apple!) Moving us to use an internal buffer that is the maximum size of a TLS packet (16kb) ensures that `SSLRead` will never buffer and it will always return everything that it read (albeit decrypted).
Josh Bleecher Snyder 216165ec 2020-02-07T10:06:28 transports: use GIT_EAUTH for authentication failures When the failure is clearly an auth failure (as opposed to possibly an auth failure), use the error code GIT_EAUTH instead of GIT_ERROR. While we're here, fix a typo and improve an error message. Fixes #5389.
Edward Thomson 3f54ba8b 2020-01-18T13:51:40 credential: change git_cred to git_credential We avoid abbreviations where possible; rename git_cred to git_credential. In addition, we have standardized on a trailing `_t` for enum types, instead of using "type" in the name. So `git_credtype_t` has become `git_credential_t` and its members have become `GIT_CREDENTIAL` instead of `GIT_CREDTYPE`. Finally, the source and header files have been renamed to `credential` instead of `cred`. Keep previous name and values as deprecated, and include the new header files from the previous ones.
Edward Thomson e9cef7c4 2020-01-11T23:53:45 http: introduce GIT_ERROR_HTTP Disambiguate between general network problems and HTTP problems in error codes.
Edward Thomson 29762e40 2020-01-01T16:14:37 httpclient: use defines for status codes
Edward Thomson 7372573b 2019-10-25T12:22:10 httpclient: support expect/continue Allow users to opt-in to expect/continue handling when sending a POST and we're authenticated with a "connection-based" authentication mechanism like NTLM or Negotiate. If the response is a 100, return to the caller (to allow them to post their body). If the response is *not* a 100, buffer the response for the caller. HTTP expect/continue is generally safe, but some legacy servers have not implemented it correctly. Require it to be opt-in.
Edward Thomson 6c21c989 2019-12-14T21:32:07 httpclient: support CONNECT proxies Fully support HTTP proxies, in particular CONNECT proxies, that allow us to speak TLS through a proxy.
Edward Thomson 6b208836 2019-12-18T21:55:28 httpclient: handle chunked responses Detect responses that are sent with Transfer-Encoding: chunked, and record that information so that we can consume the entire message body.
Edward Thomson 6a095679 2019-12-14T10:34:36 httpclient: support authentication Store the last-seen credential challenges (eg, all the 'WWW-Authenticate' headers in a response message). Given some credentials, find the best (first) challenge whose mechanism supports these credentials. (eg, 'Basic' supports username/password credentials, 'Negotiate' supports default credentials). Set up an authentication context for this mechanism and these credentials. Continue exchanging challenge/responses until we're authenticated.
Edward Thomson 1152f361 2019-12-13T18:37:19 httpclient: consume final chunk message When sending a new request, ensure that we got the entirety of the response body. Our caller may have decided that they were done reading. If we were not at the end of the message, this means that we need to tear down the connection and cannot do keep-alive. However, if the caller read all of the message, but we still have a final end-of-response chunk signifier (ie, "0\r\n\r\n") on the socket, then we should consider that the response was successfully copmleted. If we're asked to send a new request, try to read from the socket, just to clear out that end-of-chunk message, marking ourselves as disconnected on any errors.
Edward Thomson 84b99a95 2019-12-12T13:53:43 httpclient: add chunk support to POST Teach httpclient how to support chunking when POSTing request bodies.
Edward Thomson eacecebd 2019-12-12T13:25:32 httpclient: introduce a simple http implementation Introduce a new http client implementation that can GET and POST to remote URLs. Consumers can use `git_http_client_init` to create a new client, `git_http_client_send_request` to send a request to the remote server and `git_http_client_read_response` to read the response. The http client implementation will perform the I/O with the remote server (http or https) but does not understand the git smart transfer protocol. This allows us to split the concerns of the http subtransport from the actual http implementation.