buffer.c


Log

Author Commit Date CI Message
Nick Mathewson 6a4ec5c2 2012-07-26T10:34:06 Avoid possible needless call to writev. Found by coverity.
Nick Mathewson e49e2891 2012-02-10T17:29:53 Update copyright notices to 2012
Zack Weinberg c986f232 2011-12-08T14:30:20 Fix behavior of evbuffer_peek(buf,-1,NULL,NULL,0) (Patch altered by nickm to not affect the behavior of evbuffer_peek(buf,-1,NULL,vec,n_vec).)
Nick Mathewson c37069cd 2011-11-02T16:09:15 Fix an evbuffer crash in evbuffer_remove_buffer() Found by Greg Hazel.
Nick Mathewson 3c824bd3 2011-10-24T13:18:09 Update copyright dates to 2011.
Nick Mathewson 0ba0af9c 2011-09-29T09:30:04 Prefer mmap to sendfile unless a DRAINS_TO_FD flag is set. Allows add_file to work with SSL. The sendfile() implementation for evbuffer_add_file is potentially more efficient, but it has a problem: you can only use it to send bytes over a socket using sendfile(). If you are writing bytes via SSL_send() or via a filter, or if you need to be able to inspect your buffer, it doesn't work. As an easy fix, this patch disables the sendfile-based implementation of evbuffer_add_file on an evbuffer unless the user sets a new EVBUFFER_FLAG_DRAINS_TO_FD flag on that evbuffer, indicating that the evbuffer will not be inspected, but only written out via evbuffer_write(), evbuffer_write_atmost(), or drained with stuff like evbuffer_drain() or evbuffer_add_buffer(). This flag is off by default, except for evbuffers used for output on bufferevent_socket. In the future, it could be interesting to make a best-effort file segment implementation that tries to send via sendfile, but mmaps on demand. That's too much complexity for a stable release series, though.
Nick Mathewson 1ef1f684 2011-09-28T09:22:17 Make evbuffer callbacks get the right n_added value after evbuffer_add Patch from Alex.
Nick Mathewson 6acfbdd8 2011-08-18T12:35:27 Make overlapped reads result in evbuffer callbacks getting invoked
Michael Herf 643922e9 2011-08-15T13:39:10 Solaris sendfile: correctly detect amount of data sent Original message: Solaris sendfile seems to fail when sending moderately large (<1GB) files. Not a 32/64 problem, but a buffer problem. Anyone else ever try this? It is definitely broken in http-server.c. It seems to be broken in the following way: When sendfile sends partial data (EAGAIN, would block), "res" is always -1, rather than the amount sent. Here's a patch that reads from the "offset" pointer instead to discover what was sent. This seems to work:
Mark Ellzey f87f5689 2011-07-04T11:47:24 Speed up invoke_callbacks on evbuffers when there are no callbacks This fixes a performance regression against 1.4
Nick Mathewson 4461f1a0 2011-06-06T15:11:28 Fix incorrect results from evbuffer_search_eol(EOL_LF) Our evbuffer_strchr() function [which was only used for search_eol(EOL_LF) could give incorrect results if it found its answer in the first chunk but didn't start searching from the front of the chunk. Also, this patch adds unit tests for evbuffer_search_eol, particularly in those cases that evbuffer_readln() tests didn't exercise.
Evan Jones b63ab177 2010-12-06T14:17:44 EVUTIL_ASSERT: Use sizeof() to avoid "unused variable" warnings.
Nick Mathewson bb0d2b4e 2010-12-09T11:47:54 Consistentize tabs
Nick Mathewson 7bcace2d 2010-11-22T21:02:34 Fix some irix compilation warnings spotted by Kevin Bowling
Nick Mathewson a3245afe 2010-11-01T14:23:33 Fix win32 build in response to fixes from win64 build.
Nick Mathewson 545a6114 2010-11-01T13:59:04 Fix even more win64 warnings: buffer, event_tagging, http, evdns, evrpc
Nick Mathewson e4f34e8a 2010-10-25T22:36:23 Correct logic for realigning a chain in evbuffer_add The old logic was both too eager to realign (it would move a whole chain to save a byte) and too reluctant to realign (it would only realign when data would fit into the misaligned portion, without considering the space at the end of the chain). The new logic matches that from evbuffer_expand_singlechain: it only realigns a chain when not much data is to be moved, and there's a bunch of space to be regained. Spotted by Yan Lin.
Nick Mathewson ac7e52d8 2010-10-25T14:29:30 Make evbuffer_add_file take ev_off_t, not off_t This change has no effect on non-windows platforms, since those either define off_t to 64-bits, or allow you to decide whether it should be 64-bits yourself via some LARGEFILE-like macro. On Windows, however, off_t is always 32-bit, so it's a bad choice for "file size" or "file offset" values. Instead, I'm adding an ev_off_t type, and using it in the one place where we used off_t to mean "the size of a file" or "an offset into a file" in the API. This breaks ABI compatibility on Windows.
Nick Mathewson 6be589ae 2010-10-14T13:48:40 Fix signed/unsigned warnings on opensolaris, where iov_len is signed
Nick Mathewson fdc640b0 2010-10-05T21:34:07 Fix an EINVAL on evbuffer_write_iovec on OpenSolaris. The writev() call is limited to at most IOV_MAX iovecs (or UIO_MAXIOV, depending on whom you ask). This isn't a problem anywhere we've tested except on OpenSolaris, where IOV_MAX was a mere 16. This patch makes us go from "use up to 128 iovecs when writing" to "use up to 128 iovecs when writing, or IOV_MAX/UIO_MAXIOV, whichever is less". This is still wrong if you somehow find a platform that defines IOV_MAX < UIO_MAXIOV, but I hereby claim that such a platform is too stupid to worry about for now. Found by Michael Herf.
Nick Mathewson 9c8db0f8 2010-09-23T22:45:55 Fix all warnings in the main codebase flagged by -Wsigned-compare Remember, the code int is_less_than(int a, unsigned b) { return a < b; } is buggy, since the C integer promotion rules basically turn it into int is_less_than(int a, unsigned b) { return ((unsigned)a) < b; } and we really want something closer to int is_less_than(int a, unsigned b) { return a < 0 || ((unsigned)a) < b; } . Suggested by an example from Ralph Castain
Christopher Davis 03afa209 2010-08-16T01:23:57 IOCP-related evbuffer fixes. - Prevent evbuffer_{add,prepend}_buffer from moving read-pinned chains. - Fix evbuffer_drain to handle read-pinned chains better. - Raise the limit on WSABUFs from two to MAX_WSABUFS for overlapped reads.
Nick Mathewson 743f8665 2010-08-23T11:48:46 Honor NDEBUG; build without warnings with NDEBUG; make NDEBUG always-off in unit test code
Nick Mathewson ec347b92 2010-07-07T16:45:03 Move event-config.h to include/event2 This change means that all required include files are in event2, and all files not in event2/* are optional.
niks 65abdc20 2010-07-16T09:11:09 Fix wrong sie calculation of iovec buffers when exact=1 The old code had a bug where the 'exact' flag to 1 in _evbuffer_read_setup_vecs would never actually make the iov_len field of the last iovec get truncated. This patch fixes that.
Nick Mathewson 3467f2fa 2010-05-28T15:05:32 Fix logic in correcting high values from FIONREAD The old logic made sense back when buffer.c was an enormous linear buffer, but it doesn't make any sense for the chain-based implementation. This patch also refactors the ioctl{socket}? call into its own function.
Nick Mathewson c44de06c 2010-05-08T18:09:27 Numerous opensolaris compilation fixes For future note, opensolaris doesn't have sys/sysctl.h, doesn't like comparing iov_buf to a chain_space_ptr without a cast, and is (predictably) unforgiving of dumb syntax errors. Also, we had accidentally broken the devpoll backend test in configure.in
Nick Mathewson dcdae6b7 2010-05-08T16:34:18 Make evbuffer_add_file() work on windows Right now only the add_file() mode is supported, when it would be nicer to have mmap support. Perhaps for Libevent 2.1.x.
Nick Mathewson d49b92a8 2010-04-23T23:04:20 Remove one last bug in last_with_datap logic. Found with valgrind
Giuseppe Scrivano d469c503 2010-04-12T12:18:57 Fix compiler warnings under WIN32
Nick Mathewson eb86c8c5 2010-04-12T22:24:54 Add evbuffer_copyout to copy data from an evbuffer without draining The evbuffer_remove() function copies data from the front of an evbuffer into an array of char, and removes the data from the buffer. This function behaves the same, but does not remove the data. This behavior can be handy for lots of protocols, where you want the evbuffer to accumulate data until a complete record has arrived. Lots of people have asked for a function more or less like this, and though it isn't too hard to code one from evbuffer_peek(), it is apparently annoying to do it in every app you write. The evbuffer_peek() function is significantly faster, but it requires that the user be able to handle data in separate extents. This patch also reimplements evbufer_remove() as evbuffer_copyout() followed by evbuffer_drain(). I am reasonably confident that this won't be a performance hit: the memcpy() overhead should dominate the cost of walking the list an extra time.
Nick Mathewson 8c83e995 2010-04-09T16:40:53 Add more unit tests for evbuffer_expand
Nick Mathewson 06a4443a 2010-04-09T15:28:26 Unit-test every evbuffer_add_file() implementation. Previously, we'd only test the default one, even if the others were still compiled in.
Christopher Davis 28bfed47 2010-04-02T19:08:32 Clean up a mistake in pointer manipulation in evbuffer_remove
Nick Mathewson d5ebcf37 2010-03-30T16:47:37 Rewrite evbuffer_expand and its users The previous evbuffer_expand was not only incorrect; it was inefficient too. On all questions of time vs memory tradeoffs, it chose to burn time in order to avoid wasting memory. The new code tries to be a little more balanced: it only resizes an existing chain when doing so doesn't require too much copying, and when failing to do so would waste a lot of the chain's space. This patch also rewrites evbuffer_chain_insert to work properly with last_with_datap, and adds a few convenience functions to buffer.c.
Nick Mathewson 45068a31 2010-03-31T12:03:43 Fix a memory leak when appending/prepending to a buffer with unused space.
Nick Mathewson 8e227b04 2010-03-27T00:09:25 Make the no_iovecs case of write_atmost compile Apparently nobody had tested it before on a system that had sendfile. Why would you have sendfile and not writev? Perhaps you're trying to test the no-iovecs code to make sure it still works.
Nick Mathewson 96865c47 2010-03-30T12:48:56 Turn the increasingly complex *_CHAIN() macros into functions
Nick Mathewson b7442f8e 2010-03-26T23:18:40 Replace last_with_data with a slightly smarter version To implement evbuffer_expand() properly, you need to be able to replace the last chunk that has data, which means that we need to keep track of the the next pointer pointing to the last_with_data chunk, not the last_with_data chunk itself.
Nick Mathewson cda56abf 2010-03-31T12:29:26 Fix critical bug in evbuffer_write when writev is not available evbuffer_pullup() returns NULL if you try to pull up more bytes than are there. But evbuffer_write_atmost would sometimes ask for more bytes to be pulled up than it had, get a NULL, and fail.
Nick Mathewson c87272b7 2010-03-26T14:51:39 Make evbuffer_prepend handle empty buffers better If the first chunk of a buffer is empty, and we're told to prepend to the buffer, we should be willing to use the entire first chunk. Instead, we were dependent on the value of chunk->misalign.
Nick Mathewson 5c0ebb33 2010-03-26T14:50:45 Do not use evbuffer_expand() to add the first chain to a buffer (It's a big function, and using it this way is overkill.)
Nick Mathewson f1bc125e 2010-03-12T23:00:49 Improve robustness for refcounting Document that we do intend to double-decref underlying bufferevents under some circumstances. Check to make sure that we don't decref past 0.
Nick Mathewson 1e7b9868 2010-03-11T14:23:02 Fix last_with_data compilation on windows
Nick Mathewson e470ad3c 2010-03-10T23:39:30 Allow evbuffer_read() to split across more than 2 iovecs Previously it would only accept 2 iovecs at most, because our previous_to_last nonsense didn't let it take any more. This forced us to do more reallocations in some cases when an extra small malloc would have sufficed.
Nick Mathewson 6f47bd12 2010-03-10T23:28:51 Remove previous_to_last from evbuffer
Nick Mathewson c8ac57f1 2010-03-10T23:24:14 Use last_with_data in place of previous_to_last This actually makes some of the code a lot simpler. The only ones that actually used previous_to_last for anything were reserving and committing space.
Nick Mathewson 2a6d2a1e 2010-03-10T22:16:14 Revise evbuffer to add last_with_data This is the first patch in a series to replace previous_to_last with last_with_data. Currently, we can only use two partially empty chains at the end of an evbuffer, so if we have one with 511 bytes free, and another with 512 bytes free, and we try to do a 1024 byte read, we can't just stick another chain on the end: we need to reallocate the last one. That's stupid and inefficient. Instead, this patch adds a last_with_data pointer to eventually replace previous_to_last. Instead of pointing to the penultimated chain (if any) as previous_to_last does, last_with_data points to the last chain that has any data in it, if any. If all chains are empty, last_with_data points to the first chain. If there are no chains, last_with_data is NULL. The next step is to start using last_with_data everywhere that we currently use previous_to_last. When that's done, we can remove previous_to_last and the code that maintains it.
Nick Mathewson 17efc1cd 2010-03-04T01:25:51 Update all our copyright notices to say "2010"
Nick Mathewson cc1600af 2010-03-02T17:00:06 Improve the speed of evbuffer_readln() This makes some cases of bench_http about 5% faster. Our internal evbuffer_strpbrk() function was overly general (it tried to handle all character sets when we only used it for "\r\n"), and not very efficient (it called memchr once for each character in the buffer until it found a \r or a \n). It actually showed up in some profiles for HTTP testing, since evbuffer_readln() calls it when doing loose CRLF detection. This patch replaces it with a faster implementation.
Nick Mathewson b2fbeb3f 2010-02-22T15:38:23 Make bufferevent_free() clear all callbacks immediately. This should end the family of bugs where we call bufferevent_free() while a pending callback is holding a reference on the bufferevent, and the callback tries to invoke the user callbacks before it releases its own final reference. This means that bufferevent_decref() is now a separate function from bufferevent_free().
Nick Mathewson e5bbd40a 2010-02-18T17:41:15 Clean up formatting: use tabs, not 8-spaces, to indent.
Nick Mathewson 7116bf23 2010-02-15T21:03:52 Fix two unlocked reads in evbuffer. Some initializers (in evbuffer_read and evbuffer_commit) were reading the last and/or previous_to_last fields without grabbing the evbuffer lock. This may fix a hard-to-trigger race condition or two.
Nick Mathewson 3fe60fdf 2010-02-12T23:40:13 Use off_t for the length parameter of evbuffer_add_file
Nick Mathewson 29151e65 2010-01-06T18:42:59 Fix byte counts when mixing deferred and non-deferred evbuffer callbacks. This patch finishes 390e0561, which was somehow committed in a half-finished state. It solves a failing unit test on windows.
Nick Mathewson a47d88d7 2009-12-23T07:53:19 Replace some cases of uint32_t with ev_uint32_t. Spotted by Roman Puls.
Nick Mathewson 390e0561 2009-12-22T15:52:02 Fix up behavior of never-defered callbacks a little
Nick Mathewson 438f9ed2 2009-11-23T15:53:24 Add the abilitity to mark some buffer callbacks as never-deferred.
Nick Mathewson 76cd2b70 2009-11-27T16:44:47 Stop passing EVTHREAD_READ and EVTHREAD_WRITE to non-rw locks. Previously, our default lock model kind of assumed that every lock was potentially a read-write lock. This was a poor choice, since read-write locks are far more expensive than regular locks, and so the lock API should only use them when we can actually take advantage of them. Neither our pthreads or win32 lock implementation provided rw locks. Now that we have a way (not currently used!) to indicate that we really want a read-write lock, we shouldn't actually say "lock this for reading" or "lock this for writing" unless we mean it.
Nick Mathewson 347952ff 2009-11-27T15:20:43 Revise the locking API: deprecate the old locking callbacks and add trylock. Previously, there was no good way to request different kinds of lock (say, read/write vs writeonly or recursive vs nonrecursive), or for a lock function to signal failure (which would be important for a trylock mode). This patch revises the lock API to be a bit more useful. The older lock calls are still supported for now. We also add a debugging mode to catch common errors in using the locking APIs.
Niels Provos f1691539 2009-11-19T23:08:50 Remove most calls to event_err() in http and deal with memory errors instead svn:r1555
Nick Mathewson d7d1f1da 2009-11-17T20:31:09 Move responsibility for IOCP callback into bufferevent_async. This patch from Chris Davis saves some callback depth, and adds proper ref-counting to bufferevents when there's a deferred evbuffer callback inflight. It could use a couple more comments to really nail down what its invariants are. svn:r1543
Nick Mathewson 784b8773 2009-11-06T21:46:57 We do not work any more without an event-config.h; stop pretending that it is meaningful to check for HAVE_CONFIG_H svn:r1516
Nick Mathewson ac633aeb 2009-11-05T21:22:23 Fix some build warnings on MSVC, mostly related to signed/unsigned comparisons. svn:r1510
Nick Mathewson 7f0ad2f6 2009-11-02T16:17:06 Fix an errant user of ssize_t to use ev_ssize_t. My usual strategy of grep '[^_]ssize_t' had apparently failed me, since this ssize_t was in the first column. Resolves bug 2890434; spotted by Mihai Draghicioiu. svn:r1484
Nick Mathewson 10cf631e 2009-10-27T04:04:07 Do not add a newline to the end of log statements. svn:r1468
Nick Mathewson 2e36dbe1 2009-10-26T20:00:43 Use EVUTIL_ASSERT() consistently instead of assert. svn:r1464
Nick Mathewson e3fd294a 2009-10-16T13:19:57 Spelling fixes in comments and strings. svn:r1445
Niels Provos 8e8d94a3 2009-09-24T22:18:19 Do not drop data from evbuffer when out of memory; reported by Jacek Masiulaniec svn:r1436
Nick Mathewson 3b461a6d 2009-09-11T18:21:37 Treat a negative number of bytes to read as the kernel saying "I don't know." svn:r1426
Nick Mathewson 22bd5b42 2009-08-16T16:40:42 Support sendfile on solaris: patch from Caitlin Mercer. svn:r1419
Nick Mathewson 8a99083f 2009-08-07T17:16:52 Add an evbuffer_search_range() to search a bounded range of a buffer This can be handy when you have one search to find the end of a header section, and then you want to find a substring within the header section without looking at the body. svn:r1410
Nick Mathewson d4134772 2009-07-31T17:35:42 Refactor evbuffer_readln() into a search-for-eol function and an extract-line function. svn:r1404
Nick Mathewson a26d2d1b 2009-07-31T17:34:47 Refactor evbuffer_readln to use evbuffer_ptr; remove old evbuffer_iterator. svn:r1403
Nick Mathewson b06b2649 2009-07-26T01:29:39 Make "deferred callback queue" independent of event_base. This way, we can more easily have an IOCP bufferevent implementation that does not need an event_base at all. Woot. svn:r1381
Nick Mathewson 8eb155a1 2009-07-23T14:48:24 Fix build on platforms (like Solaris 10, reportedly) which lack a MAP_FILE. svn:r1375
Nick Mathewson 5aefb8a6 2009-06-25T15:22:36 Fix type on freebsd sendfile. Patch from navin seshadri. Fixes bug 2811991 svn:r1330
Nick Mathewson 0b22ca19 2009-05-22T19:11:48 Use ev_ssize_t in place of ssize_t *everywhere*. svn:r1309
Nick Mathewson 8997f234 2009-05-21T20:59:00 Use the native "struct iovec" as our "struct evbuffer_iovec" when available, so we do not need to copy more pointers than necessary. svn:r1299
Nick Mathewson 23243b8a 2009-05-19T21:39:35 Replace reserve/commit with new iovec-based interface. Add a new evbuffer_peek. svn:r1296
Nick Mathewson dc4c7b95 2009-05-15T22:44:18 Change the interface of evbuffer_add_reference: give the cleanup function more info. svn:r1294
Nick Mathewson bba69e03 2009-05-15T20:23:59 New semantics for evbuffer_cb_set_flags(). Previously, set_flags() would replace all previous user-visible flags. Now it just sets the flags, and there is a clear_flags() function to clear other flags. svn:r1293
Nick Mathewson e865eb93 2009-05-01T00:54:14 More msvc build tweaks. svn:r1262
Nick Mathewson 24607a39 2009-04-29T20:48:43 Note a place we might do better about lock releasing. svn:r1252
Nick Mathewson 11cab334 2009-04-28T19:08:07 Fix compile: #elif FOO is not the same as #elif defined(FOO). svn:r1245
Niels Provos 5c4c13d8 2009-04-24T03:24:22 make sendfile work on freebsd svn:r1239
Nick Mathewson ec6bfd03 2009-04-23T21:41:53 Fix for evbuffer_read() when all data fits in penultimate chain. Previously we were reading into the next-to-last chain, but incrementing the fullness of the last. Bug found by Victor Goya. svn:r1237
Nick Mathewson a8f6d961 2009-04-17T06:56:09 Actually stop using EVBUFFER_LENGTH/DATA, and move them to buffer_compat.h svn:r1183
Nick Mathewson 93d4f884 2009-04-14T20:11:10 Make buffer iocp stuff compile happily svn:r1174
Nick Mathewson 0e32ba54 2009-04-13T03:06:59 Do not remove an empty chain that we have pinned for reading when we drain the whole buffer. svn:r1166
Nick Mathewson 9f1a94ec 2009-04-13T03:06:47 add pin/unpin functions, and a deref-and-free pair. svn:r1165
Nick Mathewson dcda7915 2009-04-13T03:06:27 Add a reference count to evbuffers. svn:r1164
Nick Mathewson b01891fe 2009-04-13T03:06:05 Make evbuffer_commit_space trigger callbacks. svn:r1163
Nick Mathewson 829b52b6 2009-04-13T03:05:46 Refactor the code that sets up iovecs for reading into its own function. iocp needs this. svn:r1162
Nick Mathewson b29b875d 2009-04-10T20:43:08 Facility to make evbuffers get their callbacks deferred. svn:r1154
Nick Mathewson 747331d1 2009-04-08T03:04:39 Add freeze support to evbuffers. From the documentation: Prevent calls that modify an evbuffer from succeeding. A buffer may frozen at the front, at the back, or at both the front and the back. If the front of a buffer is frozen, operations that drain data from the front of the buffer, or that prepend data to the buffer, will fail until it is unfrozen. If the back a buffer is frozen, operations that append data from the buffer will fail until it is unfrozen. We'll use this to ensure correctness on an evbuffer when we're waiting for an overlapped IO call to finish. svn:r1143
Nick Mathewson d9086fc0 2009-04-08T03:03:59 Add a new facility to "pin" the memory in an evbuffer chain. For overlapped IO (and possibly other stuff) we need to be able to label an evbuffer_chain as "pinned", meaning that every byte in it must remain at the same address as it is now until it unpinned. This differs from being "immutable": it is okay to add data to the end of a pinned chain, so long as existing data is not moved. svn:r1142
Nick Mathewson 60e0d59b 2009-04-05T02:44:17 Add locking to evbuffers. svn:r1134
Nick Mathewson f1b1bad4 2009-04-03T14:27:03 Make the new evbuffer callbacks use a new struct-based interface. The old interface would fail pretty hard when we had to batch up multiple adds and drains in a single call. svn:r1131
Nick Mathewson f90500a5 2009-04-03T01:21:36 Add a new improved search function. The old evbuffer_find didn't allow iterative searching, and forced us to repack the buffer completely every time we searched in it. The new evbuffer_search addresses both of these. As a side-effect, the evbuffer_find implementation is now a little more efficient. svn:r1130