buffer.c


Log

Author Commit Date CI Message
Christopher Davis 03afa209 2010-08-16T01:23:57 IOCP-related evbuffer fixes. - Prevent evbuffer_{add,prepend}_buffer from moving read-pinned chains. - Fix evbuffer_drain to handle read-pinned chains better. - Raise the limit on WSABUFs from two to MAX_WSABUFS for overlapped reads.
Nick Mathewson 743f8665 2010-08-23T11:48:46 Honor NDEBUG; build without warnings with NDEBUG; make NDEBUG always-off in unit test code
Nick Mathewson ec347b92 2010-07-07T16:45:03 Move event-config.h to include/event2 This change means that all required include files are in event2, and all files not in event2/* are optional.
niks 65abdc20 2010-07-16T09:11:09 Fix wrong sie calculation of iovec buffers when exact=1 The old code had a bug where the 'exact' flag to 1 in _evbuffer_read_setup_vecs would never actually make the iov_len field of the last iovec get truncated. This patch fixes that.
Nick Mathewson 3467f2fa 2010-05-28T15:05:32 Fix logic in correcting high values from FIONREAD The old logic made sense back when buffer.c was an enormous linear buffer, but it doesn't make any sense for the chain-based implementation. This patch also refactors the ioctl{socket}? call into its own function.
Nick Mathewson c44de06c 2010-05-08T18:09:27 Numerous opensolaris compilation fixes For future note, opensolaris doesn't have sys/sysctl.h, doesn't like comparing iov_buf to a chain_space_ptr without a cast, and is (predictably) unforgiving of dumb syntax errors. Also, we had accidentally broken the devpoll backend test in configure.in
Nick Mathewson dcdae6b7 2010-05-08T16:34:18 Make evbuffer_add_file() work on windows Right now only the add_file() mode is supported, when it would be nicer to have mmap support. Perhaps for Libevent 2.1.x.
Nick Mathewson d49b92a8 2010-04-23T23:04:20 Remove one last bug in last_with_datap logic. Found with valgrind
Giuseppe Scrivano d469c503 2010-04-12T12:18:57 Fix compiler warnings under WIN32
Nick Mathewson eb86c8c5 2010-04-12T22:24:54 Add evbuffer_copyout to copy data from an evbuffer without draining The evbuffer_remove() function copies data from the front of an evbuffer into an array of char, and removes the data from the buffer. This function behaves the same, but does not remove the data. This behavior can be handy for lots of protocols, where you want the evbuffer to accumulate data until a complete record has arrived. Lots of people have asked for a function more or less like this, and though it isn't too hard to code one from evbuffer_peek(), it is apparently annoying to do it in every app you write. The evbuffer_peek() function is significantly faster, but it requires that the user be able to handle data in separate extents. This patch also reimplements evbufer_remove() as evbuffer_copyout() followed by evbuffer_drain(). I am reasonably confident that this won't be a performance hit: the memcpy() overhead should dominate the cost of walking the list an extra time.
Nick Mathewson 8c83e995 2010-04-09T16:40:53 Add more unit tests for evbuffer_expand
Nick Mathewson 06a4443a 2010-04-09T15:28:26 Unit-test every evbuffer_add_file() implementation. Previously, we'd only test the default one, even if the others were still compiled in.
Christopher Davis 28bfed47 2010-04-02T19:08:32 Clean up a mistake in pointer manipulation in evbuffer_remove
Nick Mathewson d5ebcf37 2010-03-30T16:47:37 Rewrite evbuffer_expand and its users The previous evbuffer_expand was not only incorrect; it was inefficient too. On all questions of time vs memory tradeoffs, it chose to burn time in order to avoid wasting memory. The new code tries to be a little more balanced: it only resizes an existing chain when doing so doesn't require too much copying, and when failing to do so would waste a lot of the chain's space. This patch also rewrites evbuffer_chain_insert to work properly with last_with_datap, and adds a few convenience functions to buffer.c.
Nick Mathewson 45068a31 2010-03-31T12:03:43 Fix a memory leak when appending/prepending to a buffer with unused space.
Nick Mathewson 8e227b04 2010-03-27T00:09:25 Make the no_iovecs case of write_atmost compile Apparently nobody had tested it before on a system that had sendfile. Why would you have sendfile and not writev? Perhaps you're trying to test the no-iovecs code to make sure it still works.
Nick Mathewson 96865c47 2010-03-30T12:48:56 Turn the increasingly complex *_CHAIN() macros into functions
Nick Mathewson b7442f8e 2010-03-26T23:18:40 Replace last_with_data with a slightly smarter version To implement evbuffer_expand() properly, you need to be able to replace the last chunk that has data, which means that we need to keep track of the the next pointer pointing to the last_with_data chunk, not the last_with_data chunk itself.
Nick Mathewson cda56abf 2010-03-31T12:29:26 Fix critical bug in evbuffer_write when writev is not available evbuffer_pullup() returns NULL if you try to pull up more bytes than are there. But evbuffer_write_atmost would sometimes ask for more bytes to be pulled up than it had, get a NULL, and fail.
Nick Mathewson c87272b7 2010-03-26T14:51:39 Make evbuffer_prepend handle empty buffers better If the first chunk of a buffer is empty, and we're told to prepend to the buffer, we should be willing to use the entire first chunk. Instead, we were dependent on the value of chunk->misalign.
Nick Mathewson 5c0ebb33 2010-03-26T14:50:45 Do not use evbuffer_expand() to add the first chain to a buffer (It's a big function, and using it this way is overkill.)
Nick Mathewson f1bc125e 2010-03-12T23:00:49 Improve robustness for refcounting Document that we do intend to double-decref underlying bufferevents under some circumstances. Check to make sure that we don't decref past 0.
Nick Mathewson 1e7b9868 2010-03-11T14:23:02 Fix last_with_data compilation on windows
Nick Mathewson e470ad3c 2010-03-10T23:39:30 Allow evbuffer_read() to split across more than 2 iovecs Previously it would only accept 2 iovecs at most, because our previous_to_last nonsense didn't let it take any more. This forced us to do more reallocations in some cases when an extra small malloc would have sufficed.
Nick Mathewson 6f47bd12 2010-03-10T23:28:51 Remove previous_to_last from evbuffer
Nick Mathewson c8ac57f1 2010-03-10T23:24:14 Use last_with_data in place of previous_to_last This actually makes some of the code a lot simpler. The only ones that actually used previous_to_last for anything were reserving and committing space.
Nick Mathewson 2a6d2a1e 2010-03-10T22:16:14 Revise evbuffer to add last_with_data This is the first patch in a series to replace previous_to_last with last_with_data. Currently, we can only use two partially empty chains at the end of an evbuffer, so if we have one with 511 bytes free, and another with 512 bytes free, and we try to do a 1024 byte read, we can't just stick another chain on the end: we need to reallocate the last one. That's stupid and inefficient. Instead, this patch adds a last_with_data pointer to eventually replace previous_to_last. Instead of pointing to the penultimated chain (if any) as previous_to_last does, last_with_data points to the last chain that has any data in it, if any. If all chains are empty, last_with_data points to the first chain. If there are no chains, last_with_data is NULL. The next step is to start using last_with_data everywhere that we currently use previous_to_last. When that's done, we can remove previous_to_last and the code that maintains it.
Nick Mathewson 17efc1cd 2010-03-04T01:25:51 Update all our copyright notices to say "2010"
Nick Mathewson cc1600af 2010-03-02T17:00:06 Improve the speed of evbuffer_readln() This makes some cases of bench_http about 5% faster. Our internal evbuffer_strpbrk() function was overly general (it tried to handle all character sets when we only used it for "\r\n"), and not very efficient (it called memchr once for each character in the buffer until it found a \r or a \n). It actually showed up in some profiles for HTTP testing, since evbuffer_readln() calls it when doing loose CRLF detection. This patch replaces it with a faster implementation.
Nick Mathewson b2fbeb3f 2010-02-22T15:38:23 Make bufferevent_free() clear all callbacks immediately. This should end the family of bugs where we call bufferevent_free() while a pending callback is holding a reference on the bufferevent, and the callback tries to invoke the user callbacks before it releases its own final reference. This means that bufferevent_decref() is now a separate function from bufferevent_free().
Nick Mathewson e5bbd40a 2010-02-18T17:41:15 Clean up formatting: use tabs, not 8-spaces, to indent.
Nick Mathewson 7116bf23 2010-02-15T21:03:52 Fix two unlocked reads in evbuffer. Some initializers (in evbuffer_read and evbuffer_commit) were reading the last and/or previous_to_last fields without grabbing the evbuffer lock. This may fix a hard-to-trigger race condition or two.
Nick Mathewson 3fe60fdf 2010-02-12T23:40:13 Use off_t for the length parameter of evbuffer_add_file
Nick Mathewson 29151e65 2010-01-06T18:42:59 Fix byte counts when mixing deferred and non-deferred evbuffer callbacks. This patch finishes 390e0561, which was somehow committed in a half-finished state. It solves a failing unit test on windows.
Nick Mathewson a47d88d7 2009-12-23T07:53:19 Replace some cases of uint32_t with ev_uint32_t. Spotted by Roman Puls.
Nick Mathewson 390e0561 2009-12-22T15:52:02 Fix up behavior of never-defered callbacks a little
Nick Mathewson 438f9ed2 2009-11-23T15:53:24 Add the abilitity to mark some buffer callbacks as never-deferred.
Nick Mathewson 76cd2b70 2009-11-27T16:44:47 Stop passing EVTHREAD_READ and EVTHREAD_WRITE to non-rw locks. Previously, our default lock model kind of assumed that every lock was potentially a read-write lock. This was a poor choice, since read-write locks are far more expensive than regular locks, and so the lock API should only use them when we can actually take advantage of them. Neither our pthreads or win32 lock implementation provided rw locks. Now that we have a way (not currently used!) to indicate that we really want a read-write lock, we shouldn't actually say "lock this for reading" or "lock this for writing" unless we mean it.
Nick Mathewson 347952ff 2009-11-27T15:20:43 Revise the locking API: deprecate the old locking callbacks and add trylock. Previously, there was no good way to request different kinds of lock (say, read/write vs writeonly or recursive vs nonrecursive), or for a lock function to signal failure (which would be important for a trylock mode). This patch revises the lock API to be a bit more useful. The older lock calls are still supported for now. We also add a debugging mode to catch common errors in using the locking APIs.
Niels Provos f1691539 2009-11-19T23:08:50 Remove most calls to event_err() in http and deal with memory errors instead svn:r1555
Nick Mathewson d7d1f1da 2009-11-17T20:31:09 Move responsibility for IOCP callback into bufferevent_async. This patch from Chris Davis saves some callback depth, and adds proper ref-counting to bufferevents when there's a deferred evbuffer callback inflight. It could use a couple more comments to really nail down what its invariants are. svn:r1543
Nick Mathewson 784b8773 2009-11-06T21:46:57 We do not work any more without an event-config.h; stop pretending that it is meaningful to check for HAVE_CONFIG_H svn:r1516
Nick Mathewson ac633aeb 2009-11-05T21:22:23 Fix some build warnings on MSVC, mostly related to signed/unsigned comparisons. svn:r1510
Nick Mathewson 7f0ad2f6 2009-11-02T16:17:06 Fix an errant user of ssize_t to use ev_ssize_t. My usual strategy of grep '[^_]ssize_t' had apparently failed me, since this ssize_t was in the first column. Resolves bug 2890434; spotted by Mihai Draghicioiu. svn:r1484
Nick Mathewson 10cf631e 2009-10-27T04:04:07 Do not add a newline to the end of log statements. svn:r1468
Nick Mathewson 2e36dbe1 2009-10-26T20:00:43 Use EVUTIL_ASSERT() consistently instead of assert. svn:r1464
Nick Mathewson e3fd294a 2009-10-16T13:19:57 Spelling fixes in comments and strings. svn:r1445
Niels Provos 8e8d94a3 2009-09-24T22:18:19 Do not drop data from evbuffer when out of memory; reported by Jacek Masiulaniec svn:r1436
Nick Mathewson 3b461a6d 2009-09-11T18:21:37 Treat a negative number of bytes to read as the kernel saying "I don't know." svn:r1426
Nick Mathewson 22bd5b42 2009-08-16T16:40:42 Support sendfile on solaris: patch from Caitlin Mercer. svn:r1419
Nick Mathewson 8a99083f 2009-08-07T17:16:52 Add an evbuffer_search_range() to search a bounded range of a buffer This can be handy when you have one search to find the end of a header section, and then you want to find a substring within the header section without looking at the body. svn:r1410
Nick Mathewson d4134772 2009-07-31T17:35:42 Refactor evbuffer_readln() into a search-for-eol function and an extract-line function. svn:r1404
Nick Mathewson a26d2d1b 2009-07-31T17:34:47 Refactor evbuffer_readln to use evbuffer_ptr; remove old evbuffer_iterator. svn:r1403
Nick Mathewson b06b2649 2009-07-26T01:29:39 Make "deferred callback queue" independent of event_base. This way, we can more easily have an IOCP bufferevent implementation that does not need an event_base at all. Woot. svn:r1381
Nick Mathewson 8eb155a1 2009-07-23T14:48:24 Fix build on platforms (like Solaris 10, reportedly) which lack a MAP_FILE. svn:r1375
Nick Mathewson 5aefb8a6 2009-06-25T15:22:36 Fix type on freebsd sendfile. Patch from navin seshadri. Fixes bug 2811991 svn:r1330
Nick Mathewson 0b22ca19 2009-05-22T19:11:48 Use ev_ssize_t in place of ssize_t *everywhere*. svn:r1309
Nick Mathewson 8997f234 2009-05-21T20:59:00 Use the native "struct iovec" as our "struct evbuffer_iovec" when available, so we do not need to copy more pointers than necessary. svn:r1299
Nick Mathewson 23243b8a 2009-05-19T21:39:35 Replace reserve/commit with new iovec-based interface. Add a new evbuffer_peek. svn:r1296
Nick Mathewson dc4c7b95 2009-05-15T22:44:18 Change the interface of evbuffer_add_reference: give the cleanup function more info. svn:r1294
Nick Mathewson bba69e03 2009-05-15T20:23:59 New semantics for evbuffer_cb_set_flags(). Previously, set_flags() would replace all previous user-visible flags. Now it just sets the flags, and there is a clear_flags() function to clear other flags. svn:r1293
Nick Mathewson e865eb93 2009-05-01T00:54:14 More msvc build tweaks. svn:r1262
Nick Mathewson 24607a39 2009-04-29T20:48:43 Note a place we might do better about lock releasing. svn:r1252
Nick Mathewson 11cab334 2009-04-28T19:08:07 Fix compile: #elif FOO is not the same as #elif defined(FOO). svn:r1245
Niels Provos 5c4c13d8 2009-04-24T03:24:22 make sendfile work on freebsd svn:r1239
Nick Mathewson ec6bfd03 2009-04-23T21:41:53 Fix for evbuffer_read() when all data fits in penultimate chain. Previously we were reading into the next-to-last chain, but incrementing the fullness of the last. Bug found by Victor Goya. svn:r1237
Nick Mathewson a8f6d961 2009-04-17T06:56:09 Actually stop using EVBUFFER_LENGTH/DATA, and move them to buffer_compat.h svn:r1183
Nick Mathewson 93d4f884 2009-04-14T20:11:10 Make buffer iocp stuff compile happily svn:r1174
Nick Mathewson 0e32ba54 2009-04-13T03:06:59 Do not remove an empty chain that we have pinned for reading when we drain the whole buffer. svn:r1166
Nick Mathewson 9f1a94ec 2009-04-13T03:06:47 add pin/unpin functions, and a deref-and-free pair. svn:r1165
Nick Mathewson dcda7915 2009-04-13T03:06:27 Add a reference count to evbuffers. svn:r1164
Nick Mathewson b01891fe 2009-04-13T03:06:05 Make evbuffer_commit_space trigger callbacks. svn:r1163
Nick Mathewson 829b52b6 2009-04-13T03:05:46 Refactor the code that sets up iovecs for reading into its own function. iocp needs this. svn:r1162
Nick Mathewson b29b875d 2009-04-10T20:43:08 Facility to make evbuffers get their callbacks deferred. svn:r1154
Nick Mathewson 747331d1 2009-04-08T03:04:39 Add freeze support to evbuffers. From the documentation: Prevent calls that modify an evbuffer from succeeding. A buffer may frozen at the front, at the back, or at both the front and the back. If the front of a buffer is frozen, operations that drain data from the front of the buffer, or that prepend data to the buffer, will fail until it is unfrozen. If the back a buffer is frozen, operations that append data from the buffer will fail until it is unfrozen. We'll use this to ensure correctness on an evbuffer when we're waiting for an overlapped IO call to finish. svn:r1143
Nick Mathewson d9086fc0 2009-04-08T03:03:59 Add a new facility to "pin" the memory in an evbuffer chain. For overlapped IO (and possibly other stuff) we need to be able to label an evbuffer_chain as "pinned", meaning that every byte in it must remain at the same address as it is now until it unpinned. This differs from being "immutable": it is okay to add data to the end of a pinned chain, so long as existing data is not moved. svn:r1142
Nick Mathewson 60e0d59b 2009-04-05T02:44:17 Add locking to evbuffers. svn:r1134
Nick Mathewson f1b1bad4 2009-04-03T14:27:03 Make the new evbuffer callbacks use a new struct-based interface. The old interface would fail pretty hard when we had to batch up multiple adds and drains in a single call. svn:r1131
Nick Mathewson f90500a5 2009-04-03T01:21:36 Add a new improved search function. The old evbuffer_find didn't allow iterative searching, and forced us to repack the buffer completely every time we searched in it. The new evbuffer_search addresses both of these. As a side-effect, the evbuffer_find implementation is now a little more efficient. svn:r1130
Nick Mathewson 01456265 2009-02-10T19:39:22 Explode less badly in the case where we're told to prepend/append/remove a buffer to itself. Note some API/implementation deficiencies. svn:r1110
Nick Mathewson edfdb698 2009-02-10T19:38:34 Add an assertion to evbuffer_chain_align so we can't reuse it in the future. svn:r1105
Nick Mathewson cc049bfc 2009-02-03T05:22:57 Enable (and debug) WSARecv for evbuffer iovec-like reads. The two things we were missing: the flags parameter is not optional, and an error can actually indicate a close. svn:r1100
Nick Mathewson e84c7656 2009-02-01T05:26:47 Allocate callback entries with contents 0d out. svn:r1093
Nick Mathewson 8d3a10f8 2009-02-01T01:43:58 Support temporarily suspending an evbuffer callback. This is different from disabling the callback, since we want to process changes, but not just yet. svn:r1091
Niels Provos a0cae310 2009-01-29T03:22:47 make it so that evbuffer_add_file where we read the complete contents of the file can fail without side effects svn:r1069
Niels Provos 66b2a7ff 2009-01-29T03:20:40 test evbuffer_add_reference svn:r1068
Nick Mathewson b85b710c 2009-01-27T22:34:36 Update copyright statements to reflect the facts that: a) this is 2009 b) niels and nick have been comaintainers for a while c) saying "all rights reserved" when you then go on to explicitly disclaim some rights is sheer cargo-cultism. svn:r1065
Nick Mathewson 8889a770 2009-01-27T22:30:46 Replace all use of config.h with event-config.h. svn:r1064
Nick Mathewson 9993137c 2009-01-27T21:10:31 Remove all trailing whitespace in all the source files. svn:r1063
Nick Mathewson 88f2b7a0 2009-01-27T13:37:09 Fix some warnings on linux gcc with --enable-gcc-warnings svn:r1060
Niels Provos 1757cf71 2009-01-27T06:21:12 use %zu for off_t svn:r1059
Niels Provos 8b5bd774 2009-01-27T06:18:45 make it compile on linux svn:r1058
Niels Provos fdf69493 2009-01-27T06:05:38 sendfile/mmap and memory reference implementation for evbuffers svn:r1057
Niels Provos b93e5054 2009-01-27T05:33:39 second argument to evbuffer_pullup should be ssize_t svn:r1056
Nick Mathewson 81dd04a7 2009-01-23T18:04:34 Add a "flags" field to evbuffer callbacks. For now, there is just one: enabled. This lets us avoid lots of mallocs/frees/tailq-manipulations just to turn a callback on and off. The revised bufferevent code wants this. svn:r1047
Nick Mathewson de7f7a84 2009-01-23T18:03:45 Remove in_callbacks check: allow full recursion in evbuffer callbacks. If you get yourself in an infinite loop, that's not our fault. Note this in the docs. Also reindent some docs now that my tabs match Niels's. svn:r1046
Nick Mathewson ec2f4cbc 2009-01-23T01:35:57 Move obsolete evbuffer function into include/event2/buffer_compat.h svn:r1043
Nick Mathewson c735f2b4 2009-01-23T01:11:13 Code to allow multiple callbacks per evbuffer. svn:r1042
Nick Mathewson f6eb1f81 2009-01-19T21:53:03 Change evbuffer_read implementation to split data across chunks, and use readv when available. This should make us use less space. svn:r1024
Nick Mathewson 84031819 2009-01-14T22:17:31 Make some evbuffer functions const svn:r1010