kc3-lang/libevent

2014-09-27T21:29:45

http: reset connection before installing retry timer (fix http retries handling) This will fix some invalid read/write: ==556== Invalid read of size 8 ==556== at 0x4E4EEC6: event_queue_remove_timeout (minheap-internal.h:178) ==556== by 0x4E508AA: event_del_nolock_ (event.c:2764) ==556== by 0x4E53535: event_base_loop (event.c:3088) ==556== by 0x406FCFA: dispatch (libcrawl.c:271) ==556== by 0x402863: main (crawler.c:49) ==556== Address 0x68a3f18 is 152 bytes inside a block of size 400 free'd ==556== at 0x4C29C97: free (in /usr/local/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==556== by 0x406F140: renew (libcrawl.c:625) ==556== by 0x4E6CDE9: evhttp_connection_cb_cleanup (http.c:1331) ==556== by 0x4E6E2B2: evhttp_connection_cb (http.c:1424) ==556== by 0x4E4DF2D: bufferevent_writecb (bufferevent_sock.c:310) ==556== by 0x4E52D1D: event_process_active_single_queue (event.c:1584) ==556== by 0x4E53676: event_base_loop (event.c:1676) ==556== by 0x406FCFA: dispatch (libcrawl.c:271) ==556== by 0x402863: main (crawler.c:49) But this one because of some invalid write before (I guess). It is 100% reproduced during massive crawling (because this process has many different servers), but after spending some time for trying to reproduce this using some simple tests/utils I gave up for a few days (I have a lot of work to do), but I'm sending this patch as a reminder. Just in case, I've tried next tests: - mixing timeouts/retries - shutdown http server and return it back - slow dns server for first request - sleep before accept - hacking libevent sources to change the behaviour of http layer (so it will go into that function which I'm insterested in).