enc/encode_parallel.cc


Log

Author Commit Date CI Message
Eugene Kliuchnikov 3ccbf05d 2016-06-13T11:01:04 Convert encoder to plain C.
Eugene Kliuchnikov 58a3023e 2016-06-03T11:19:23 Transform most of C++ comments to C-style.
Eugene Kliuchnikov f1c9ab29 2016-06-03T10:51:04 Extract common parts: constants, dictionary, etc.
Eugene Kliuchnikov b972c677 2016-06-13T11:01:04 Convert encoder to plain C.
Eugene Kliuchnikov 352b0b28 2016-06-03T11:19:23 Transform most of C++ comments to C-style.
Eugene Kliuchnikov 02829186 2016-06-03T10:51:04 Extract common parts: constants, dictionary, etc.
Zoltan Szabadka b820c39b 2016-03-15T10:50:16 Reduce memory usage of brotli encoder at quality 10 and 11.
Zoltan Szabadka dbb53e64 2016-01-27T09:50:39 Use a hash-to-binary-tree data structure for quality 11 as suggested by Issue #180.
Zoltan Szabadka 8844b7f0 2016-01-07T16:27:49 Fix more conversion warnings.
Eugene Klyuchnikov 24ffa784 2015-12-11T11:11:51 Fix headers
Eugene Klyuchnikov 771eb107 2015-11-27T11:27:11 Update license statement in source files.
Zoltan Szabadka ea48ce5a 2015-10-28T17:44:47 Fix --Wconversion and --pedantic-erros for the encoder.
Zoltan Szabadka 2726b8a4 2015-10-06T11:23:44 Encoder fixes. * Remove default constructors. * Initialize bit_cost in histogram.Clear(). * Check fseek result in FileSize. * Replace malloc in BrotliFileIn constructor with "new". * Catch bad_alloc in bro tool.
Zoltan Szabadka b39eec88 2015-10-05T11:43:49 Remove C++11 vector::data() calls from encoder.
Zoltan Szabadka 4c37566f 2015-10-01T15:10:42 Move literal cost computation to where it's used. Move utf8 heuristics functions to their own file.
Zoltan Szabadka 4a7024dc 2015-10-01T12:08:14 Make the brotli encoder C++98 compatible.
Lode Vandevenne 17ed2589 2015-08-10T13:13:58 msan bugfixes to the brotli encoder
Zoltan Szabadka 618287b3 2015-06-12T16:50:49 Deprecate greedy_block_split and enable_context_modeling brotli params. These affected only quality 11, and now it does not make sense to disable block splitting or context modeling because most of the time is spent in zopfli anyway. Now all speed vs size compromises are controlled by the quality param.
Zoltan Szabadka 66098830 2015-06-12T16:45:17 Use a static hash table to look up dictionary words and transforms. This is used for quality 11, for qualities <= 9 we already have a simpler hash table. The static data size is 252 kB, and this removes the need to initialize a huge hash map at startup, which was the reason why transforms had to be disabled by default. In comparison, the static dictionary itself is 120 kB. This supports every transform, except the kOmitFirstN.
Zoltan Szabadka b3d3723f 2015-06-12T16:25:41 Add "zopfli"-style backward reference search to brotli. This commit adopts the backward reference search algorithm from the zopfli project (see https://github.com/google/zopfli) to brotli. This slower backward reference search is run only in quality 11 and it runs two iterations of entropy cost modeling and shortest path search. As a result, the original backward reference search function can be simplified a bit, since we can remove some heuristics that were replaced with the zopfli-style search.
Zoltan Szabadka 65f3fc55 2015-06-12T16:11:50 Bug fixes for the brotli encoder. * Fix an out-of-bounds access to depth_histo in the bit cost calculation function. * Change type of distance symbol to uint16_t in block splitter, because if all postfix bits are used, there can be 520 distance symbols. * Save the distance cache between meta-blocks at the correct place. This fixes a roundtrip failure that can occur when there is an uncompressed metablock between two compressed metablocks. * Fix a bug when setting lgwin to 24 in the encoder parameters It ended up making metablocks larger than 24 bits in size. * Fix out-of-bounds memory accesses in parallel encoder. CreateBackwardReferences can read up to 4 bytes past end of input if the end of input is before mask. * Add missing header for memcpy() in port.h
Zoltan Szabadka 54f69c9e 2015-05-07T17:44:33 Support window bits 10 - 15 in the decoder. The previous window bit value 17 is used to extend the range, since it has not been used in any previous encoders.
Zoltan Szabadka 0f726df1 2015-04-28T10:12:47 Don't do any block splitting for quality 1.
Zoltan Szabadka 98539223 2015-04-23T16:20:29 Remove quality parameter from bitstream writing functions. Fix a few crashes related to some quality and param combinations.
Zoltan Szabadka 2fd80cdc 2015-04-23T15:43:37 Encoder support for new empty meta-block format. Changed the parallel implementation to sync meta-blocks to byte boundary by emitting empty meta-blocks.
Zoltan Szabadka 89a6fb85 2015-04-23T13:15:42 Add params to disable static dictionary and context modeling. Disable all slow features for quality <= 9 (literal cost modeling, dictionary, context modeling, advanced block splitting). Change vector<Command> arguments of internal functions to Command* and size_t.
Zoltan Szabadka 1428d541 2015-04-01T16:35:52 Proof-of-concept encoder for parallel compression. Add a version of the brotli encoder that compresses each meta-block independently, only using the original input data from previous meta-blocks and nothing from the compressor state. This is a proof-of-concept to show that the current format is flexible enough to support parallel multi-threaded compression.