enc/encode.h


Log

Author Commit Date CI Message
Eugene Kliuchnikov 3ccbf05d 2016-06-13T11:01:04 Convert encoder to plain C.
Eugene Kliuchnikov 986aa099 2016-06-14T13:42:47 Fix VS compilation warnings; cleanup API.
Eugene Kliuchnikov 58a3023e 2016-06-03T11:19:23 Transform most of C++ comments to C-style.
Eugene Kliuchnikov f1c9ab29 2016-06-03T10:51:04 Extract common parts: constants, dictionary, etc.
Eugene Kliuchnikov be1a53a6 2016-06-14T13:42:47 Fix VS compilation warnings; cleanup API.
Eugene Kliuchnikov b972c677 2016-06-13T11:01:04 Convert encoder to plain C.
Eugene Kliuchnikov 352b0b28 2016-06-03T11:19:23 Transform most of C++ comments to C-style.
Eugene Kliuchnikov 02829186 2016-06-03T10:51:04 Extract common parts: constants, dictionary, etc.
eustas 226d21c4 2016-05-18T11:01:16 Make encoder API more safe Add flag to check if "last" block has been already processed
Zoltan Szabadka b820c39b 2016-03-15T10:50:16 Reduce memory usage of brotli encoder at quality 10 and 11.
Eugene Kliuchnikov 25e3796f 2016-02-29T14:41:24 Fix most of ()->(void) and some missing includes.
Zoltan Szabadka dbb53e64 2016-01-27T09:50:39 Use a hash-to-binary-tree data structure for quality 11 as suggested by Issue #180.
Zoltan Szabadka 82c9e197 2016-01-12T14:45:35 Relax the prerequisites of WriteBrotliData(). Instead of returning false, if it is called with no new input for a non-last block, just check if it has any already processed data to flush, and if not, return true with empty output.
Zoltan Szabadka 417107b3 2016-01-11T11:21:42 Add two more fast modes to the brotli compressor. The new modes process the input data in independent blocks, using backward references only from within an input block. The new modes can be used by specifying quality 0 or quality 1, the old quality 1 and quality 2 modes are renamed quality 2 and quality 3, respectively, and the old quality 3 mode is removed.
Zoltan Szabadka 8844b7f0 2016-01-07T16:27:49 Fix more conversion warnings.
Eugene Klyuchnikov 24ffa784 2015-12-11T11:11:51 Fix headers
Eugene Klyuchnikov 771eb107 2015-11-27T11:27:11 Update license statement in source files.
Eugene Klyuchnikov 152e33c3 2015-11-17T13:45:41 Add more explicit type conversions. Remove dead code. Fix includes.
Zoltan Szabadka ea48ce5a 2015-10-28T17:44:47 Fix --Wconversion and --pedantic-erros for the encoder.
Zoltan Szabadka a89b57b9 2015-10-26T17:08:57 Use uint32_t positions in the hasher and compute distances modulo 2^32.
Zoltan Szabadka 754deaed 2015-10-01T17:08:59 Reduce command buffer memory usage.
Zoltan Szabadka 4c37566f 2015-10-01T15:10:42 Move literal cost computation to where it's used. Move utf8 heuristics functions to their own file.
Zoltan Szabadka 4a7024dc 2015-10-01T12:08:14 Make the brotli encoder C++98 compatible.
Lode Vandevenne 6511d6b0 2015-08-28T16:09:23 update brotli encoder with latest improvements
Zoltan Szabadka 618287b3 2015-06-12T16:50:49 Deprecate greedy_block_split and enable_context_modeling brotli params. These affected only quality 11, and now it does not make sense to disable block splitting or context modeling because most of the time is spent in zopfli anyway. Now all speed vs size compromises are controlled by the quality param.
Zoltan Szabadka 66098830 2015-06-12T16:45:17 Use a static hash table to look up dictionary words and transforms. This is used for quality 11, for qualities <= 9 we already have a simpler hash table. The static data size is 252 kB, and this removes the need to initialize a huge hash map at startup, which was the reason why transforms had to be disabled by default. In comparison, the static dictionary itself is 120 kB. This supports every transform, except the kOmitFirstN.
Zoltan Szabadka 65f3fc55 2015-06-12T16:11:50 Bug fixes for the brotli encoder. * Fix an out-of-bounds access to depth_histo in the bit cost calculation function. * Change type of distance symbol to uint16_t in block splitter, because if all postfix bits are used, there can be 520 distance symbols. * Save the distance cache between meta-blocks at the correct place. This fixes a roundtrip failure that can occur when there is an uncompressed metablock between two compressed metablocks. * Fix a bug when setting lgwin to 24 in the encoder parameters It ended up making metablocks larger than 24 bits in size. * Fix out-of-bounds memory accesses in parallel encoder. CreateBackwardReferences can read up to 4 bytes past end of input if the end of input is before mask. * Add missing header for memcpy() in port.h
Zoltan Szabadka b43df8f6 2015-06-12T15:43:54 Brotli custom LZ77 dictionary support. Adds functions to prepend such dictionary to the encoder and decoder, and twiddles their internal parameters to do as if that was a previous part of the input. This dictionary is just a prefilled LZ77 window, it is not related to the built in transformable brotli dictionary.
Zoltan Szabadka aa853f3c 2015-05-11T11:33:19 Add a MODE_GENERIC compression mode to the interface. With this the users can distinguish between not knowing what the input is (ddefault) and knowing that it is text, and thus can be relied on to force some UTF-8 specific settings.
Zoltan Szabadka 0f726df1 2015-04-28T10:12:47 Don't do any block splitting for quality 1.
Zoltan Szabadka 98539223 2015-04-23T16:20:29 Remove quality parameter from bitstream writing functions. Fix a few crashes related to some quality and param combinations.
Zoltan Szabadka 2fd80cdc 2015-04-23T15:43:37 Encoder support for new empty meta-block format. Changed the parallel implementation to sync meta-blocks to byte boundary by emitting empty meta-blocks.
Zoltan Szabadka 3dbe2e03 2015-04-23T15:26:08 Encoder implementation using input/output classes. Add a BrotliCompress() method to the public encoder API that uses the BrotliIn and BrotliOut classes and use that in the 'bro' command-line tool. Use the streaming api in BrotliCompressBuffer() and BrotliCompressor::WriteMetaBlock(). Use the appropiate hashers for quality <= 9.
Zoltan Szabadka 89a6fb85 2015-04-23T13:15:42 Add params to disable static dictionary and context modeling. Disable all slow features for quality <= 9 (literal cost modeling, dictionary, context modeling, advanced block splitting). Change vector<Command> arguments of internal functions to Command* and size_t.
Zoltan Szabadka e377e65f 2015-04-02T11:12:04 Limit the max input meta-block size to 16MB.
Zoltan Szabadka 817a3edd 2015-04-01T16:29:04 Add an input block size parameter to brotli. This will enable processing the input in smaller chunks than the currently default 2MB for the slow brotli, while still benefiting from the larger sliding window.
Zoltan Szabadka d6d69ec4 2015-04-01T16:10:15 Add quality and lgwin to the BrotliParams. Remove the hard-coded constants for window size and meta-block size. Initialize internal storage for each metablock separately and reserve only as much as needed for the actual input.
Zoltan Szabadka 534654de 2015-03-27T14:20:35 Add a faster but less dense compression mode. The new mode can be used by setting the greedy_block_split field of BrotliParams to true. This commit moves all the meta-block processing code into its own library and moves the meta-block encoding code to brotli_bit_stream.cc from encode.cc
Zoltan Szabadka 96d04e53 2014-10-29T15:39:35 Disable transforms in the encoder by default. This change reduces the startup-time of the encoder considerably.
Zoltan Szabadka 485ad82e 2014-10-28T14:05:53 Fix potential output buffer overflow in encoder.
Zoltan Szabadka b4f39bf5 2014-10-28T13:25:22 New version of the backward reference search code. The new interface of the backward reference search function makes it possible to use it in a streaming manner. Using the advanced cost model and static dictionary can be turned on/off by template parameters. The distance short codes are now computed as part of the backward reference search. Added a faster version of the Hasher.
Zoltan Szabadka d6d9fc60 2014-10-15T14:01:36 Factor out serialization functions into their own file. Create a brotli_bit_stream library that is responsible for writing various structures (headers, Huffman codes, etc.) directly into the bit-stream.
Zoltan Szabadka e7650080 2014-03-20T14:32:35 Updates to Brotli compression format, decoder and encoder This commit contains a batch of changes that were made to the Brotli compression algorithm in the last month. Most important changes: * Format change: don't push distances representing static dictionary words to the distance cache. * Fix decoder invalid memory access bug caused by building a non-complete Huffman tree. * Add a mode parameter to the encoder interface. * Use different hashers for text and font mode. * Add a heuristics to the hasher for skipping non-compressible data. * Exhaustive search of static dictionary during backward reference search.
Zoltan Szabadka 2f268ad1 2014-02-17T14:25:36 Add the initial version of the static dictionary and transforms to Brotli.
Zoltan Szabadka 60c24c0c 2013-12-12T13:18:04 Updates to Brotli compression format, decoder and encoder This commit contains a batch of changes that were made to the Brotli compression algorithm in the last month. Most important changes: * Updated spec * Changed Huffman code length alphabet to use run length codes more efficiently, based on a suggestion by Robert Obryk * Changed encoding of the number of Huffman code lengths (HLEN) * Changed encoding of the number of Huffman trees (NTREES) * Added support for uncompressed meta-blocks
Roderick Sheeter 1cdcbd85 2013-11-19T14:32:56 Added Brotli compress/decompress utilities and makefiles
Zoltan Szabadka c6b9c7c5 2013-11-15T19:02:17 Updates to Brotli compression format, decoder and encoder This commit contains a batch of changes that were made to the Brotli compression algorithm in the last three weeks. Most important changes: * Added UTF8 context model for good text compression. * Simplified context modeling by having only 4 context modes. * Per-block context mode selection. * Faster backward copying and bit reading functions. * More efficient histogram coding. * Streaming support for the decoder and encoder.
Zoltan Szabadka c66e4e3e 2013-10-23T13:06:13 Add brotli compressor This commit is for the encoder for brotli compression format. Brotli is a generic byte-level compression algorithm.