encode.cc

Branch :

Log

Commit	Date	Message
17ed2589	2015-08-10 13:13:58	msan bugfixes to the brotli encoder
bad0f4ed	2015-06-26 17:37:00	Brotli Bug Fixes
618287b3	2015-06-12 16:50:49	Deprecate greedy_block_split and enable_context_modeling brotli params. These affected only quality 11, and now it does not make sense to disable block splitting or context modeling because most of the time is spent in zopfli anyway. Now all speed vs size compromises are controlled by the quality param.
66098830	2015-06-12 16:45:17	Use a static hash table to look up dictionary words and transforms. This is used for quality 11, for qualities <= 9 we already have a simpler hash table. The static data size is 252 kB, and this removes the need to initialize a huge hash map at startup, which was the reason why transforms had to be disabled by default. In comparison, the static dictionary itself is 120 kB. This supports every transform, except the kOmitFirstN.
b3d3723f	2015-06-12 16:25:41	Add "zopfli"-style backward reference search to brotli. This commit adopts the backward reference search algorithm from the zopfli project (see https://github.com/google/zopfli) to brotli. This slower backward reference search is run only in quality 11 and it runs two iterations of entropy cost modeling and shortest path search. As a result, the original backward reference search function can be simplified a bit, since we can remove some heuristics that were replaced with the zopfli-style search.
65f3fc55	2015-06-12 16:11:50	Bug fixes for the brotli encoder. * Fix an out-of-bounds access to depth_histo in the bit cost calculation function. * Change type of distance symbol to uint16_t in block splitter, because if all postfix bits are used, there can be 520 distance symbols. * Save the distance cache between meta-blocks at the correct place. This fixes a roundtrip failure that can occur when there is an uncompressed metablock between two compressed metablocks. * Fix a bug when setting lgwin to 24 in the encoder parameters It ended up making metablocks larger than 24 bits in size. * Fix out-of-bounds memory accesses in parallel encoder. CreateBackwardReferences can read up to 4 bytes past end of input if the end of input is before mask. * Add missing header for memcpy() in port.h
b43df8f6	2015-06-12 15:43:54	Brotli custom LZ77 dictionary support. Adds functions to prepend such dictionary to the encoder and decoder, and twiddles their internal parameters to do as if that was a previous part of the input. This dictionary is just a prefilled LZ77 window, it is not related to the built in transformable brotli dictionary.
6622355a	2015-05-11 14:11:07	Use the same hasher for text and font mode. We use 4-byte hashing in both and look for length 3 matches separately.
cc8d64df	2015-05-11 13:51:47	Fix broken quality 0, make it same as quality 1.
aa853f3c	2015-05-11 11:33:19	Add a MODE_GENERIC compression mode to the interface. With this the users can distinguish between not knowing what the input is (ddefault) and knowing that it is text, and thus can be relied on to force some UTF-8 specific settings.
54f69c9e	2015-05-07 17:44:33	Support window bits 10 - 15 in the decoder. The previous window bit value 17 is used to extend the range, since it has not been used in any previous encoders.
7cde616c	2015-05-07 17:30:10	Faster encoding for low quality settings. With this commit, the encoder will skip some compression optimization steps for quality <= 4, which results in faster compression but higher compressed sizes.
945b0d02	2015-05-07 17:23:07	Use a static context map with two buckets for UTF8 data. Enabled for quality >= 4, and if there are no obvious UTF8 violations detected. For each block, we gather two separate histograms, one for continuation bytes and one for ASCII or lead bytes.
0f726df1	2015-04-28 10:12:47	Don't do any block splitting for quality 1.
98539223	2015-04-23 16:20:29	Remove quality parameter from bitstream writing functions. Fix a few crashes related to some quality and param combinations.
2fd80cdc	2015-04-23 15:43:37	Encoder support for new empty meta-block format. Changed the parallel implementation to sync meta-blocks to byte boundary by emitting empty meta-blocks.
3dbe2e03	2015-04-23 15:26:08	Encoder implementation using input/output classes. Add a BrotliCompress() method to the public encoder API that uses the BrotliIn and BrotliOut classes and use that in the 'bro' command-line tool. Use the streaming api in BrotliCompressBuffer() and BrotliCompressor::WriteMetaBlock(). Use the appropiate hashers for quality <= 9.
89a6fb85	2015-04-23 13:15:42	Add params to disable static dictionary and context modeling. Disable all slow features for quality <= 9 (literal cost modeling, dictionary, context modeling, advanced block splitting). Change vector<Command> arguments of internal functions to Command* and size_t.
817a3edd	2015-04-01 16:29:04	Add an input block size parameter to brotli. This will enable processing the input in smaller chunks than the currently default 2MB for the slow brotli, while still benefiting from the larger sliding window.
d6d69ec4	2015-04-01 16:10:15	Add quality and lgwin to the BrotliParams. Remove the hard-coded constants for window size and meta-block size. Initialize internal storage for each metablock separately and reserve only as much as needed for the actual input.
534654de	2015-03-27 14:20:35	Add a faster but less dense compression mode. The new mode can be used by setting the greedy_block_split field of BrotliParams to true. This commit moves all the meta-block processing code into its own library and moves the meta-block encoding code to brotli_bit_stream.cc from encode.cc
497814ee	2015-03-24 10:18:06	Remove the redundant EncodeMetaBlockHeader() function. Use Store{Compressed,Uncompressed}MetaBlockHeader() instead.
96d04e53	2014-10-29 15:39:35	Disable transforms in the encoder by default. This change reduces the startup-time of the encoder considerably.
485ad82e	2014-10-28 14:05:53	Fix potential output buffer overflow in encoder.
0428f2d1	2014-10-28 13:47:21	Move the context map encoding function to the brotli_bit_stream library.
b4f39bf5	2014-10-28 13:25:22	New version of the backward reference search code. The new interface of the backward reference search function makes it possible to use it in a streaming manner. Using the advanced cost model and static dictionary can be turned on/off by template parameters. The distance short codes are now computed as part of the backward reference search. Added a faster version of the Hasher.
467e6eef	2014-10-28 11:53:52	Move the block switch stroing functions to the brotli_bit_stream library.
d6d9fc60	2014-10-15 14:01:36	Factor out serialization functions into their own file. Create a brotli_bit_stream library that is responsible for writing various structures (headers, Huffman codes, etc.) directly into the bit-stream.
3f655b63	2014-03-27 16:38:07	Fix buffer overflow bug in the brotli encoder.
34778194	2014-03-25 16:48:25	Update the dictionary and the transforms.
e7650080	2014-03-20 14:32:35	Updates to Brotli compression format, decoder and encoder This commit contains a batch of changes that were made to the Brotli compression algorithm in the last month. Most important changes: * Format change: don't push distances representing static dictionary words to the distance cache. * Fix decoder invalid memory access bug caused by building a non-complete Huffman tree. * Add a mode parameter to the encoder interface. * Use different hashers for text and font mode. * Add a heuristics to the hasher for skipping non-compressible data. * Exhaustive search of static dictionary during backward reference search.
2f268ad1	2014-02-17 14:25:36	Add the initial version of the static dictionary and transforms to Brotli.
0454ab4e	2014-02-14 15:04:23	Updates to Brotli compression format, decoder and encoder This commit contains a batch of changes that were made to the Brotli compression algorithm in the last month. Most important changes: * Fixes to the spec. * Change of code length code order. * Use a 2-level Huffman lookup table in the decoder. * Faster uncompressed meta-block decoding. * Optimized encoding of the Huffman code. * Detection of UTF-8 input encoding. * UTF-8 based literal cost modeling for improved backward reference selection.
2bcd58bb	2014-01-08 12:28:28	Brotli format change: small improvement to the encoding of Huffman codes Combine the HSKIP and the simple/complex Huffman code type bits.
1447345c	2013-12-17 17:17:57	Brotli format change: improved encoding of Huffman codes This change removes the redundant HCLEN, HLENINC and HLEN fields from the encoding of the complex Huffman codes and derives these from an invariant of the code length sequence. Based on a patch by Robert Obryk.
60c24c0c	2013-12-12 13:18:04	Updates to Brotli compression format, decoder and encoder This commit contains a batch of changes that were made to the Brotli compression algorithm in the last month. Most important changes: * Updated spec * Changed Huffman code length alphabet to use run length codes more efficiently, based on a suggestion by Robert Obryk * Changed encoding of the number of Huffman code lengths (HLEN) * Changed encoding of the number of Huffman trees (NTREES) * Added support for uncompressed meta-blocks
8d7081f2	2013-11-28 17:37:13	Add draft specification of the brotli format
1cdcbd85	2013-11-19 14:32:56	Added Brotli compress/decompress utilities and makefiles
c6b9c7c5	2013-11-15 19:02:17	Updates to Brotli compression format, decoder and encoder This commit contains a batch of changes that were made to the Brotli compression algorithm in the last three weeks. Most important changes: * Added UTF8 context model for good text compression. * Simplified context modeling by having only 4 context modes. * Per-block context mode selection. * Faster backward copying and bit reading functions. * More efficient histogram coding. * Streaming support for the decoder and encoder.
c66e4e3e	2013-10-23 13:06:13	Add brotli compressor This commit is for the encoder for brotli compression format. Brotli is a generic byte-level compression algorithm.

kc3-lang/brotli/enc/encode.cc

Log

kc3-lang/brotli /enc/encode.cc