|
35e69fc7
|
2018-02-26T09:04:36
|
|
New feature: "Large Window Brotli" (#640)
* New feature: "Large Window Brotli"
By setting special encoder/decoder flag it is now possible to extend
LZ-window up to 30 bits; though produced stream will not be RFC7932
compliant.
Added new dictionary generator - "DSH". It combines speed of "Sieve"
and quality of "DM". Plus utilities to prepare train corpora
(remove unique strings).
Improved compression ratio: now two sub-blocks could be stitched:
the last copy command could be extended to span the next sub-block.
Fixed compression ineffectiveness caused by floating numbers rounding and
wrong cost heuristic.
Other C changes:
- combined / moved `context.h` to `common`
- moved transforms to `common`
- unified some aspects of code formatting
- added an abstraction for encoder (static) dictionary
- moved default allocator/deallocator functions to `common`
brotli CLI:
- window size is auto-adjusted if not specified explicitly
Java:
- added "eager" decoding both to JNI wrapper and pure decoder
- huge speed-up of `DictionaryData` initialization
* Add dictionaryless compressed dictionary
* Fix `sources.lst`
* Fix `sources.lst` and add a note that `libtool` is also required.
* Update setup.py
* Fix `EagerStreamTest`
* Fix BUILD file
* Add missing `libdivsufsort` dependency
* Fix "unused parameter" warning.
|
|
b5033d0e
|
2018-02-08T12:48:24
|
|
Fix brotlidump.py crashing when complex prefix code has exactly 1 non-zero code length (#635)
According to the format specification regarding complex prefix codes:
> If there are at least two non-zero code lengths, any trailing zero
> code lengths are omitted, i.e., the last code length in the
> sequence must be non-zero. In this case, the sum of (32 >> code
> length) over all the non-zero code lengths must equal to 32.
> If the lengths have been read for the entire code length alphabet
> and there was only one non-zero code length, then the prefix code
> has one symbol whose code has zero length.
The script does not handle a case where there is just 1 non-zero code
length where the sum rule doesn't apply, which causes a StopIteration
exception when it attempts to read past the list boundaries.
An example of such file is tests/testdata/mapsdatazrh.compressed. I made
sure this change doesn't break anything by processing all *.compressed
files from the testdata folder with no thrown exceptions.
|
|
|
0ad94eed
|
2017-11-28T15:37:28
|
|
Update (#620)
* add autotools build
* separate semantic and ABI version
* extract sources.lst (used by CMake and Automake)
* share pkgconfig templates (used by CMake and Automake)
* decoder: always set `total_out`
* encoder: fix `BROTLI_ENSURE_CAPACITY` macro (no-op after preprocessor)
* decoder/encoder: refine `free_func` contract
|
|
|
39ef4bbd
|
2017-10-13T11:25:03
|
|
Add new (fast) dictionary generator engine. (#616)
Add CLI for dictionary generation.
Add BUILD file for research folder
|
|
a0c7dafe
|
2017-10-10T11:24:13
|
|
Fix permissions of various files in project (#613)
Move from 755 to 644.
|
|
|
a629289e
|
2017-08-28T11:31:29
|
|
Update (#590)
* add transpiled JS decoder
* make PY wrapper accept memview
* fix dictionary generator
* speedup compression of RLEish data
|
|
|
52441069
|
2017-07-21T10:07:24
|
|
Update (#574)
* Update
* decoder: better behavior after failure
* encoder: replace "len_x_code" with delta
* research: add experimental dictionary generator
* python: test combing
|
|
|
27d94590
|
2016-12-22T13:03:28
|
|
Research (#491)
* add advanced mode for optimal references generator
* fix #489
Thanks to Ivan Nikulin for working on it.
|
|
|
fd96151b
|
2016-12-20T18:00:51
|
|
Move brotlidump.py to research/ (#487)
|
|
|
dd8fa3e8
|
2016-09-22T11:32:23
|
|
Update research
* don't use `assert` when side-effect is desired
* use `gflags` to pick options from args
Other changes:
* teach stub `Makefile` to do partial rebuild
* remove obsolete `tools/version.h`
|
|
92940229
|
2016-09-19T19:12:30
|
|
Replace sais.hxx by submodule hillbig/esaxx.
|
|
|
42919320
|
2016-09-15T17:19:26
|
|
Update research tools description.
|
|
|
0e52c59a
|
2016-09-15T16:59:52
|
|
Update variable naming.
|
|
|
9589396e
|
2016-09-15T11:34:19
|
|
Add description of research tools.
|
|
|
58cecf17
|
2016-09-15T10:44:19
|
|
Add distance encoding research tools.
|