kc3-lang/brotli

Diff

  • Show log

    Commit

  • Hash : 35e69fc7
    Author : Eugene Kliuchnikov
    Date : 2018-02-26T09:04:36 

    New feature: "Large Window Brotli" (#640)

* New feature: "Large Window Brotli"

By setting special encoder/decoder flag it is now possible to extend
LZ-window up to 30 bits; though produced stream will not be RFC7932
compliant.

Added new dictionary generator - "DSH". It combines speed of "Sieve"
and quality of "DM". Plus utilities to prepare train corpora
(remove unique strings).

Improved compression ratio: now two sub-blocks could be stitched:
the last copy command could be extended to span the next sub-block.

Fixed compression ineffectiveness caused by floating numbers rounding and
wrong cost heuristic.

Other C changes:
 - combined / moved `context.h` to `common`
 - moved transforms to `common`
 - unified some aspects of code formatting
 - added an abstraction for encoder (static) dictionary
 - moved default allocator/deallocator functions to `common`

brotli CLI:
 - window size is auto-adjusted if not specified explicitly

Java:
 - added "eager" decoding both to JNI wrapper and pure decoder
 - huge speed-up of `DictionaryData` initialization

* Add dictionaryless compressed dictionary

* Fix `sources.lst`

* Fix `sources.lst` and add a note that `libtool` is also required.

* Update setup.py

* Fix `EagerStreamTest`

* Fix BUILD file

* Add missing `libdivsufsort` dependency

* Fix "unused parameter" warning.

  • README

  • BROTLI DATA COMPRESSIOM LIBRARY

Brotli is a generic-purpose lossless compression algorithm that compresses data
using a combination of a modern variant of the LZ77 algorithm, Huffman coding
and 2nd order context modeling, with a compression ratio comparable to the best
currently available general-purpose compression methods. It is similar in speed
with deflate but offers more dense compression.

The specification of the Brotli Compressed Data Format is defined in RFC 7932
https://tools.ietf.org/html/rfc7932

Brotli is open-sourced under the MIT License, see the LICENSE file.

Brotli mailing list:
https://groups.google.com/forum/#!forum/brotli

  • README.md

  • Brotli

    Introduction

    Brotli is a generic-purpose lossless compression algorithm that compresses data using a combination of a modern variant of the LZ77 algorithm, Huffman coding and 2nd order context modeling, with a compression ratio comparable to the best currently available general-purpose compression methods. It is similar in speed with deflate but offers more dense compression.

    The specification of the Brotli Compressed Data Format is defined in RFC 7932.

    Brotli is open-sourced under the MIT License, see the LICENSE file.

    Brotli mailing list: https://groups.google.com/forum/#!forum/brotli

    TravisCI Build Status AppVeyor Build Status

    Build instructions

    Autotools-style CMake

    configure-cmake is an autotools-style configure script for CMake-based projects (not supported on Windows).

    The basic commands to build, test and install brotli are:

    $ mkdir out && cd out
$ ../configure-cmake
$ make
$ make test
$ make install

    By default, debug binaries are built. To generate “release” Makefile specify --disable-debug option to configure-cmake.

    Bazel

    See Bazel

    CMake

    The basic commands to build and install brotli are:

    $ mkdir out && cd out
$ cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=./installed ..
$ cmake --build . --config Release --target install

    You can use other CMake configuration.

    Premake5

    See Premake5

    Python

    To install the latest release of the Python module, run the following:

    $ pip install brotli

    To install the tip-of-the-tree version, run:

    $ pip install --upgrade git+https://github.com/google/brotli

    See the Python readme for more details on installing from source, development, and testing.

    Benchmarks

    Related projects

    Disclaimer: Brotli authors take no responsibility for the third party projects mentioned in this section.

    Independent decoder implementation by Mark Adler, based entirely on format specification.

    JavaScript port of brotli decoder. Could be used directly via npm install brotli

    Hand ported decoder / encoder in haxe by Dominik Homberger. Output source code: JavaScript, PHP, Python, Java and C#

    7Zip plugin

kmx.io     mail     discord kmxgit v0.4.0