kc3-lang/brotli/docs

Branch :

Log

Author Commit Date CI Message
916e4a46 2025-10-01 02:36:24 update docs PiperOrigin-RevId: 813658707
3396c67f 2024-01-15 12:49:21 add brcat alias + flag to decompress concatenated streams PiperOrigin-RevId: 598652401
27a9a809 2023-07-30 03:44:38 simplify CMake build PiperOrigin-RevId: 552238545
b2c86d18 2023-01-30 18:34:24 Decoder API: added API to attach metadata blocks callbacks PiperOrigin-RevId: 505734532
36533a86 2023-01-16 18:04:35 Internal change PiperOrigin-RevId: 502401179
a8f5813b 2022-11-17 13:03:09 Update Documentation: - add note that brotli is a "stream" format, not an archive-like - regenerate .1 with Pandoc Build: - drop legacy "BROTLI_BUILD_PORTABLE" option - drop "BROTLI_SANITIZED" definition Code: - c: comb includes - c/enc: extract encoder state into separate header - c/enc: drop designated q10 codepath - c/enc: dealing better with flushing of empty stream - fix MSVC compilation API: - py: use library version instead of one in version.h - c: add plugable API to report consumed input / produced output - c/java: support "lean" prepared dictionaries (without copy of source)
62662f87 2021-09-08 09:18:45 Strip "./" in includes (#925) Co-authored-by: Eugene Kliuchnikov <eustas@chromium.org>
0e42caf3 2021-08-31 14:07:17 Migrate to github actions (#920) Not all combinations are migrated to the initial configuration; corresponding TODOs added. Drive-by: additional combinations uncovered minor portability problems -> fixed Drive-by: remove no-longer used "script" files. Co-authored-by: Eugene Kliuchnikov <eustas@chromium.org>
fcda9db7 2020-10-08 14:50:33 Shorten docs/brotli.svg Kudos to @alrra
f6b3aa6d 2020-09-24 13:43:44 Add brotli logo (#845) Co-authored-by: Eugene Kliuchnikov <eustas@chromium.org>
7f740f13 2020-05-15 11:06:21 Update (#807) - fix formatting - fix type conversion - fix no-op arithmetic with null-pointer - improve performance of hash_longest_match64 - go: detect read after close - java decoder: support compound dictionary - remove executable flag on non-scripts
4b2b2d4f 2019-04-12 13:57:42 Update (#749) Update: * Bazel: fix MSVC configuration * C: common: extended documentation and helpers around distance codes * C: common: enable BROTLI_DCHECK in "debug" builds * C: common: fix implicit trailing zero in `kPrefixSuffix` * C: dec: fix possible bit reader discharge for "large-window" mode * C: dec: simplify distance decoding via lookup table * C: dec: reuse decoder state members memory via union with lookup table * C: dec: add decoder state diagram * C: enc: clarify access to static dictionary * C: enc: improve static dictionary hash * C: enc: add "stream offset" parameter for parallel encoding * C: enc: reorganize hasher; now Q2-Q3 require exactly 256KiB to avoid global TCMalloc lock * C: enc: fix rare access to uninitialized data in ring-buffer * C: enc: reorganize logging / checks in `write_bits.h` * Java: dec: add "large-window" support * Java: dec: improve speed * Java: dec: debug and 32-bit mode are now activated via system properties * Java: dec: demystify some state variables (use better names) * Dictionary generator: add single input mode * Java: dec: modernize tests * Bazel: js: pick working commit for closure rules
631fe194 2018-03-20 17:37:41 Update (#651) * fix `bazel` build (ignore switch case fall-through) * add `NPOSTFIX` / `NDIRECT` encoder parameters * fix source file lists (add `params.h`) * fix bug in `durchschlag` * print clarifying messages wheb CLI argument parsing fails
35e69fc7 2018-02-26 09:04:36 New feature: "Large Window Brotli" (#640) * New feature: "Large Window Brotli" By setting special encoder/decoder flag it is now possible to extend LZ-window up to 30 bits; though produced stream will not be RFC7932 compliant. Added new dictionary generator - "DSH". It combines speed of "Sieve" and quality of "DM". Plus utilities to prepare train corpora (remove unique strings). Improved compression ratio: now two sub-blocks could be stitched: the last copy command could be extended to span the next sub-block. Fixed compression ineffectiveness caused by floating numbers rounding and wrong cost heuristic. Other C changes: - combined / moved `context.h` to `common` - moved transforms to `common` - unified some aspects of code formatting - added an abstraction for encoder (static) dictionary - moved default allocator/deallocator functions to `common` brotli CLI: - window size is auto-adjusted if not specified explicitly Java: - added "eager" decoding both to JNI wrapper and pure decoder - huge speed-up of `DictionaryData` initialization * Add dictionaryless compressed dictionary * Fix `sources.lst` * Fix `sources.lst` and add a note that `libtool` is also required. * Update setup.py * Fix `EagerStreamTest` * Fix BUILD file * Add missing `libdivsufsort` dependency * Fix "unused parameter" warning.
da254cff 2017-12-12 14:33:12 Update (#630) * merge {dec|enc}/port.h into common/platform.h * fix one-shot q=10 1-byte input compression * fix some unprefixed definitions * make hashers host-endianness-independent * extract enc/params.h from enc/quality.h * fix API documentation / typos * improve `BrotliEncoderMaxCompressedSize`
3e58ea5f 2017-10-13 14:50:51 Update (#617) * remove `const` on `BrotliDictionary` members * extend `ZofliNode` distance range to 128MiB * add missing `port.h` include to `quality.h` * fix typo in encoder API-doc * regenerate `decode.min.js`
a0c7dafe 2017-10-10 11:24:13 Fix permissions of various files in project (#613) Move from 755 to 644.
c6056359 2017-09-20 15:02:01 Fix API documentation + theoretical NPEs (#602)
d63e8f75 2017-08-04 10:02:56 Update API, and more (#581) Update API, and more: * remove "custom dictionary" support * c/encoder: fix #580: big-endian build * Java: reduce jar size * Java: speedup decoding * Java: add 32-bit CPU support * Java: make source code JS transpiler-ready
05d5f3d7 2017-06-13 12:52:56 Update (#560) Update: * add decoder API to avoid ringbuffer reallocation * fix MSVC warnings * remove dead code
03739d2b 2017-05-29 17:55:14 Update (#555) Update: * new CLI; bro -> brotli; + man page * JNI wrappers preparation (for bazel build) * add raw binary dictionary representation `dictionary.bin` * add ability to side-load brotli RFC dictionary * decoder persists last error now * fix `BrotliDecoderDecompress` documentation * go reader don't block until necessary * more consistent bazel target names * Java dictionary data compiled footprint reduced * Java tests refactoring
c931e576 2017-02-28 16:59:52 Move `java/` to `java/org/brotli/` to fix `sources.jar` structure (#517) Also added man pages to `docs/`
af176847 2016-08-02 13:27:29 Update the spec reference to RFC 7932, remove the old internet draft.
5f02d612 2016-05-26 12:20:42 Update the spec to latest published version.
1841f7cc 2016-05-26 12:19:31 Create -11 version of the spec.
3a9032ba 2016-05-04 21:27:56 Address the DISCUSS ballot position from the IESG review of the spec.
136d39bd 2016-05-04 21:25:37 Create -10 version of the specification.
e96d5b29 2016-04-20 10:48:14 Address review comments in the specification. This commit updates the draft to the ietf -09 version: https://www.ietf.org/id/draft-alakuijala-brotli-09.txt In this version review comments from Jean-loup Gailly and the ietf secdir review were addressed.
26cf47f3 2016-04-20 10:45:40 Create -09 version of the draft.
6a92849c 2016-01-11 13:36:41 Change the title and the expiration date of the -08 draft.
8ef0a202 2016-01-11 13:35:30 Create -08 version of the draft.
3178f4bc 2015-12-10 11:03:22 Add Robert Obryk to the Acknowledgements section of the spec for his work on the first version of the spec in designing the format of the compressed prefix codes.
3fe5c247 2015-11-10 15:09:40 Fix 72-char line length violator
77db683f 2015-11-10 16:06:47 Merge pull request #255 from ende76/master FIX: Typo in reference to NBLTYPESL, Minor: added missing word _lengths_ to insert-and-copy lengths
0e3329d5 2015-11-10 09:52:10 Fixed accidental plural plural wording
e33ff0a6 2015-11-10 05:32:50 Rearranged wording to stay under 72 character limit
1b8b8010 2015-11-07 17:55:22 Fixed two references to wrong NBLTYPESx
9bb41938 2015-11-06 14:14:18 Minor: added missing word _lengths_ to insert-and-copy lengths
652ca06b 2015-11-04 14:54:59 Update the Acknowledgments section of the spec.
af61a513 2015-11-03 17:22:53 Update .txt version of the spec.
ce2bb01f 2015-11-02 09:39:11 Revert accidental deletion in Section 10.
2421ed92 2015-11-02 03:34:42 Clarify pseudo-code in Section 10.
1a50dc9b 2015-11-02 03:30:21 fix formatting of Section 12.
902e8159 2015-11-02 03:27:27 Fix formatting, section references, and grammar * Add .nf and .fi tags everywhere they were missing * Consistently use Section X.X. instead of the following: Paragraph X.X. section X * Fix minor grammar issues
3ab98536 2015-11-01 23:00:07 Fix grammar in Section 2. s/copy length determine /copy length determines /g
e57dbc0f 2015-11-01 18:23:20 Minor capitalization fix
5c869c9d 2015-11-01 17:01:38 Clarify simple and complex prefix codes * At the beginning of the simple prefix code section, telling us that "a value of 1 indicates the number of leading zeros" is not very helpful. Instead, it should indicate that it means a complex prefix code and point the reader to the relevant section (which repeats this information in more detail) * Clearly indicate that reusing a value is an error! This seems to be the behavior of the of the reference implementation. * Clarify what the termination conditions are while reading the prefix codes. Also, indicate that it is an error if the prefix tree is over-subscribed or under-subscribed. * Clearly state what is the maximum number of individual symbols that may be read. This ensures that it is forbidden to an stream that continually says that the symbols have zero length.
c5b6b5c7 2015-11-01 16:50:13 Minor formatting changes * In the description about "three categories", explicitly number them instead of using a giant paragraph that is harder to follow. * Switch lists of items to consistently use American style commas. The American style lists is better for clarity purposes. Consider the following: -Each category of value (insert and copy lengths, literals and distances) +Each category of value (insert and copy lengths, literals, and distances) * Make sure not to break a hyphenated phrase with a newline. When the nroff file is processed, "insert-\nand-copy" becomes "insert- and-copy", making it inconsistent with other uses of the hyphenated phrase. * Consistently use the same hyphenated phrase if referred to as a single unit. "insert and copy" -> "insert-and-copy" "least significant" -> "least-significant" "most significant" -> "most-significant" "fixed length" -> "fixed-length" "block switch" -> "block-switch". * Consistently use "indexes" instead of "indices"
166edb02 2015-11-01 16:28:11 Minor formatting of Section 9.2. and Section 9.3. Many of the fields are copy-pastes of each other, but differ slightly in placement of words, capitalization, or other random oddities. This commit makes it so that if you simply do a search replace on these following passages, you get the same thing: s/NBLTYPESX/(NBLTYPESI|NBLTYPESL|NBLTYPESD)/g s/CATEGORY/(insert-and-copy|literal|distance)/g >>> 1-11 bits: NBLTYPESX, # of CATEGORY block types, encoded with the same variable length code as above Prefix code over the block type code alphabet for CATEGORY block types, appears only if NBLTYPESX >= 2 Prefix code over the block count code alphabet for CATEGORY block counts, appears only if NBLTYPESX >= 2 Block count code + Extra bits for first CATEGORY block count, appears only if NBLTYPESX >= 2 <<< >>> Block type code for next CATEGORY block type, appears only if NBLTYPESX >= 2 and the previous CATEGORY block count is zero Block count code + extra bits for next CATEGORY block count, appears only if NBLTYPESX >= 2 and the previous CATEGORY block count is zero <<<
542a8b77 2015-10-29 09:50:19 Clarify Section 7.3 * Acknowledge the fact that the context map is conceptually really a two-dimensional matrix with 2 different keys, but in reality stored as a one-dimensional array. * Mention that InverseMoveToFrontTransform will not cause the context map to have invalid indexes. This gives someone implementing a decoder sanity that they do not have to go through the context map again and check that all values are less than NTREES.
ff3897df 2015-10-29 09:44:23 Clarify Section 8. * The phrase "difference between these distances" can either refer to the conceptual difference (i.e. they hae different semantic meaning) or to the mathematical difference (i.e. use substraction for the two). Instead, just remove the sentence since the equations below make it clear what we're supposed to do here.
2ffe45bd 2015-10-29 09:42:00 Clarify Section 4. * If NDIRECT is zero, then the paragraph reads "from 16 to 15", which doesn't make much sense. Thus, add a conditional to avoid this minor oddity.
185cb9ea 2015-10-29 09:40:41 Define the maximum number of bytes transforms may add to a word * This value is useful in implementing the decoder since we can know ahead-of-time what size buffer is needed to contain the output of a transformed word.
6d2575ea 2015-10-29 09:39:06 Use consistent bit convention in Section 5. * Rather than say "lower 3 bits" in one sentence and "bits 3-5" in the sentence right after, just consistently use the same convention and say "0-2" and "3-5".
0e4cb52a 2015-10-29 08:32:11 Clarify Section 7.1. * Provide exhaustive list of all the ways the last two bytes can be sourced from. * Also make a clear connection in this section that there are only 64 context IDs for literals. This is important for the indexing math in context maps to make sense.
8523d36e 2015-10-27 12:19:53 Merge pull request #242 from ende76/spec_suggest_block_switch Added note about invalid block type value in block switch commands
11286539 2015-10-27 07:04:38 Removed previous change, fixed typo NBLTYPES -> NBLTYPESL #242
d1cd34f6 2015-10-27 06:35:30 Moved not about invalid distances up to 0-15 section
e544a185 2015-10-26 16:22:28 Added note about invalid block type value
a05fa625 2015-10-26 16:00:20 Added note about invalid distance values
ae04a34c 2015-10-26 12:06:29 Generate new .txt version of the spec. Based on the changes in the .nroff source in PR #231.
816153cc 2015-10-26 12:02:36 Merge pull request #231 from dsnet/master Use consistent bit ordering and variable names
ec8756d7 2015-10-23 15:38:45 Remove note at end of section 3.1 about switching prefix conventions
0a9f65aa 2015-10-22 09:13:59 s/static prefix code/variable length code/g
efeb59c4 2015-10-22 09:11:04 Placed explicit bit pattern table for MNIBBLES to avoid any doubts
c996c06e 2015-10-20 14:54:51 Use consistent bit ordering and variable names If bit-orderings are to be parsed from left-to-right, then make the bit-strings left-justified. If bit-orderings are to be parsed from right-to-left, then make the bit-strings right-justified. Section 3.1, which describes how prefix codes work shows prefix codes that are "left-to-right", which is better for demonstrating how the work. However, most of the rest of the document uses a "right-to-left" convention. We should distinctly say at the end of section 3.1 that we are switching conventions. Thus, change the prefix code in section 3.5 to be "right-to-left" to be consistent with sections 9.1 and 9.2. Also, change the variable names in section 7.3 to be consistent with those used in section 10. Also, change the description of MNIBBLES to be "MNIBBLES - 4", similar to the convention of saying "MLEN - 1". Beforehand, the phrase "If MNIBBLES is 0, then ..." was unclear whether it meant MNIBBLES before the "plus 4" or after.
676bc91c 2015-10-20 12:27:09 Generate new .txt version of the spec. Based on the changes in the .nroff source in PR #229
4f1fce16 2015-10-20 03:02:55 Make code and paragraph both use 3-space indents
f908a4eb 2015-10-20 02:43:25 Fix spelling of "Acknowledgments" Made tab-space of code snippet to be 3-space instead of 2-space
fa1c60e3 2015-10-20 02:39:09 Addressed comments about whitespace
1486df76 2015-10-19 13:53:24 Fixed minor whitespace formatting and ordering of elements Fixed minor whitespacing issues that caused print-out to be slightly confusing. Biggest change is in section 9.2, where an indent seemed to indicate that some fields were part of the previous field, when they were not related. Also, changed the order that transforms are described in section 8 to match the enumeration values that are explicitly defined in Appendix B.
e92afe07 2015-10-19 13:50:23 Add a summary table of alphabet sizes to the spec. Based on a suggestion from Thomas Pickert.
2c3d8eae 2015-10-19 12:16:00 Change the title and the expiration date of the -07 draft.
9bc4008f 2015-10-19 12:15:05 Create -07 version of the draft.
c4f439db 2015-10-06 16:54:04 Change the content encoding type from "bro" to "br".
4d7de651 2015-10-06 15:22:18 Fix the introduction part of the specification. - window bits can be 10 to 24 - meta block can have 0 length
534072ad 2015-10-02 14:40:56 Add brotli comparison study to the docs.
100a2382 2015-10-02 13:08:43 Update the spec with IANA Considerations.
2faed4ab 2015-10-02 13:08:10 Create -06 version of the spec.
cacd294e 2015-09-21 13:29:47 Change the expiration date and title of the -05 draft.
d1341bdd 2015-09-21 13:27:22 Create -05 version of the draft.
e9edf7eb 2015-09-21 13:26:17 Fix typo in the specification.
075b3ad5 2015-09-15 15:35:48 Clarifications to the spec regarding when the stream should be rejected as invalid. Based on Mark Adler's review findings.
ea359368 2015-05-11 17:04:13 Change the expiration date and title of the -04 draft.
14ea2b58 2015-05-11 17:03:35 Create -04 version of the draft.
78350a91 2015-05-07 20:10:22 Add an Acknowledgements section to the spec.
54f69c9e 2015-05-07 17:44:33 Support window bits 10 - 15 in the decoder. The previous window bit value 17 is used to extend the range, since it has not been used in any previous encoders.
94bc27d8 2015-04-27 18:25:59 Fix the year on the copyright message.
fd4a0481 2015-04-27 18:12:09 Change the expiration date and title of the -03 draft.
98bd8841 2015-04-27 17:52:21 Create -03 version of the internet draft.
2d8b2ec1 2015-04-22 12:41:57 Support empty meta-blocks with optional ignored metadata. This is a partially backward incompatible format change, that makes previously valid brotli streams that contain larger than 16MB meta-blocks invalid. The impact of this should be minimal, since the 'bro' command-line tool does not create larger than 2MB meta-blocks, so the only streams this change could break are those created by a custom brotli encoder. This commit contains only the specification update, implementation in the decoder and encoder will follow in later commits.
5b80ef0f 2015-04-22 12:08:16 Change the specification to be less strict in some cases. In the following three cases we allow more choices for the compressor, which can potentially lead to less compressed bits. (1) Allow brotli streams where the block counts do not count down to exactly zero at the end of the meta-block. This makes it possible for compressors to sometimes choose a block count which can be represented with less bits than the exact block count. (2) Remove the restriction that prefix code descriptions with exactly one non-zero length symbol in the code length alphabet must have 1 bit depth. This is because bit depth 1 requires the most bits to encode. (3) Allow any copy length value in the last command where the copy part is ignored. This makes it possible for a compressor to choose a copy length which can be represented with the least amount of bits. In addition to the changes above, this commit also has a wording clarification in the overview section where the use of the 'context ID' expression is changed to be consistent with the rest of the specification, i.e. that it is a function of the last two literals or the copy length.
206d067c 2015-04-22 11:55:29 Use consistent sentence spacing in the specification. All sentence spacing was changed to one space, except in the boilerplate which must be preserved verbatim.
e9fd1a4f 2015-04-22 11:33:38 Add Mark Adler's edits to the specification. The specification source is changed in this commit to exactly mirror the specification edited by Mark Adler: https://github.com/madler/brotli/blob/master/brotli-02-edit.nroff (version 70e53d7)
f80ccecd 2015-04-08 16:15:09 Spec clarifications for Section 7. Based on Mark Adler's review comments.
dcdc68e6 2015-04-08 11:07:00 Spec clarifications for Section 8. Based on Mark Adler's review comments.
92b55173 2015-04-07 17:23:37 Some wording changes to Section 2 of the spec.
707b78ae 2015-04-07 16:58:07 Add the current version of the brotli specification. Add the .nroff source file and a python script to generate the .txt version (requires the nroff command).