c/common


Log

Author Commit Date CI Message
Brotli fe754f34 2024-05-30T09:50:58 Use a hash table header and SIMD to speed up hash table operations (similar to [Swiss Tables](https://abseil.io/about/design/swisstables)). PiperOrigin-RevId: 638686412
Brotli 04388304 2024-05-09T12:41:57 Use a hash table header and SIMD to speed up hash table operations (similar to [Swiss Tables](https://abseil.io/about/design/swisstables)). PiperOrigin-RevId: 632238409
Brotli bb809ac9 2024-05-08T17:59:09 Use a hash table header and SIMD to speed up hash table operations (similar to [Swiss Tables](https://abseil.io/about/design/swisstables)). PiperOrigin-RevId: 631982664
Brotli 1b3a5ccb 2024-04-18T19:59:28 Prefetch the backreference hashtable bucket. Place the prefetch before the last distance checks, to give the prefetch enough time to work. PiperOrigin-RevId: 626228820
Evgenii Kliuchnikov d5e697b3 2024-01-22T07:24:07 remove dependency on os-specific defines PiperOrigin-RevId: 600449944
Evgenii Kliuchnikov 6b6adb7a 2024-01-03T02:00:39 fix build for Microsoft-designed ARM64 ABI PiperOrigin-RevId: 595334901
Evgenii Kliuchnikov feb6d8bc 2023-08-21T03:34:13 prepare for 1.1.0rc PiperOrigin-RevId: 558736892
Evgenii Kliuchnikov 3ebb2d30 2023-08-18T02:55:01 Move serialized dictionary feature behind the flag. BROTLI_SHARED_DICTIONARY_SERIALIZED enum value is a part of API, but it should not be used (will cause failures). Changing how serialized dictionaries work won't be considered as an API change, until this feature is enabled. Enabling this feature in the future will be considered as a "compatible" change. PiperOrigin-RevId: 558091676
Evgenii Kliuchnikov 27a9a809 2023-07-30T03:44:38 simplify CMake build PiperOrigin-RevId: 552238545
Evgenii Kliuchnikov 6ee96e29 2023-07-03T19:13:24 Internal changes PiperOrigin-RevId: 545262005
Evgenii Kliuchnikov e5dba91c 2023-03-28T16:17:13 Add BROTLI_ENABLE_DUMP build option PiperOrigin-RevId: 520047051
Evgenii Kliuchnikov 6db17c87 2023-03-01T16:51:03 0.4-1.5% decoder speedup PiperOrigin-RevId: 513248503
Zhang Na 1d8452b7 2023-06-20T15:44:23 Add loongarch64 support (#1022)
Evgenii Kliuchnikov a8f5813b 2022-11-17T13:03:09 Update Documentation: - add note that brotli is a "stream" format, not an archive-like - regenerate .1 with Pandoc Build: - drop legacy "BROTLI_BUILD_PORTABLE" option - drop "BROTLI_SANITIZED" definition Code: - c: comb includes - c/enc: extract encoder state into separate header - c/enc: drop designated q10 codepath - c/enc: dealing better with flushing of empty stream - fix MSVC compilation API: - py: use library version instead of one in version.h - c: add plugable API to report consumed input / produced output - c/java: support "lean" prepared dictionaries (without copy of source)
Keith Smiley 6d03dfbe 2022-10-25T12:29:58 Fix -Wstrict-prototypes warnings (#985) Envoy builds brotli with -Werror, and these strict prototypes are picked up by newer versions of clang.
Jyrki Alakuijala 4ec67035 2021-12-07T01:47:07 Merge pull request #929 from jbms/fix-vla-parameter Fix -Werror=vla-parameter errors with GCC 11.2.0
Eugene Kliuchnikov 8376f72e 2021-11-10T10:34:39 Prepare for copybara (#939) Co-authored-by: Eugene Kliuchnikov <eustas@chromium.org>
Jeremy Maitin-Shepard 27dd7265 2021-09-14T12:27:45 Fix -Werror=vla-parameter errors with GCC 11.2.0
Eugene Kliuchnikov 62662f87 2021-09-08T09:18:45 Strip "./" in includes (#925) Co-authored-by: Eugene Kliuchnikov <eustas@chromium.org>
Eugene Kliuchnikov 19d86fb9 2021-08-04T14:42:02 Merge-in SharedDictionary feature (#916) Co-authored-by: Eugene Kliuchnikov <eustas@chromium.org>
Gabriel 97006561 2020-09-21T13:22:56 Fix VC C++ 12.0 BROTLI_MSVC_VERSION_CHECK calls (#843)
Evgenii Kliuchnikov e61745a6 2020-08-27T16:12:55 Re-release
Eugene Kliuchnikov 665e81dc 2020-08-26T14:36:02 New version: 1.0.8 (#827)
Eugene Kliuchnikov 223d80cf 2020-08-26T12:32:27 Update (#826) * IMPORTANT: decoder: fix potential overflow when input chunk is >2GiB * simplify max Huffman table size calculation * eliminate symbol duplicates (static arrays in .h files) * minor combing in research/ code
Eugene Kliuchnikov 0c5603e0 2020-07-09T16:40:01 Fix output parameter type for _BitScanReverse (#819) Fix #811
fisherwky d2ea1982 2020-06-30T17:23:07 Update platform.h (#813) fix compile error (platform.h:362: error: cast discards qualifiers from pointer target type)
Eugene Kliuchnikov 7f740f13 2020-05-15T11:06:21 Update (#807) - fix formatting - fix type conversion - fix no-op arithmetic with null-pointer - improve performance of hash_longest_match64 - go: detect read after close - java decoder: support compound dictionary - remove executable flag on non-scripts
Clinton Ingram 924b2b2b 2020-03-19T03:57:56 Move TZCNT and BSR intrinsics to platform.h, add MSVC versions (#636)
agrieve 4b5771be 2019-12-19T18:15:58 Add missing "const" to a couple of kConstants (#780) These showed up in a Chromium audit: https://bugs.chromium.org/p/chromium/issues/detail?id=747064#c8 Although already effectively const, adding "const" causes the symbols to be moved into the read-only section of the binary.
Eugene Kliuchnikov 3d176718 2019-07-30T10:01:21 Fix include for EMCC build (#765)
Eugene Kliuchnikov f1124c85 2019-07-22T14:29:51 More careful sanitizer detection (#764)
Eugene Kliuchnikov 4b2b2d4f 2019-04-12T13:57:42 Update (#749) Update: * Bazel: fix MSVC configuration * C: common: extended documentation and helpers around distance codes * C: common: enable BROTLI_DCHECK in "debug" builds * C: common: fix implicit trailing zero in `kPrefixSuffix` * C: dec: fix possible bit reader discharge for "large-window" mode * C: dec: simplify distance decoding via lookup table * C: dec: reuse decoder state members memory via union with lookup table * C: dec: add decoder state diagram * C: enc: clarify access to static dictionary * C: enc: improve static dictionary hash * C: enc: add "stream offset" parameter for parallel encoding * C: enc: reorganize hasher; now Q2-Q3 require exactly 256KiB to avoid global TCMalloc lock * C: enc: fix rare access to uninitialized data in ring-buffer * C: enc: reorganize logging / checks in `write_bits.h` * Java: dec: add "large-window" support * Java: dec: improve speed * Java: dec: debug and 32-bit mode are now activated via system properties * Java: dec: demystify some state variables (use better names) * Dictionary generator: add single input mode * Java: dec: modernize tests * Bazel: js: pick working commit for closure rules
Eugene Kliuchnikov d0ffe60b 2018-10-24T16:06:09 Verbose CLI + start pulling "Shared-Brotli" (#722) * Verbose CLI + start pulling "Shared-Brotli" * vesbose CLI output; fix #666 * pull `SHIFT` transforms; currently this is semantically dead code; later it will be used by "Shared-Brotli"
Eugene Kliuchnikov d6d98957 2018-10-23T12:24:40 Ramp up version to 1.0.7
Eugene Kliuchnikov a1e44975 2018-10-19T17:01:54 Fix #698
Eugene Kliuchnikov f7cbc97c 2018-10-16T16:46:54 Fix typo / minor formatting (#716) * Fix typo / minor formatting / pull computable constant to the place of use.
Stephen Kyle cc7a74f1 2018-10-08T14:40:11 decode: fix NEON inclusion (#714) The macro that checks for NEON support should be __ARM_NEON, not __ARM_NEON__. [1] AArch64 compilers define __ARM_NEON but not __ARM_NEON__. AArch32 compilers currently seem to define both, but could be within their rights to drop __ARM_NEON__ in future versions. This change moves the check into the common/platform.h file, checks for both forms, and sets BROTLI_TARGET_NEON if NEON support is available. [1] Section 6.5.4 of the ARM C Language Extensions. (At the time of writing, the latest version was Release 2.1.)
Stephen Kyle 9402ac5c 2018-09-27T12:15:46 decode: faster huffman code loading on 32-bit Arm (#703) * platform: add macro for using the 'aligned' attribute * decode: add accessor macros for HuffmanCode fields Adds a constructor function for building HuffmanCode values so they can be accessed quickly on different architectures. Also adds macros for marking a HuffmanCode table pointer that can be accessed quickly (BROTLI_HC_MARK_TABLE_FOR_FAST_LOAD), adjusting the index into that table (BROTLI_HC_ADJUST_TABLE_INDEX), and getting the .bits or .value fields out of the table at the current index (BROTLI_HC_GET_BITS/VALUE). For example, assuming |table| contains a HuffmanCode pointer: BROTLI_HC_MARK_TABLE_FOR_FAST_LOAD(table); BROTLI_HC_ADJUST_TABLE_INDEX(table, index_into_table); *bits = BROTLI_HC_GET_BITS(table); *value = BROTLI_HC_GET_VALUE(table); BROTLI_HC_ADJUST_TABLE_INDEX(table, offset); *bits2 = BROTLI_HC_GET_BITS(table); *value2 = BROTLI_HC_GET_VALUE(table); All uses of the HuffmanCode have been updated appropriately. * decode: add alternative accessors for HuffmanCode on Arm AArch32
Eugene Kliuchnikov 2216a0dd 2018-09-13T08:09:32 Update (#706) Update * add ASAN/MSAN unaligned read specializations * add "brotli" prefix to u_uint64 type * increment version to 1.0.06 * fix CoverityScan "unused assignment" warning * fix JDK 8<->9 incompatibility * add encoder optimization for empty input * regenerate JS decoder * unbreak Travis builds
Stephen Kyle d4cd6cdf 2018-07-25T10:43:06 platform: fix unaligned 64-bit accesses on AArch32 (#702) Ensures that Aarch32 Arm builds with an Armv8 compiler do not set BROTLI_64_BITS. This scenario is possible with ChromeOS builds, as they may use a toolchain with the target armv7-cros-gnueabi, but with -march=armv8. This will set __ARM_ARCH to 8 (defining BROTLI_TARGET_ARMV8), but will also set __ARM_32BIT_STATE and not __ARM_64BIT_STATE. Without this, illegal 64-bit non-word-aligned reads (LDRD) may be emitted. Also fix unaligned 64-bit reads on AArch32 - STRD was still possible to emit.
Eugene Kliuchnikov 8a073bd9 2018-07-24T17:32:13 Revert "platform: fix unaligned 64-bit accesses on AArch32 (#699)" (#701) This reverts commit 6d027d1648d957c5fee92e535a66420413fd6537.
Stephen Kyle 6d027d16 2018-07-24T16:29:50 platform: fix unaligned 64-bit accesses on AArch32 (#699) Ensures that Aarch32 Arm builds with an Armv8 compiler do not set BROTLI_64_BITS. This scenario is possible with ChromeOS builds, as they may use a toolchain with the target armv7-cros-gnueabi, but with -march=armv8. This will set __ARM_ARCH to 8 (defining BROTLI_TARGET_ARMV8), but will also set __ARM_32BIT_STATE and not __ARM_64BIT_STATE. Without this, illegal 64-bit non-word-aligned reads (LDRD) may be emitted. Also fix unaligned 64-bit reads on AArch32 - STRD was still possible to emit.
Eugene Kliuchnikov b601fe81 2018-06-27T17:03:45 Ramp up version to 1.0.5
Eugene Kliuchnikov eb12ec04 2018-06-20T15:14:10 Update (#688) * add rolling-composite-hasher for large-window mode * make API methods explicitly public
Eugene Kliuchnikov 09cd3e87 2018-06-11T15:17:26 Update
Eugene Kliuchnikov 8544ae85 2018-06-09T11:17:13 Update (#680) * fix MSVC warnings * cleanups
Eugene Kliuchnikov 1e7ea1d8 2018-06-04T17:53:16 Inverse bazel project/workspace tree (#677) * Inverse bazel workspace tree. Now each subproject directly depends on root (c) project. This helps to mitigate Bazel bug bazelbuild/bazel#2391; short summary: Bazel does not work if referenced subproject `WORKSPACE` uses any repositories that embedding project does not. Bright side: building C project is much faster; no need to download closure, go and JDK...
davidlt f9b8c026 2018-05-22T14:35:04 Add RISC-V 64-bit (riscv64) platform configuration (#669) Signed-off-by: David Abdurachmanov <david.abdurachmanov@gmail.com>
Eugene Kliuchnikov f5ed35d0 2018-05-03T11:16:21 Update (#664) * Update * fix ifdef style * get back to fine-compiler-version-based-macros (use Hedley) * fix q=0 histogram collection for very long copy/insert commands
Eugene Kliuchnikov 60003961 2018-04-20T14:10:55 Remove unprefixed macros from public headers (#662)
Eugene Kliuchnikov 0f3c84e7 2018-03-27T22:29:22 Update (#656) * proper fix for the "fall through" warning" * automatic NDIRECT/NPOSTFIX tuning (better compression) * fix unaligned access for `aarch64`-cross-`armhf` build * fix `aarch64` detection (10% decoder speedup) * expose `large_window` CLI option * make default window size 16MiB * ramp up version to 1.0.4
Eugene Kliuchnikov 533843e3 2018-03-02T15:49:58 Update (#643) Update * make the zopflification aware of `NDIRECT`, `NPOSTFIX` (better compression in `font` mode) * add small and simple decoder tool * fix typo * Java: wrapper: make decoder channel more async-friendly Ramp up version to 1.0.3 / 1.0.3
Eugene Kliuchnikov 35e69fc7 2018-02-26T09:04:36 New feature: "Large Window Brotli" (#640) * New feature: "Large Window Brotli" By setting special encoder/decoder flag it is now possible to extend LZ-window up to 30 bits; though produced stream will not be RFC7932 compliant. Added new dictionary generator - "DSH". It combines speed of "Sieve" and quality of "DM". Plus utilities to prepare train corpora (remove unique strings). Improved compression ratio: now two sub-blocks could be stitched: the last copy command could be extended to span the next sub-block. Fixed compression ineffectiveness caused by floating numbers rounding and wrong cost heuristic. Other C changes: - combined / moved `context.h` to `common` - moved transforms to `common` - unified some aspects of code formatting - added an abstraction for encoder (static) dictionary - moved default allocator/deallocator functions to `common` brotli CLI: - window size is auto-adjusted if not specified explicitly Java: - added "eager" decoding both to JNI wrapper and pure decoder - huge speed-up of `DictionaryData` initialization * Add dictionaryless compressed dictionary * Fix `sources.lst` * Fix `sources.lst` and add a note that `libtool` is also required. * Update setup.py * Fix `EagerStreamTest` * Fix BUILD file * Add missing `libdivsufsort` dependency * Fix "unused parameter" warning.
Eugene Kliuchnikov da254cff 2017-12-12T14:33:12 Update (#630) * merge {dec|enc}/port.h into common/platform.h * fix one-shot q=10 1-byte input compression * fix some unprefixed definitions * make hashers host-endianness-independent * extract enc/params.h from enc/quality.h * fix API documentation / typos * improve `BrotliEncoderMaxCompressedSize`
Eugene Kliuchnikov 0ad94eed 2017-11-28T15:37:28 Update (#620) * add autotools build * separate semantic and ABI version * extract sources.lst (used by CMake and Automake) * share pkgconfig templates (used by CMake and Automake) * decoder: always set `total_out` * encoder: fix `BROTLI_ENSURE_CAPACITY` macro (no-op after preprocessor) * decoder/encoder: refine `free_func` contract
Eugene Kliuchnikov 3e58ea5f 2017-10-13T14:50:51 Update (#617) * remove `const` on `BrotliDictionary` members * extend `ZofliNode` distance range to 128MiB * add missing `port.h` include to `quality.h` * fix typo in encoder API-doc * regenerate `decode.min.js`
Tomáš Popela a0c7dafe 2017-10-10T11:24:13 Fix permissions of various files in project (#613) Move from 755 to 644.
Eugene Kliuchnikov 5b476999 2017-09-22T14:05:06 Ramp up to version to 1.0.1
Eugene Kliuchnikov d63e8f75 2017-08-04T10:02:56 Update API, and more (#581) Update API, and more: * remove "custom dictionary" support * c/encoder: fix #580: big-endian build * Java: reduce jar size * Java: speedup decoding * Java: add 32-bit CPU support * Java: make source code JS transpiler-ready
Eugene Kliuchnikov 03739d2b 2017-05-29T17:55:14 Update (#555) Update: * new CLI; bro -> brotli; + man page * JNI wrappers preparation (for bazel build) * add raw binary dictionary representation `dictionary.bin` * add ability to side-load brotli RFC dictionary * decoder persists last error now * fix `BrotliDecoderDecompress` documentation * go reader don't block until necessary * more consistent bazel target names * Java dictionary data compiled footprint reduced * Java tests refactoring
Eugene Kliuchnikov 6ece1d87 2017-04-23T14:07:08 Move files & update paths (#541) * Move files & update paths * Rename build to scripts. * Fix paths * Fix script.