Commit e0c53aa38f65915661fcb7d448d50d1c3c7b8391
2023-06-30T17:58:45
jchuff.c: Test for out-of-range coefficients Restore two coefficient range checks from libjpeg to the C baseline Huffman encoder. This fixes an issue (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60253) whereby the encoder could read from uninitialized memory when attempting to transform a specially-crafted malformed arithmetic-coded JPEG source image into a baseline Huffman-coded JPEG destination image with default Huffman tables. More specifically, the out-of-range coefficients caused r to equal 256, which overflowed the actbl->ehufsi[] array. Because the overflow was contained within the huff_entropy_encoder structure, this issue was not exploitable (nor was it observable at all on x86 or Arm CPUs unless JSIMD_NOHUFFENC=1 or JSIMD_FORCENONE=1 was set in the environment or unless libjpeg-turbo was built with WITH_SIMD=0.) The fix is performance-neutral (+/- 1-2%) for x86-64 code and causes a 0-4% (avg. 1-2%, +/- 1-2%) compression regression for i386 code on Intel CPUs when the C baseline Huffman encoder is used (JSIMD_NOHUFFENC=1). The fix is performance-neutral (+/- 1-2%) on Intel CPUs when all of the libjpeg-turbo SIMD extensions are disabled (JSIMD_FORCENONE=1). The fix causes a 0-2% (avg. <1%, +/- 1%) compression regression for PowerPC code.