Commit 3a53627306233013dcec61a90f0e9ed302ea5156
2023-04-06T18:33:41
jpeg_crop_scanline: Fix calc w/sclg + 2x4,4x2 samp When computing the downsampled width for a particular component, jpeg_crop_scanline() needs to take into account the fact that the libjpeg code uses a combination of IDCT scaling and upsampling to implement 4x2 and 2x4 upsampling with certain decompression scaling factors. Failing to account for that led to incomplete upsampling of 4x2- or 2x4-subsampled components, which caused the color converter to read from uninitialized memory. With 12-bit data precision, this caused a buffer overrun or underrun and subsequent segfault if the uninitialized memory contained a value that was outside of the valid sample range (because the color converter uses the value as an array index.) Fixes #669