-
Show log
Commit
-
Hash : c8d52f1c
Author :
Date : 2023-06-26T11:53:03
tj3Transform: Calc dst buf size from xformed dims When used with TJPARAM_NOREALLOC and with TJXOP_TRANSPOSE, TJXOP_TRANSVERSE, TJXOP_ROT90, or TJXOP_ROT270, tj3Transform() incorrectly based the destination buffer size for a transform on the source image dimensions rather than the transformed image dimensions. This was apparently a long-standing bug that had existed in the tj*Transform() function since its inception. As initially implemented in the evolving libjpeg-turbo v1.2 code base, tjTransform() required dstSizes[i] to be set regardless of whether TJFLAG_NOREALLOC (the predecessor to TJPARAM_NOREALLOC) was set. ff78e37595c8462f64fd100f928aa1d08539527e, which was introduced later in the evolving libjpeg-turbo v1.2 code base, removed that requirement and planted the seed for the bug. However, the bug was not activated until 9b49f0e4c77c727648c6d3a4915eefdf5436de4a was introduced still later in the evolving libjpeg-turbo v1.2 code base, adding a subsampling type argument to the (new at the time) tjBufSize() function and thus making the width and height arguments no longer commutative. The bug opened up the possibility that a JPEG source image could cause tj3Transform() to overflow the destination buffer for a transform if all of the following were true: - The JPEG source image used 4:2:2, 4:4:0, 4:1:1, or 4:4:1 subsampling. (These are the only subsampling types for which the width and height arguments to tj3JPEGBufSize() are not commutative.) - The width and height of the JPEG source image were such that tj3JPEGBufSize(height, width, subsamplingType) returned a smaller value than tj3JPEGBufSize(width, height, subsamplingType). - The JPEG source image contained enough metadata that the size of the transformed image was larger than tj3JPEGBufSize(height, width, subsamplingType). - TJPARAM_NOREALLOC was set. - TJXOP_TRANSPOSE, TJXOP_TRANSVERSE, TJXOP_ROT90, or TJXOP_ROT270 was used. - TJXOPT_COPYNONE was not set. - TJXOPT_CROP was not set. - The calling program allocated tj3JPEGBufSize(height, width, subsamplingType) bytes for the destination buffer, as the API documentation instructs. The API documentation cautions that JPEG source images containing a large amount of extraneous metadata (EXIF, IPTC, ICC, etc.) cannot reliably be transformed if TJPARAM_NOREALLOC is set and TJXOPT_COPYNONE is not set. Irrespective of the bug, there are still cases in which a JPEG source image with a large amount of metadata can, when transformed, exceed the worst-case transformed JPEG image size. For instance, if you try to losslessly crop a JPEG image with 3 kB of EXIF data to 16x16 pixels, then you are guaranteed to exceed the worst-case 16x16 JPEG image size unless you discard the EXIF data. Even without the bug, tj3Transform() will still fail with "Buffer passed to JPEG library is too small" when attempting to transform JPEG source images that meet the aforementioned criteria. The bug is that the function segfaults rather than failing gracefully, but the chances of that occurring in a real-world application are very slim. Any real-world application developers who attempted to transform arbitrary JPEG source images with TJPARAM_NOREALLOC set would very quickly realize that they cannot reliably do that without also setting TJXOPT_COPYNONE. Thus, I posit that the actual risk posed by this bug is low. Applications such as web browsers that are the most exposed to security risks from arbitrary JPEG source images do not use the TurboJPEG lossless transform feature. (None of those applications even use the TurboJPEG API, to the best of my knowledge, and the public libjpeg API has no equivalent transform function.) Our only command-line interface to the tj3Transform() function, TJBench, was not exposed to the bug because it had a compatible bug whereby it allocated the JPEG destination buffer to the same size that tj3Transform() erroneously expected. The TurboJPEG Java API was also not exposed to the bug because of a similar compatible bug in the Java_org_libjpegturbo_turbojpeg_TJTransformer_transform() JNI function. (This commit fixes both compatible bugs.) In short, best practices for tj3Transform() are to use TJPARAM_NOREALLOC only with JPEG source images that are known to be free of metadata (such as images generated by tj3Compress*()) or to use TJXOPT_COPYNONE along with TJPARAM_NOREALLOC. Still, however, the function shouldn't segfault as long as the calling program allocates the suggested amount of space for the JPEG destination buffer. Usability notes: tj3Transform() could hypothetically require dstSizes[i] to be set regardless of the value of TJPARAM_NOREALLOC, but there are usability pitfalls either way. The main pitfall I sought to avoid with ff78e37595c8462f64fd100f928aa1d08539527e was a calling program failing to set dstSizes[i] at all, thus leaving its value undefined. It could be argued that requiring dstSizes[i] to be set in all cases is more consistent, but it could also be argued that not requiring it to be set when TJPARAM_NOREALLOC is set is more user-proof. tj3Transform() could also hypothetically set TJXOPT_COPYNONE automatically when TJPARAM_NOREALLOC is set, but that could lead to user confusion. Ultimately, I would like to address these issues in TurboJPEG v4 by using managed buffer objects, but that would be an extensive overhaul.
