kmx git

Commit	Date	Message
f74989d8	2025-09-25T11:32:45	Clean up #include directives This is subtle, but #include <header.h> searches directories specified with -I, then system include directories. #include "header.h" searches the current source directory, then directories specified with -I, then system include directories. Using bracketed #include directives for jpeglib.h, jinclude.h, jerror.h, cdjpeg.h, and turbojpeg.h only worked because the build system explicitly passed -I{source_directory}/src/ to the compiler. Referring to 51cee0362998ec6f1eabac1e795f3b6e3ee639ea, it's better for the source code to have as few dependencies on our specific build system as possible. Since jpeglib.h, jinclude.h, jerror.h, and turbojpeg.h can be installed in system include directories, it's also better for internal references to those headers to resolve internally first, to avoid potential conflicts between the system-installed version of libjpeg-turbo and the version being built. (Such conflicts would never have occurred with our build system, but they might have occurred due to misintegration with a downstream build system.)
2a0c8627	2025-04-03T13:05:01	Fuzz: Fix -Wsign-compare warnings
9b01f5a0	2024-09-14T11:56:14	TJ: Add func/method for computing xformed buf size
437a4993	2024-09-12T21:11:00	Merge branch 'main' into dev
bcbca8b9	2024-09-12T20:55:39	Fuzz: Calc. xformed buf size based on dst. subsamp (oversight from b3f0abe377f2dd83396c9d0d4176f684c122af3f)
c519d7b6	2024-09-05T11:10:44	Don't ignore JPEG buf size with TJPARAM_NOREALLOC Since the introduction of TJFLAG_NOREALLOC in libjpeg-turbo 1.2.x, the TurboJPEG C API documentation has (confusingly) stated that: - if the JPEG buffer pointer points to a pre-allocated buffer, then the JPEG buffer size must be specified, and - the JPEG buffer size should be specified if the JPEG buffer is pre-allocated to an arbitrary size. The documentation never explicitly stated that the JPEG buffer size should be specified if the JPEG buffer is pre-allocated to a worst-case size, but since focus does not imply exclusion, it also never explicitly stated the reverse. Furthermore, the documentation never stated that this was contingent upon TJPARAM_NOREALLOC/TJFLAG_NOREALLOC. However, effectively the compression and lossless transformation functions ignored the JPEG buffer size(s) passed to them, and assumed that the JPEG buffer(s) had been allocated to a worst-case size, if TJPARAM_NOREALLOC/TJFLAG_NOREALLOC was set. This behavior was an accidental and undocumented throwback to libjpeg-turbo 1.1.x, in which the tjCompress() function provided no way to specify the JPEG buffer size. It was always a bad idea for applications to rely upon that behavior (although our own TJBench application unfortunately did.) However, if such applications exist in the wild, the new behavior would constitute a breaking change, so it has been introduced only into libjpeg-turbo 3.1.x and only into TurboJPEG 3 API functions. The previous behavior has been retained when calling functions from the TurboJPEG 2.1.x API and prior versions. Did I mention that APIs are hard?
b3f0abe3	2024-09-06T10:23:02	TJ: Calc. xformed buf sizes based on dst. subsamp With respect to tj3Transform(), this addresses an oversight from bb1d540a807783a3db8b85bab2993d70b1330287. Note to self: A convenience function/method for computing the worst-case transformed JPEG size for a particular transform would be nice.
562ad761	2024-08-19T10:06:59	OSS-Fuzz: More MSan fixes We need to use tj3Alloc() (which, when ZERO_BUFFERS is defined, calls calloc() instead of malloc()) to allocate all destination buffers. Otherwise, if the compression/decompression/transform operation fails, then the buffer checksum (which is computed to prevent the compiler from optimizing out the whole test, since the destination buffer is never used otherwise) will depend upon values in the destination buffer that were never written, and MSan will complain.
488d42a8	2024-08-16T12:12:09	OSS-Fuzz: Define ZERO_BUFFERS for MSan build ... and use tj3Alloc() to allocate compression/transformation destination buffers.
b4336c3a	2024-08-13T15:41:54	Work around valgrind/MSan SIMD false positives Referring to https://sourceforge.net/p/libjpeg-turbo/bugs/48, https://sourceforge.net/p/libjpeg-turbo/bugs/82, #15, #238, #253, and #619, valgrind and MSan have failed to properly detect data initialization by libjpeg-turbo's x86 SIMD extensions for the entire 14 years that libjpeg-turbo has been a project, resulting in false positives unless libjpeg-turbo is built with WITH_SIMD=0 or run with JSIMD_FORCENONE=1. This commit introduces a new C preprocessor macro (ZERO_BUFFERS) that, if set, causes libjpeg-turbo to zero certain buffers in order to work around the specific valgrind/MSan test failures caused by the aforementioned false positives. This allows us to more closely approximate the production configuration of libjpeg-turbo when testing with valgrind or MSan. Closes #781
ebca79d5	2023-07-25T16:46:07	xform fuzz: Test optimized baseline entropy coding Because of d011622f4b5b2c3f0141e93fc3e1da6169915c18, optimized baseline entropy coding wasn't actually being tested.
d011622f	2023-07-06T10:29:27	Restore xform fuzzer behavior from before c8d52f1c The intent was for the final transform operation to be the same as the first transform operation but without TJXOPT_COPYNONE or TJPARAM_NOREALLOC. Unrolling the transform operations in c8d52f1c4c7480277b91420c27b2548d4c8e9043 accidentally changed that.
89528757	2023-07-05T15:35:21	xform fuzz: Use src subsamp to calc dst buf size Referring to https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60379 there are some specially-crafted malformed JPEG images that, when transformed to grayscale, will exceed the worst-case transformed grayscale JPEG image size. This is similar in nature to the issue fixed by c8d52f1c4c7480277b91420c27b2548d4c8e9043, except that in this case, the issue occurs regardless of the amount of metadata in the source image. Also, the tj3Transform() function, the Java_org_libjpegturbo_turbojpeg_TJTransformer_transform() JNI function, and TJBench were behaving correctly in this case, because the TurboJPEG API documentation specifies that the source image's subsampling type should be used when computing the worst-case transformed JPEG image size. (However, only the Java API documentation specified that. Oops. The C API documentation now does as well.) The documented usage mitigates the issue, and only the transform fuzzer did not adhere to that. Thus, this was an issue with the fuzzer itself rather than an issue with the library.
30c21e55	2023-07-05T13:51:35	OSS-Fuzz: Ignore tj3DecompressHeader() return val Unlike its predecessors, tj3DecompressHeader() does not fail if passed a JPEG image with an unknown subsampling type. This led me to believe that it was OK for the fuzzers to abort if tj3DecompressHeader() returned an error. However, there are apparently some malformed JPEG images that can expose issues in libjpeg-turbo while also causing tj3DecompressHeader() to complain about header errors. Thus, it is best to ignore the return value of tj3DecompressHeader(), as the fuzzers in libjpeg-turbo 2.1.x and prior did.
02b074fd	2023-07-01T08:09:23	xform fuzz: Use only xform opts to set entropy alg This is subtle, but the tj3DecompressHeader() function sets the values of TJPARAM_ARITHMETIC, TJPARAM_OPTIMIZE, and TJPARAM_PROGRESSIVE. Unless we unset those values, the entropy algorithm used in the transformed JPEG image will be determined by the union of the parameter values and the transform options, which isn't what we want.
c8d52f1c	2023-06-26T11:53:03	tj3Transform: Calc dst buf size from xformed dims When used with TJPARAM_NOREALLOC and with TJXOP_TRANSPOSE, TJXOP_TRANSVERSE, TJXOP_ROT90, or TJXOP_ROT270, tj3Transform() incorrectly based the destination buffer size for a transform on the source image dimensions rather than the transformed image dimensions. This was apparently a long-standing bug that had existed in the tjTransform() function since its inception. As initially implemented in the evolving libjpeg-turbo v1.2 code base, tjTransform() required dstSizes[i] to be set regardless of whether TJFLAG_NOREALLOC (the predecessor to TJPARAM_NOREALLOC) was set. ff78e37595c8462f64fd100f928aa1d08539527e, which was introduced later in the evolving libjpeg-turbo v1.2 code base, removed that requirement and planted the seed for the bug. However, the bug was not activated until 9b49f0e4c77c727648c6d3a4915eefdf5436de4a was introduced still later in the evolving libjpeg-turbo v1.2 code base, adding a subsampling type argument to the (new at the time) tjBufSize() function and thus making the width and height arguments no longer commutative. The bug opened up the possibility that a JPEG source image could cause tj3Transform() to overflow the destination buffer for a transform if all of the following were true: - The JPEG source image used 4:2:2, 4:4:0, 4:1:1, or 4:4:1 subsampling. (These are the only subsampling types for which the width and height arguments to tj3JPEGBufSize() are not commutative.) - The width and height of the JPEG source image were such that tj3JPEGBufSize(height, width, subsamplingType) returned a smaller value than tj3JPEGBufSize(width, height, subsamplingType). - The JPEG source image contained enough metadata that the size of the transformed image was larger than tj3JPEGBufSize(height, width, subsamplingType). - TJPARAM_NOREALLOC was set. - TJXOP_TRANSPOSE, TJXOP_TRANSVERSE, TJXOP_ROT90, or TJXOP_ROT270 was used. - TJXOPT_COPYNONE was not set. - TJXOPT_CROP was not set. - The calling program allocated tj3JPEGBufSize(height, width, subsamplingType) bytes for the destination buffer, as the API documentation instructs. The API documentation cautions that JPEG source images containing a large amount of extraneous metadata (EXIF, IPTC, ICC, etc.) cannot reliably be transformed if TJPARAM_NOREALLOC is set and TJXOPT_COPYNONE is not set. Irrespective of the bug, there are still cases in which a JPEG source image with a large amount of metadata can, when transformed, exceed the worst-case transformed JPEG image size. For instance, if you try to losslessly crop a JPEG image with 3 kB of EXIF data to 16x16 pixels, then you are guaranteed to exceed the worst-case 16x16 JPEG image size unless you discard the EXIF data. Even without the bug, tj3Transform() will still fail with "Buffer passed to JPEG library is too small" when attempting to transform JPEG source images that meet the aforementioned criteria. The bug is that the function segfaults rather than failing gracefully, but the chances of that occurring in a real-world application are very slim. Any real-world application developers who attempted to transform arbitrary JPEG source images with TJPARAM_NOREALLOC set would very quickly realize that they cannot reliably do that without also setting TJXOPT_COPYNONE. Thus, I posit that the actual risk posed by this bug is low. Applications such as web browsers that are the most exposed to security risks from arbitrary JPEG source images do not use the TurboJPEG lossless transform feature. (None of those applications even use the TurboJPEG API, to the best of my knowledge, and the public libjpeg API has no equivalent transform function.) Our only command-line interface to the tj3Transform() function, TJBench, was not exposed to the bug because it had a compatible bug whereby it allocated the JPEG destination buffer to the same size that tj3Transform() erroneously expected. The TurboJPEG Java API was also not exposed to the bug because of a similar compatible bug in the Java_org_libjpegturbo_turbojpeg_TJTransformer_transform() JNI function. (This commit fixes both compatible bugs.) In short, best practices for tj3Transform() are to use TJPARAM_NOREALLOC only with JPEG source images that are known to be free of metadata (such as images generated by tj3Compress()) or to use TJXOPT_COPYNONE along with TJPARAM_NOREALLOC. Still, however, the function shouldn't segfault as long as the calling program allocates the suggested amount of space for the JPEG destination buffer. Usability notes: tj3Transform() could hypothetically require dstSizes[i] to be set regardless of the value of TJPARAM_NOREALLOC, but there are usability pitfalls either way. The main pitfall I sought to avoid with ff78e37595c8462f64fd100f928aa1d08539527e was a calling program failing to set dstSizes[i] at all, thus leaving its value undefined. It could be argued that requiring dstSizes[i] to be set in all cases is more consistent, but it could also be argued that not requiring it to be set when TJPARAM_NOREALLOC is set is more user-proof. tj3Transform() could also hypothetically set TJXOPT_COPYNONE automatically when TJPARAM_NOREALLOC is set, but that could lead to user confusion. Ultimately, I would like to address these issues in TurboJPEG v4 by using managed buffer objects, but that would be an extensive overhaul.
fd93d98a	2023-01-28T12:13:11	Fix i386 transform fuzzer build Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=55447
fc01f467	2023-01-05T06:36:46	TurboJPEG 3 API overhaul (ChangeLog update forthcoming) - Prefix all function names with "tj3" and remove version suffixes from function names. (Future API overhauls will increment the prefix to "tj4", etc., thus retaining backward API/ABI compatibility without versioning each individual function.) - Replace stateless boolean flags (including TJFLAG_ARITHMETIC and TJFLAG_LOSSLESS, which were never released) with stateful integer parameters, the value of which persists between function calls. * Use parameters for the JPEG quality and subsampling as well, in order to eliminate the awkwardness of specifying function arguments that weren't relevant for lossless compression. * tj3DecompressHeader() now stores all relevant information about the JPEG image, including the width, height, subsampling type, entropy coding type, etc. in parameters rather than returning that information in its arguments. * TJFLAG_LIMITSCANS has been reimplemented as an integer parameter (TJPARAM_SCANLIMIT) that allows the number of scans to be specified. - Use the const keyword for all pointer arguments to unmodified buffers, as well as for both dimensions of 2D pointers. Addresses #395. - Use size_t rather than unsigned long to represent buffer sizes, since unsigned long is a 32-bit type on Windows. Addresses #24. - Return 0 from all buffer size functions if an error occurs, rather than awkwardly trying to return -1 in an unsigned data type. - Implement 12-bit and 16-bit data precision using dedicated compression, decompression, and image I/O functions/methods. * Suffix the names of all data-precision-specific functions with 8, 12, or 16. * Because the YUV functions are intended to be used for video, they are currently only implemented with 8-bit data precision, but they can be expanded to 12-bit data precision in the future, if necessary. * Extend TJUnitTest and TJBench to test 12-bit and 16-bit data precision, using a new -precision option. * Add appropriate regression tests for all of the above to the 'test' target. * Extend tjbenchtest to test 12-bit and 16-bit data precision, and add separate 'tjtest12' and 'tjtest16' targets. * BufferedImage I/O in the Java API is currently limited to 8-bit data precision, since the BufferedImage class does not straightforwardly support higher data precisions. * Extend the PPM reader to convert 12-bit and 16-bit PBMPLUS files to grayscale or CMYK pixels, as it already does for 8-bit files. - Properly accommodate lossless JPEG using dedicated parameters (TJPARAM_LOSSLESS, TJPARAM_LOSSLESSPSV, and TJPARAM_LOSSLESSPT), rather than using a flag and awkwardly repurposing the JPEG quality. Update TJBench to properly reflect whether a JPEG image is lossless. - Re-organize the TJBench usage screen. - Update the Java docs using Java 11, to improve the formatting and eliminate HTML frames. - Use the accurate integer DCT algorithm by default for both compression and decompression, since the "fast" algorithm is a legacy feature, it does not pass the ISO compliance tests, and it is not actually faster on modern x86 CPUs. Remove the -accuratedct option from TJBench and TJExample. - Re-implement the 'tjtest' target using a CMake script that enables the appropriate tests, depending on the data precision and whether or not the Java API is part of the build. - Consolidate the C and Java versions of tjbenchtest into one script. - Consolidate the C and Java versions of tjexampletest into one script. - Combine all initialization functions into a single function (tj3Init()) that accepts an integer parameter specifying the subsystems to initialize. - Enable decompression scaling explicitly, using a new function/method (tj3SetScalingFactor()/TJDecompressor.setScalingFactor()), rather than implicitly using awkward "desired width"/"desired height" parameters. - Introduce a new macro/constant (TJUNSCALED/TJ.UNSCALED) that maps to a scaling factor of 1/1. - Implement partial image decompression, using a new function/method (tj3SetCroppingRegion()/TJDecompressor.setCroppingRegion()) and TJBench option (-crop). Extend tjbenchtest to test the new feature. Addresses #1. - Allow the JPEG colorspace to be specified explicitly when compressing, using a new parameter (TJPARAM_COLORSPACE). This allows JPEG images with the RGB and CMYK colorspaces to be created. - Remove the error/difference image feature from TJBench. Identical images to the ones that TJBench created can be generated using ImageMagick with 'magick composite <original_image> <output_image> -compose difference <diff_image>' - Handle JPEG images with unknown subsampling types. TJPARAM_SUBSAMP is set to TJSAMP_UNKNOWN (== -1) for such images, but they can still be decompressed fully into packed-pixel images or losslessly transformed (with the exception of lossless cropping.) They cannot be partially decompressed or decompressed into planar YUV images. Note also that TJBench, due to its lack of support for imperfect transforms, requires that the subsampling type be known when rotating, flipping, or transversely transposing an image. Addresses #436 - The Java version of TJBench now has identical functionality to the C version. This was accomplished by (somewhat hackishly) calling the TurboJPEG C image I/O functions through JNI and copying the pixels between the C heap and the Java heap. - Add parameters (TJPARAM_RESTARTROWS and TJPARAM_RESTARTBLOCKS) and a TJBench option (-restart) to allow the restart marker interval to be specified when compressing. Eliminate the undocumented TJ_RESTART environment variable. - Add a parameter (TJPARAM_OPTIMIZE), a transform option (TJOPT_OPTIMIZE), and a TJBench option (-optimize) to allow optimized baseline Huffman coding to be specified when compressing. Eliminate the undocumented TJ_OPTIMIZE environment variable. - Add parameters (TJPARAM_XDENSITY, TJPARAM_DENSITY, and TJDENSITYUNITS) to allow the pixel density to be specified when compressing or saving a Windows BMP image and to be queried when decompressing or loading a Windows BMP image. Addresses #77. - Refactor the fuzz targets to use the new API. * Extend decompression coverage to 12-bit and 16-bit data precision. * Replace the awkward cjpeg12 and cjpeg16 targets with proper TurboJPEG-based compress12, compress12-lossless, and compress16-lossless targets - Fix innocuous UBSan warnings uncovered by the new fuzzers. - Implement previous versions of the TurboJPEG API by wrapping the new functions (tested by running the 2.1.x versions of TJBench, via tjbenchtest, and TJUnitTest against the new implementation.) * Remove all JNI functions for deprecated Java methods and implement the deprecated methods using pure Java wrappers. It should be understood that backward API compatibility in Java applies only to the Java classes and that one cannot mix and match a JAR file from one version of libjpeg-turbo with a JNI library from another version. - tj3Destroy() now silently accepts a NULL handle. - tj3Alloc() and tj3Free() now return/accept void pointers, as malloc() and free() do. - The image I/O functions now accept a TurboJPEG instance handle, which is used to transmit/receive parameters and to receive error information. Closes #517
6002720c	2022-11-15T23:10:35	TurboJPEG: Opt. enable arithmetic entropy coding
e0606daf	2021-04-21T14:49:06	TurboJPEG: Update JPEG buf ptrs on comp/xform err When using the in-memory destination manager, it is necessary to explicitly call the destination manager's term_destination() method if an error occurs. That method is called by jpeg_finish_compress() but not by jpeg_abort_compress(). This fixes a potential double free() that could occur if tjCompress*() or tjTransform() returned an error and the calling application tried to clean up a JPEG buffer that was dynamically re-allocated by one of those functions.
3e68a5ee	2021-04-12T14:37:43	jchuff.c: Fix MSan error Certain rare malformed input images can cause the Huffman encoder to generate a value for nbits that corresponds to an uninitialized member of the DC code table. The ramifications of this are minimal and would basically amount to a different bogus JPEG image being generated from a particular bogus input image.
5536ace1	2021-04-05T21:12:29	OSS-Fuzz: Fix C++11 compiler warnings in targets
c81e91e8	2021-04-05T16:08:22	TurboJPEG: New flag for limiting prog JPEG scans This also fixes timeouts reported by OSS-Fuzz.
6ad658be	2021-04-02T14:50:35	OSS-Fuzz: Build fuzz targets using C++ compiler Otherwise, the targets will require libstdc++, the i386 version of which is not available in the OSS-Fuzz runtime environment. The OSS-Fuzz build environment passes -stdlib:libc++ in the CXXFLAGS environment variable in order to mitigate this issue, since the runtime environment has the i386 version of libc++, but using that compiler flag requires using the C++ compiler.

f74989d8

2025-09-25T11:32:45

Clean up #include directives This is subtle, but #include <header.h> searches directories specified with -I, then system include directories. #include "header.h" searches the current source directory, then directories specified with -I, then system include directories. Using bracketed #include directives for jpeglib.h, jinclude.h, jerror.h, cdjpeg.h, and turbojpeg.h only worked because the build system explicitly passed -I{source_directory}/src/ to the compiler. Referring to 51cee0362998ec6f1eabac1e795f3b6e3ee639ea, it's better for the source code to have as few dependencies on our specific build system as possible. Since jpeglib.h, jinclude.h, jerror.h, and turbojpeg.h can be installed in system include directories, it's also better for internal references to those headers to resolve internally first, to avoid potential conflicts between the system-installed version of libjpeg-turbo and the version being built. (Such conflicts would never have occurred with our build system, but they might have occurred due to misintegration with a downstream build system.)

2a0c8627

2025-04-03T13:05:01

Fuzz: Fix -Wsign-compare warnings

9b01f5a0

2024-09-14T11:56:14

TJ: Add func/method for computing xformed buf size

437a4993

2024-09-12T21:11:00

Merge branch 'main' into dev

bcbca8b9

2024-09-12T20:55:39

Fuzz: Calc. xformed buf size based on dst. subsamp (oversight from b3f0abe377f2dd83396c9d0d4176f684c122af3f)

c519d7b6

2024-09-05T11:10:44

Don't ignore JPEG buf size with TJPARAM_NOREALLOC Since the introduction of TJFLAG_NOREALLOC in libjpeg-turbo 1.2.x, the TurboJPEG C API documentation has (confusingly) stated that: - if the JPEG buffer pointer points to a pre-allocated buffer, then the JPEG buffer size must be specified, and - the JPEG buffer size should be specified if the JPEG buffer is pre-allocated to an arbitrary size. The documentation never explicitly stated that the JPEG buffer size should be specified if the JPEG buffer is pre-allocated to a worst-case size, but since focus does not imply exclusion, it also never explicitly stated the reverse. Furthermore, the documentation never stated that this was contingent upon TJPARAM_NOREALLOC/TJFLAG_NOREALLOC. However, effectively the compression and lossless transformation functions ignored the JPEG buffer size(s) passed to them, and assumed that the JPEG buffer(s) had been allocated to a worst-case size, if TJPARAM_NOREALLOC/TJFLAG_NOREALLOC was set. This behavior was an accidental and undocumented throwback to libjpeg-turbo 1.1.x, in which the tjCompress() function provided no way to specify the JPEG buffer size. It was always a bad idea for applications to rely upon that behavior (although our own TJBench application unfortunately did.) However, if such applications exist in the wild, the new behavior would constitute a breaking change, so it has been introduced only into libjpeg-turbo 3.1.x and only into TurboJPEG 3 API functions. The previous behavior has been retained when calling functions from the TurboJPEG 2.1.x API and prior versions. Did I mention that APIs are hard?

b3f0abe3

2024-09-06T10:23:02

TJ: Calc. xformed buf sizes based on dst. subsamp With respect to tj3Transform(), this addresses an oversight from bb1d540a807783a3db8b85bab2993d70b1330287. Note to self: A convenience function/method for computing the worst-case transformed JPEG size for a particular transform would be nice.

562ad761

2024-08-19T10:06:59

OSS-Fuzz: More MSan fixes We need to use tj3Alloc() (which, when ZERO_BUFFERS is defined, calls calloc() instead of malloc()) to allocate all destination buffers. Otherwise, if the compression/decompression/transform operation fails, then the buffer checksum (which is computed to prevent the compiler from optimizing out the whole test, since the destination buffer is never used otherwise) will depend upon values in the destination buffer that were never written, and MSan will complain.

488d42a8

2024-08-16T12:12:09

OSS-Fuzz: Define ZERO_BUFFERS for MSan build ... and use tj3Alloc() to allocate compression/transformation destination buffers.

b4336c3a

2024-08-13T15:41:54

Work around valgrind/MSan SIMD false positives Referring to https://sourceforge.net/p/libjpeg-turbo/bugs/48, https://sourceforge.net/p/libjpeg-turbo/bugs/82, #15, #238, #253, and #619, valgrind and MSan have failed to properly detect data initialization by libjpeg-turbo's x86 SIMD extensions for the entire 14 years that libjpeg-turbo has been a project, resulting in false positives unless libjpeg-turbo is built with WITH_SIMD=0 or run with JSIMD_FORCENONE=1. This commit introduces a new C preprocessor macro (ZERO_BUFFERS) that, if set, causes libjpeg-turbo to zero certain buffers in order to work around the specific valgrind/MSan test failures caused by the aforementioned false positives. This allows us to more closely approximate the production configuration of libjpeg-turbo when testing with valgrind or MSan. Closes #781

ebca79d5

2023-07-25T16:46:07

xform fuzz: Test optimized baseline entropy coding Because of d011622f4b5b2c3f0141e93fc3e1da6169915c18, optimized baseline entropy coding wasn't actually being tested.

d011622f

2023-07-06T10:29:27

Restore xform fuzzer behavior from before c8d52f1c The intent was for the final transform operation to be the same as the first transform operation but without TJXOPT_COPYNONE or TJPARAM_NOREALLOC. Unrolling the transform operations in c8d52f1c4c7480277b91420c27b2548d4c8e9043 accidentally changed that.

89528757

2023-07-05T15:35:21

xform fuzz: Use src subsamp to calc dst buf size Referring to https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60379 there are some specially-crafted malformed JPEG images that, when transformed to grayscale, will exceed the worst-case transformed grayscale JPEG image size. This is similar in nature to the issue fixed by c8d52f1c4c7480277b91420c27b2548d4c8e9043, except that in this case, the issue occurs regardless of the amount of metadata in the source image. Also, the tj3Transform() function, the Java_org_libjpegturbo_turbojpeg_TJTransformer_transform() JNI function, and TJBench were behaving correctly in this case, because the TurboJPEG API documentation specifies that the source image's subsampling type should be used when computing the worst-case transformed JPEG image size. (However, only the Java API documentation specified that. Oops. The C API documentation now does as well.) The documented usage mitigates the issue, and only the transform fuzzer did not adhere to that. Thus, this was an issue with the fuzzer itself rather than an issue with the library.

30c21e55

2023-07-05T13:51:35

OSS-Fuzz: Ignore tj3DecompressHeader() return val Unlike its predecessors, tj3DecompressHeader() does not fail if passed a JPEG image with an unknown subsampling type. This led me to believe that it was OK for the fuzzers to abort if tj3DecompressHeader() returned an error. However, there are apparently some malformed JPEG images that can expose issues in libjpeg-turbo while also causing tj3DecompressHeader() to complain about header errors. Thus, it is best to ignore the return value of tj3DecompressHeader(), as the fuzzers in libjpeg-turbo 2.1.x and prior did.

02b074fd

2023-07-01T08:09:23

xform fuzz: Use only xform opts to set entropy alg This is subtle, but the tj3DecompressHeader() function sets the values of TJPARAM_ARITHMETIC, TJPARAM_OPTIMIZE, and TJPARAM_PROGRESSIVE. Unless we unset those values, the entropy algorithm used in the transformed JPEG image will be determined by the union of the parameter values and the transform options, which isn't what we want.

c8d52f1c

2023-06-26T11:53:03

tj3Transform: Calc dst buf size from xformed dims When used with TJPARAM_NOREALLOC and with TJXOP_TRANSPOSE, TJXOP_TRANSVERSE, TJXOP_ROT90, or TJXOP_ROT270, tj3Transform() incorrectly based the destination buffer size for a transform on the source image dimensions rather than the transformed image dimensions. This was apparently a long-standing bug that had existed in the tj*Transform() function since its inception. As initially implemented in the evolving libjpeg-turbo v1.2 code base, tjTransform() required dstSizes[i] to be set regardless of whether TJFLAG_NOREALLOC (the predecessor to TJPARAM_NOREALLOC) was set. ff78e37595c8462f64fd100f928aa1d08539527e, which was introduced later in the evolving libjpeg-turbo v1.2 code base, removed that requirement and planted the seed for the bug. However, the bug was not activated until 9b49f0e4c77c727648c6d3a4915eefdf5436de4a was introduced still later in the evolving libjpeg-turbo v1.2 code base, adding a subsampling type argument to the (new at the time) tjBufSize() function and thus making the width and height arguments no longer commutative. The bug opened up the possibility that a JPEG source image could cause tj3Transform() to overflow the destination buffer for a transform if all of the following were true: - The JPEG source image used 4:2:2, 4:4:0, 4:1:1, or 4:4:1 subsampling. (These are the only subsampling types for which the width and height arguments to tj3JPEGBufSize() are not commutative.) - The width and height of the JPEG source image were such that tj3JPEGBufSize(height, width, subsamplingType) returned a smaller value than tj3JPEGBufSize(width, height, subsamplingType). - The JPEG source image contained enough metadata that the size of the transformed image was larger than tj3JPEGBufSize(height, width, subsamplingType). - TJPARAM_NOREALLOC was set. - TJXOP_TRANSPOSE, TJXOP_TRANSVERSE, TJXOP_ROT90, or TJXOP_ROT270 was used. - TJXOPT_COPYNONE was not set. - TJXOPT_CROP was not set. - The calling program allocated tj3JPEGBufSize(height, width, subsamplingType) bytes for the destination buffer, as the API documentation instructs. The API documentation cautions that JPEG source images containing a large amount of extraneous metadata (EXIF, IPTC, ICC, etc.) cannot reliably be transformed if TJPARAM_NOREALLOC is set and TJXOPT_COPYNONE is not set. Irrespective of the bug, there are still cases in which a JPEG source image with a large amount of metadata can, when transformed, exceed the worst-case transformed JPEG image size. For instance, if you try to losslessly crop a JPEG image with 3 kB of EXIF data to 16x16 pixels, then you are guaranteed to exceed the worst-case 16x16 JPEG image size unless you discard the EXIF data. Even without the bug, tj3Transform() will still fail with "Buffer passed to JPEG library is too small" when attempting to transform JPEG source images that meet the aforementioned criteria. The bug is that the function segfaults rather than failing gracefully, but the chances of that occurring in a real-world application are very slim. Any real-world application developers who attempted to transform arbitrary JPEG source images with TJPARAM_NOREALLOC set would very quickly realize that they cannot reliably do that without also setting TJXOPT_COPYNONE. Thus, I posit that the actual risk posed by this bug is low. Applications such as web browsers that are the most exposed to security risks from arbitrary JPEG source images do not use the TurboJPEG lossless transform feature. (None of those applications even use the TurboJPEG API, to the best of my knowledge, and the public libjpeg API has no equivalent transform function.) Our only command-line interface to the tj3Transform() function, TJBench, was not exposed to the bug because it had a compatible bug whereby it allocated the JPEG destination buffer to the same size that tj3Transform() erroneously expected. The TurboJPEG Java API was also not exposed to the bug because of a similar compatible bug in the Java_org_libjpegturbo_turbojpeg_TJTransformer_transform() JNI function. (This commit fixes both compatible bugs.) In short, best practices for tj3Transform() are to use TJPARAM_NOREALLOC only with JPEG source images that are known to be free of metadata (such as images generated by tj3Compress*()) or to use TJXOPT_COPYNONE along with TJPARAM_NOREALLOC. Still, however, the function shouldn't segfault as long as the calling program allocates the suggested amount of space for the JPEG destination buffer. Usability notes: tj3Transform() could hypothetically require dstSizes[i] to be set regardless of the value of TJPARAM_NOREALLOC, but there are usability pitfalls either way. The main pitfall I sought to avoid with ff78e37595c8462f64fd100f928aa1d08539527e was a calling program failing to set dstSizes[i] at all, thus leaving its value undefined. It could be argued that requiring dstSizes[i] to be set in all cases is more consistent, but it could also be argued that not requiring it to be set when TJPARAM_NOREALLOC is set is more user-proof. tj3Transform() could also hypothetically set TJXOPT_COPYNONE automatically when TJPARAM_NOREALLOC is set, but that could lead to user confusion. Ultimately, I would like to address these issues in TurboJPEG v4 by using managed buffer objects, but that would be an extensive overhaul.

fd93d98a

2023-01-28T12:13:11

Fix i386 transform fuzzer build Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=55447

fc01f467

2023-01-05T06:36:46

TurboJPEG 3 API overhaul (ChangeLog update forthcoming) - Prefix all function names with "tj3" and remove version suffixes from function names. (Future API overhauls will increment the prefix to "tj4", etc., thus retaining backward API/ABI compatibility without versioning each individual function.) - Replace stateless boolean flags (including TJ*FLAG_ARITHMETIC and TJ*FLAG_LOSSLESS, which were never released) with stateful integer parameters, the value of which persists between function calls. * Use parameters for the JPEG quality and subsampling as well, in order to eliminate the awkwardness of specifying function arguments that weren't relevant for lossless compression. * tj3DecompressHeader() now stores all relevant information about the JPEG image, including the width, height, subsampling type, entropy coding type, etc. in parameters rather than returning that information in its arguments. * TJ*FLAG_LIMITSCANS has been reimplemented as an integer parameter (TJ*PARAM_SCANLIMIT) that allows the number of scans to be specified. - Use the const keyword for all pointer arguments to unmodified buffers, as well as for both dimensions of 2D pointers. Addresses #395. - Use size_t rather than unsigned long to represent buffer sizes, since unsigned long is a 32-bit type on Windows. Addresses #24. - Return 0 from all buffer size functions if an error occurs, rather than awkwardly trying to return -1 in an unsigned data type. - Implement 12-bit and 16-bit data precision using dedicated compression, decompression, and image I/O functions/methods. * Suffix the names of all data-precision-specific functions with 8, 12, or 16. * Because the YUV functions are intended to be used for video, they are currently only implemented with 8-bit data precision, but they can be expanded to 12-bit data precision in the future, if necessary. * Extend TJUnitTest and TJBench to test 12-bit and 16-bit data precision, using a new -precision option. * Add appropriate regression tests for all of the above to the 'test' target. * Extend tjbenchtest to test 12-bit and 16-bit data precision, and add separate 'tjtest12' and 'tjtest16' targets. * BufferedImage I/O in the Java API is currently limited to 8-bit data precision, since the BufferedImage class does not straightforwardly support higher data precisions. * Extend the PPM reader to convert 12-bit and 16-bit PBMPLUS files to grayscale or CMYK pixels, as it already does for 8-bit files. - Properly accommodate lossless JPEG using dedicated parameters (TJ*PARAM_LOSSLESS, TJ*PARAM_LOSSLESSPSV, and TJ*PARAM_LOSSLESSPT), rather than using a flag and awkwardly repurposing the JPEG quality. Update TJBench to properly reflect whether a JPEG image is lossless. - Re-organize the TJBench usage screen. - Update the Java docs using Java 11, to improve the formatting and eliminate HTML frames. - Use the accurate integer DCT algorithm by default for both compression and decompression, since the "fast" algorithm is a legacy feature, it does not pass the ISO compliance tests, and it is not actually faster on modern x86 CPUs. * Remove the -accuratedct option from TJBench and TJExample. - Re-implement the 'tjtest' target using a CMake script that enables the appropriate tests, depending on the data precision and whether or not the Java API is part of the build. - Consolidate the C and Java versions of tjbenchtest into one script. - Consolidate the C and Java versions of tjexampletest into one script. - Combine all initialization functions into a single function (tj3Init()) that accepts an integer parameter specifying the subsystems to initialize. - Enable decompression scaling explicitly, using a new function/method (tj3SetScalingFactor()/TJDecompressor.setScalingFactor()), rather than implicitly using awkward "desired width"/"desired height" parameters. - Introduce a new macro/constant (TJUNSCALED/TJ.UNSCALED) that maps to a scaling factor of 1/1. - Implement partial image decompression, using a new function/method (tj3SetCroppingRegion()/TJDecompressor.setCroppingRegion()) and TJBench option (-crop). Extend tjbenchtest to test the new feature. Addresses #1. - Allow the JPEG colorspace to be specified explicitly when compressing, using a new parameter (TJ*PARAM_COLORSPACE). This allows JPEG images with the RGB and CMYK colorspaces to be created. - Remove the error/difference image feature from TJBench. Identical images to the ones that TJBench created can be generated using ImageMagick with 'magick composite <original_image> <output_image> -compose difference <diff_image>' - Handle JPEG images with unknown subsampling types. TJ*PARAM_SUBSAMP is set to TJ*SAMP_UNKNOWN (== -1) for such images, but they can still be decompressed fully into packed-pixel images or losslessly transformed (with the exception of lossless cropping.) They cannot be partially decompressed or decompressed into planar YUV images. Note also that TJBench, due to its lack of support for imperfect transforms, requires that the subsampling type be known when rotating, flipping, or transversely transposing an image. Addresses #436 - The Java version of TJBench now has identical functionality to the C version. This was accomplished by (somewhat hackishly) calling the TurboJPEG C image I/O functions through JNI and copying the pixels between the C heap and the Java heap. - Add parameters (TJ*PARAM_RESTARTROWS and TJ*PARAM_RESTARTBLOCKS) and a TJBench option (-restart) to allow the restart marker interval to be specified when compressing. Eliminate the undocumented TJ_RESTART environment variable. - Add a parameter (TJ*PARAM_OPTIMIZE), a transform option (TJ*OPT_OPTIMIZE), and a TJBench option (-optimize) to allow optimized baseline Huffman coding to be specified when compressing. Eliminate the undocumented TJ_OPTIMIZE environment variable. - Add parameters (TJ*PARAM_XDENSITY, TJ*PARAM_DENSITY, and TJ*DENSITYUNITS) to allow the pixel density to be specified when compressing or saving a Windows BMP image and to be queried when decompressing or loading a Windows BMP image. Addresses #77. - Refactor the fuzz targets to use the new API. * Extend decompression coverage to 12-bit and 16-bit data precision. * Replace the awkward cjpeg12 and cjpeg16 targets with proper TurboJPEG-based compress12, compress12-lossless, and compress16-lossless targets - Fix innocuous UBSan warnings uncovered by the new fuzzers. - Implement previous versions of the TurboJPEG API by wrapping the new functions (tested by running the 2.1.x versions of TJBench, via tjbenchtest, and TJUnitTest against the new implementation.) * Remove all JNI functions for deprecated Java methods and implement the deprecated methods using pure Java wrappers. It should be understood that backward API compatibility in Java applies only to the Java classes and that one cannot mix and match a JAR file from one version of libjpeg-turbo with a JNI library from another version. - tj3Destroy() now silently accepts a NULL handle. - tj3Alloc() and tj3Free() now return/accept void pointers, as malloc() and free() do. - The image I/O functions now accept a TurboJPEG instance handle, which is used to transmit/receive parameters and to receive error information. Closes #517

6002720c

2022-11-15T23:10:35

TurboJPEG: Opt. enable arithmetic entropy coding

e0606daf

2021-04-21T14:49:06

TurboJPEG: Update JPEG buf ptrs on comp/xform err When using the in-memory destination manager, it is necessary to explicitly call the destination manager's term_destination() method if an error occurs. That method is called by jpeg_finish_compress() but not by jpeg_abort_compress(). This fixes a potential double free() that could occur if tjCompress*() or tjTransform() returned an error and the calling application tried to clean up a JPEG buffer that was dynamically re-allocated by one of those functions.

3e68a5ee

2021-04-12T14:37:43

jchuff.c: Fix MSan error Certain rare malformed input images can cause the Huffman encoder to generate a value for nbits that corresponds to an uninitialized member of the DC code table. The ramifications of this are minimal and would basically amount to a different bogus JPEG image being generated from a particular bogus input image.

5536ace1

2021-04-05T21:12:29

OSS-Fuzz: Fix C++11 compiler warnings in targets

c81e91e8

2021-04-05T16:08:22

TurboJPEG: New flag for limiting prog JPEG scans This also fixes timeouts reported by OSS-Fuzz.

6ad658be

2021-04-02T14:50:35

OSS-Fuzz: Build fuzz targets using C++ compiler Otherwise, the targets will require libstdc++, the i386 version of which is not available in the OSS-Fuzz runtime environment. The OSS-Fuzz build environment passes -stdlib:libc++ in the CXXFLAGS environment variable in order to mitigate this issue, since the runtime environment has the i386 version of libc++, but using that compiler flag requires using the C++ compiler.

kc3-lang/libjpeg-turbo/fuzz/transform.cc

fuzz/transform.cc

Log