kc3-lang/libjpeg-turbo

Log

Author Commit Date CI Message
DRC 562ad761 2024-08-19T10:06:59 OSS-Fuzz: More MSan fixes We need to use tj3Alloc() (which, when ZERO_BUFFERS is defined, calls calloc() instead of malloc()) to allocate all destination buffers. Otherwise, if the compression/decompression/transform operation fails, then the buffer checksum (which is computed to prevent the compiler from optimizing out the whole test, since the destination buffer is never used otherwise) will depend upon values in the destination buffer that were never written, and MSan will complain.
DRC 488d42a8 2024-08-16T12:12:09 OSS-Fuzz: Define ZERO_BUFFERS for MSan build ... and use tj3Alloc() to allocate compression/transformation destination buffers.
DRC 0c23b0ad 2024-08-14T09:21:54 Various doc tweaks - "Optimized baseline entropy coding" = "Huffman table optimization" "Optimized baseline entropy coding" was meant to emphasize that the feature is only useful when generating baseline (single-scan lossy 8-bit-per-sample Huffman-coded) JPEG images, because it is automatically enabled when generating Huffman-coded progressive (multi-scan), 12-bit-per-sample, and lossless JPEG images. However, Huffman table optimization isn't actually an integral part of those non-baseline modes. You can forego Huffman table optimization with 12-bit data precision if you supply your own Huffman tables. The spec doesn't require it with progressive or lossless mode, either, although our implementation does. Furthermore, "baseline" describes more than just the type of entropy coding used. It was incorrect to say that optimized "baseline" entropy coding is automatically enabled for Huffman-coded progressive, 12-bit-per-sample, and lossless JPEG images, since those are clearly not baseline images. - "Progressive entropy coding" = "Progressive JPEG" "Progressive" describes more than just the type of entropy coding used. (In fact, both Huffman-coded and arithmetic-coded images can be progressive.) - Mention that TJPARAM_OPTIMIZE/TJ.PARAM_OPTIMIZE can be used with lossless transformation as well. - General wordsmithing - Formatting tweaks
DRC b4336c3a 2024-08-13T15:41:54 Work around valgrind/MSan SIMD false positives Referring to https://sourceforge.net/p/libjpeg-turbo/bugs/48, https://sourceforge.net/p/libjpeg-turbo/bugs/82, #15, #238, #253, and #619, valgrind and MSan have failed to properly detect data initialization by libjpeg-turbo's x86 SIMD extensions for the entire 14 years that libjpeg-turbo has been a project, resulting in false positives unless libjpeg-turbo is built with WITH_SIMD=0 or run with JSIMD_FORCENONE=1. This commit introduces a new C preprocessor macro (ZERO_BUFFERS) that, if set, causes libjpeg-turbo to zero certain buffers in order to work around the specific valgrind/MSan test failures caused by the aforementioned false positives. This allows us to more closely approximate the production configuration of libjpeg-turbo when testing with valgrind or MSan. Closes #781
DRC 8db03126 2024-08-02T08:45:36 example.c: Write correct dimensions to PPM header The dimensions in the PPM header of the output file generated by 'example decompress' were always 640 x 480, regardless of the size of the JPEG image being decompressed. Our regression tests (which this commit also fixes) missed the bug because they decompressed the 640 x 480 image generated by 'example compress'. Fixes #778
DRC 0566d51e 2024-07-09T17:18:53 GitHub Actions: Specify Monterey for macOS build The Big Sur hosted runner is no longer available.
DRC e287a357 2024-07-04T10:03:50 Build: Put gastest.o in ${CMAKE_BINARY_DIR}/simd Closes #776
DRC 51d021bf 2024-06-24T12:17:22 TurboJPEG: Fix 12-bit-per-sample arith-coded compr (Regression introduced by 7bb958b732e6b4f261595e2d1527d46964fe3aed) Because of 7bb958b732e6b4f261595e2d1527d46964fe3aed, the TurboJPEG compression and encoding functions no longer transfer the value of TJPARAM_OPTIMIZE into cinfo->data_precision unless the data precision is 8. The intent of that was to prevent using_std_huff_tables() from being called more than once when reusing the same compressor object to generate multiple 12-bit-per-sample JPEG images. However, because cinfo->optimize_coding is always set to TRUE by jpeg_set_defaults() if the data precision is 12, calling applications that use 12-bit data precision had to unset cinfo->optimize_coding if they set cinfo->arith_code after calling jpeg_set_defaults(). Because of 7bb958b732e6b4f261595e2d1527d46964fe3aed, the TurboJPEG API stopped doing that except with 8-bit data precision. Thus, attempting to generate a 12-bit-per-sample arithmetic-coded lossy JPEG image using the TurboJPEG API failed with "Requested features are incompatible." Since the compressor will always fail if cinfo->arith_code and cinfo->optimize_coding are both set, and since cinfo->optimize_coding has no relevance for arithmetic coding, the most robust and user-proof solution is for jinit_c_master_control() to set cinfo->optimize_coding to FALSE if cinfo->arith_code is TRUE. This commit also: - modifies TJBench so that it no longer reports that it is using optimized baseline entropy coding in modes where that setting is irrelevant, - amends the cjpeg documentation to clarify that -optimize is implied when specifying -progressive or '-precision 12' without -arithmetic, and - prevents jpeg_set_defaults() from uselessly checking the value of cinfo->arith_code immediately after it has been set to FALSE.
DRC bb3ab531 2024-06-19T17:27:01 cjpeg: Don't enable lossless until precision known jpeg_enable_lossless() checks the point transform value against the data precision, so we need to defer calling jpeg_enable_lossless() until after all command-line options have been parsed.
DRC a8aaaf5d 2024-06-21T10:58:17 cjpeg -verbose: Print PBMPLUS input file precision This gives the user a hint as to whether converting the input file into a JPEG file with a particular data precision will result in a loss of precision. Also modify usage.txt and the cjpeg man page to warn the user about that possibility.
DRC 94c64ead 2024-06-17T20:27:57 Various doc tweaks - "bits per component" = "bits per sample" Describing the data precision of a JPEG image using "bits per component" is technically correct, but "bits per sample" is the terminology that the JPEG-1 spec uses. Also, "bits per component" is more commonly used to describe the precision of packed-pixel formats (as opposed to "bits per pixel") rather than planar formats, in which all components are grouped together. - Unmention legacy display technologies. Colormapped and monochrome displays aren't a thing anymore, and even when they were still a thing, it was possible to display full-color images to them. In 1991, when JPEG decompression time was measured in minutes per megapixel, it made sense to keep a decompressed copy of JPEG images on disk, in a format that could be displayed without further color conversion (since color conversion was slow and memory-intensive.) In 2024, JPEG decompression time is measured in milliseconds per megapixel, and color conversion is even faster. Thus, JPEG images can be decompressed, displayed, and color-converted (if necessary) "on the fly" at speeds too fast for human vision to perceive. (In fact, your TV performs much more complicated decompression algorithms at least 60 times per second.) - Document that color quantization (and associated features), GIF input/output, Targa input/output, and OS/2 BMP input/output are legacy features. Legacy status doesn't necessarily mean that the features are deprecated. Rather, it is meant to discourage users from using features that may be of little or no benefit on modern machines (such as low-quality modes that had significant performance advantages in the early 1990s but no longer do) and that are maintained on a break/fix basis only. - General wordsmithing, grammar/punctuation policing, and formatting tweaks - Clarify which data precisions each cjpeg input format and each djpeg output format supports. - cjpeg.1: Remove unnecessary and impolitic statement about the -targa switch. - Adjust or remove performance claims to reflect the fact that: * On modern machines, the djpeg "-fast" switch has a negligible effect on performance. * There is a measurable difference between the performance of Floyd- Steinberg dithering and no dithering, but it is not likely perceptible to most users. * There is a measurable difference between the performance of 1-pass and 2-pass color quantization, but it is not likely perceptible to most users. * There is a measurable difference between the performance of full-color and grayscale output when decompressing a full-color JPEG image, but it is not likely perceptible to most users. * IDCT scaling does not necessarily improve performance. (It generally does if the scaling factor is <= 1/2 and generally doesn't if the scaling factor is > 1/2, at least on my machine. The performance claim made in jpeg-6b was probably invalidated when we merged the additional scaling factors from jpeg-7.) - Clarify which djpeg switches/output formats cannot be used when decompressing lossless JPEG images. - Remove djpeg hints, since those involve quality vs. speed tradeoffs that are no longer relevant for modern machines. - Remove documentation regarding using color quantization with 16-bit data precision. (Color quantization requires lossy mode.) - Java: Fix typos in TJDecompressor.decompress12() and TJDecompressor.decompress16() documentation. - jpegtran.1: Fix truncated paragraph In a man page, a single quote at the start of a line is interpreted as a macro. Closes #775 - libjpeg.txt: * Mention J16SAMPLE data type (oversight.) * Remove statement about extending jdcolor.c. (libjpeg-turbo is not quite as DIY as libjpeg once was.) * Remove paragraph about tweaking the various typedefs in jmorecfg.h. It is no longer relevant for modern machines. * Remove caveat regarding systems with ints less than 16 bits wide. (ANSI/ISO C requires an int to be at least 16 bits wide, and libjpeg-turbo has never supported non-ANSI compilers.) - usage.txt: * Add copyright header. * Document cjpeg -icc, -memdst, -report, -strict, and -version switches. * Document djpeg -icc, -maxscans, -memsrc, -report, -skip, -crop, -strict, and -version switches. * Document jpegtran -icc, -maxscans, -report, -strict, and -version switches.
DRC 2c5312fd 2024-06-13T21:16:20 cderror.h: Always include all img I/O err messages The 8-bit-per-sample image I/O modules have always been built with BMP, GIF, PBMPLUS, and Targa support, and the 12-bit-per-sample image I/O modules have always been built with only GIF and PBMPLUS support. In libjpeg-turbo 2.1.x and prior, cjpeg and djpeg were built with the same image format support as the image I/O modules. However, because of the multi-precision feature introduced in libjpeg-turbo 3.0.x, cjpeg and djpeg are now always built with support for all image formats. Thus, the error message table compiled into cjpeg and djpeg was a superset of the error message table compiled into the 12-bit-per-sample and 16-bit-per-sample image I/O modules. If, for example, the 12-bit-per-sample PPM writer threw JERR_PPM_COLORSPACE (== 15, because it was built with only GIF and PBMPLUS support), then djpeg interpreted that as JERR_GIF_COLORSPACE (== 15, because it was built with support for all image formats.) There was no chance of a buffer overrun, since the error message table lookup was performed against the table compiled into cjpeg and djpeg, which contained all possible entries. However, this issue caused an incorrect error message to be displayed by cjpeg or djpeg if a 12-bit-per-sample or 16-bit-per-sample image I/O module threw an error. This commit simply removes the *_SUPPORTED #ifdefs from cderror.h so that all image I/O error messages are always included in every instance of the image I/O error message table. Note that a similar issue could have also occurred with the 12-bit-per-sample and 16-bit-per-sample TurboJPEG image I/O functions, since the 12-bit-per-sample and 16-bit-per-sample image I/O modules supporting those functions are built with only PBMPLUS support whereas the library as a whole is built with BMP and PBMPLUS support.
DRC 3c17063e 2024-05-27T11:41:35 Guard against dupe SOF w/ incorrect source manager Referring to https://bugzilla.mozilla.org/show_bug.cgi?id=1898606, attempting to decompress a specially-crafted malformed JPEG image (specifically an image with a complete 12-bit Start Of Frame segment followed by an incomplete 8-bit Start Of Frame segment) using the default marker processor, buffered-image mode, and input prefetching triggered the following sequence of events: - When the 12-bit SOF segment was encountered (in the body of jpeg_read_header()), the marker processor's read_markers() method called the get_sof() function, which processed the 12-bit SOF segment and set cinfo->data_precision to 12. - If the application subsequently called jpeg_consume_input() in a loop to prefetch input data, and it didn't stop calling jpeg_consume_input() when the function returned JPEG_REACHED_SOS, then the 8-bit SOF segment was encountered in the body of jpeg_consume_input(). As a result, the marker processor's read_markers() method called get_sof(), which started to process the 8-bit SOF segment and set cinfo->data_precision to 8. - Since the 8-bit SOF segment was incomplete, the end of the JPEG data stream was encountered when get_sof() attempted to read the image height, width, and number of components. - If the fill_input_buffer() method in the application's custom source manager incorrectly returned FALSE in response to a prematurely- terminated JPEG data stream, then get_sof() returned FALSE while attempting to read the image height, width, and number of components (before the duplicate SOF check was reached.) That caused the default marker processor's read_markers() method, and subsequently jpeg_consume_input(), to return JPEG_SUSPENDED. - If the application failed to respond to the JPEG_SUSPENDED return value and subsequently attempted to call jpeg_read_scanlines(), then the data precision check in jpeg_read_scanlines() succeeded (because cinfo->data_precision was now 8.) However, because cinfo->data_precision had been 12 during the previous call to jpeg_start_decompress(), only the 12-bit version of the main controller was initialized, and the cinfo->main->process_data() method was undefined. Thus, a segfault occurred when jpeg_read_scanlines() attempted to invoke that method. Scenarios in which the issue was thwarted: 1. The default source managers handle a prematurely-terminated JPEG data stream by inserting a fake EOI marker into the data stream. Thus, when using one of those source managers, the INPUT_2BYTES() and INPUT_BYTE() macros (which get_sof() invokes to read the image height, width, and number of components) succeeded-- albeit with bogus data, since the fake EOI marker was read into those fields. The duplicate SOF check in get_sof() then failed, generating a fatal libjpeg error. 2. When using a custom source manager that correctly returns TRUE in response to a prematurely-terminated JPEG data stream, the aforementioned INPUT_2BYTES() and INPUT_BYTE() macros also succeeded (albeit with bogus data read from the previous bytes of the data stream), and the duplicate SOF check failed. 3. If the application did not prefetch input data, or if it stopped invoking jpeg_consume_input() when the function returned JPEG_REACHED_SOS, then the duplicate SOF segment was not read prior to the first call to jpeg_read_scanlines(). Thus, the data precision check in jpeg_read_scanlines() failed. If the application instead called jpeg12_read_scanlines() (that is, if it properly supported multiple data precisions), then the duplicate SOF segment was not read until the body of jpeg_finish_decompress(). At that point, its only negative effect was to cause jpeg_finish_decompress() to return FALSE before the duplicate SOF check was reached. In other words, this issue depended not only upon an incorrectly-written source manager but also upon a very specific sequence of API calls. It also depended upon the multi-precision feature introduced in libjpeg-turbo 3.0.x. When using an 8-bit-per-sample build of libjpeg-turbo 2.1.x, jpeg_read_header() failed with "Unsupported JPEG data precision 12" after the 12-bit SOF segment was processed. When using a 12-bit-per-sample build of libjpeg-turbo 2.1.x, the behavior was the same as if the application called jpeg12_read_scanlines() in Scenario 3 above. This commit simply moves the duplicate SOF check to the top of get_sof() so the check will fail before the marker processor attempts to read the duplicate SOF. It should be noted that this issue isn't a libjpeg-turbo bug per se, because it occurs only when the calling application does something it shouldn't. It is, rather, an issue of API hardening/caller-proofing.
DRC 46173635 2024-05-28T23:04:37 Remove jpeg_std_message_table[] extern sym comment jpeg_std_message_table[] was never a documented part of the libjpeg API, nor was it exposed in jpegint.h or the Windows libjpeg API DLL. Thus, it was never a good idea (nor was it even remotely necessary) for applications to use it. However, referring to #763 and #767, one application (RawTherapee) did use it. 34c055851ecb66a2d9bee1a3318c55cd9acd6586 hid the symbol, which broke the Un*x builds of RawTherapee. (RawTherapee already did the right thing on Windows, because jpeg_std_message_table[] was not exposed in the Windows libjpeg API DLL. Referring to https://github.com/Beep6581/RawTherapee/issues/7074, RawTherapee now does the right thing on Un*x platforms as well.) The comment in jerror.c indicated that Tom Lane intended the symbol to be external "just in case any applications want to refer to it directly." However, with respect to libjpeg-turbo, the comment was effectively already a lie on Windows. My choices at this point are: 1. Revert 34c055851ecb66a2d9bee1a3318c55cd9acd6586 and start exposing the symbol on Windows, thus making the comment true again. 2. Remove the comment. Option 1 would have created whiplash, since 3.0.3 has already been released with the symbol hidden, and that option would have further encouraged ill-advised behavior on the part of applications. Since the issue has already been fixed in RawTherapee, and since it is known to affect only that application, I chose Option 2. In my professional opinion, a code comment does not rise to the level of a contract between a library and a calling application. In other words, the comment did not make the symbol part of the API, even though the symbol was exposed on some platforms. Applications that use internal symbols (even the symbols defined in jpegint.h) do so at their own risk. There is no guarantee that those symbols will remain unchanged from release to release, as it is sometimes necessary to modify the internal (opaque) structures and methods in order to implement new features or even fix bugs. Some implementations of the libjpeg API (such as the one in Jpegli, for instance), do not provide those internal symbols at all. Best practices are for applications that use the internal libjpeg-turbo symbols to provide their own copy of libjpeg-turbo (for instance, via static linking), so they can manage any unexpected changes in those symbols. (In fact, that is how libjpeg was originally used. Application developers built it from source with compile-time options to exclude unneeded functionality. Thus, no two builds of libjpeg were guaranteed to be API/ABI-compatible. The idea of a libjpeg shared library that exposes a pseudo-standard ABI came later, and that has always been an awkward fit for an API that was intended to be modified at compile time to suit specific application needs. libjpeg-turbo's colorspace extensions are but one example, among many, of our attempts to reconcile that awkwardness while preserving backward compatibility with the aforementioned pseudo-standard ABI.)
DRC 9ddcae4a 2024-05-20T12:13:51 jpeglib.h: Document the need to include stdio.h libjpeg.txt documents the need to "include system headers that define at least the typedefs FILE and size_t" before including jpeglib.h. However, some software developers unfortunately assume that any downstream build failure is due to an upstream oversight (as if such an oversight would have gone unnoticed for 14 years in a library as ubiquitous as libjpeg-turbo) and "come in hot" with a proposal and arguments that have already been carefully considered and rejected multiple times (as opposed to grepping the documentation or searching existing issues and PRs to find out whether the upstream behavior is intentional.) Multiple issues and PRs (including #17, #18, #116, #737, and #766) have been filed regarding this, so this commit adds a comment to jpeglib.h in the hope of heading off future issues and PRs. I suspect that some people will still choose to waste my time by arguing about it, even though the decision was made by Tom Lane nearly 20 years before libjpeg-turbo even existed, the decision was supportable based on the needs of early 1990s computing systems, and reversing that decision would break backward API compatibility (which is one of the reasons that many downstream projects adopted libjpeg-turbo in the first place.) At least now, if someone posts an issue or PR about this again, I can just link to this commit and close the issue/PR without comment.
DRC bc491b16 2024-05-16T17:32:02 ChangeLog.md: Document previous commit
DRC 0fc7313e 2024-05-14T11:41:16 Don't traverse linked list when saving a marker If the calling application invokes jpeg_save_markers() to save a particular type of marker, then the save_marker() function will be invoked for every marker of that type that is encountered. Previously, only the head of the marker linked list was stored (in jpeg_decompress_struct), so save_marker() had to traverse the entire linked list before it could add a new marker to the tail of the list. That caused CPU usage to grow exponentially with the number of markers. Referring to #764, it is possible to create a JPEG image that contains an excessive number of markers. The specific reproducer that uncovered this issue is a specially-crafted 1-megabyte malformed JPEG image with tens of thousands of APP1 markers, which required approximately 30 seconds of CPU time (on a modern Intel processor) to process. However, it should also be possible to create a legitimate JPEG image that reproduces the issue (such as an image with tens of thousands of duplicate EXIF tags.) This commit introduces a new pointer (in jpeg_decomp_master, in order to preserve backward ABI compatibility) that is used to store the tail of the marker linked list whenever a marker is added to it. Thus, it is no longer necessary to traverse the list when adding a marker, and CPU usage will grow linearly rather than exponentially with the number of markers. Fixes #764
DRC 7fa4b5b7 2024-05-06T17:28:07 jerror.c: Silence MSan uninitialized value warning If an error manager instance is passed to jpeg_std_error(), then its format_message() method will point to the format_message() function in jerror.c. The format_message() function passes all eight values from the jpeg_error_mgr::msg_parm.i[] array as arguments to snprintf()/_snprintf_s(), even if the format string doesn't use all of those values. Subsequently invoking one of the ERREXIT[1-6]() macros will leave the unused values uninitialized, and if the -fsanitize-memory-param-retval option (introduced in Clang 14) is enabled (which it is by default in Clang 16 and later), then MSan will complain when the format_message() function tries to pass the uninitialized-but-unused values as function arguments. This commit modifies jpeg_std_error() so that it zeroes out the error manager instance passed to it, thus working around the warning as well as simplifying the code. Closes #761
DRC 3f43bb33 2024-05-06T13:03:46 Build: Don't use COMPONENT w/install(INCLUDES ...) (Regression introduced by 24e09baaf024e71841a92d30d0e763242ed959ef) The install() INCLUDES option is not an artifact option. It specifies a list of directories that will be added to the INTERFACE_INCLUDE_DIRECTORIES target property when the target is exported using the install() EXPORT option, which occurs when CMake package config files are generated. Specifying 'COMPONENT include' with the install() INCLUDES option caused the INTERFACE_INCLUDE_DIRECTORIES properties in our CMake package config files to contain '${_IMPORT_PREFIX}/COMPONENT', which caused errors of the form 'Imported target "libjpeg-turbo::XXXX" includes non-existent path' when downstream build systems attempted to include libjpeg-turbo using find_package(). Fixes #759 Closes #760
Kleis Auke Wolthuizen 24e09baa 2024-04-12T11:46:21 Build: Add COMPONENT to all install() commands This makes it possible for downstream packagers and other integrators of libjpeg-turbo to include only specific directories from the libjpeg-turbo installation (or to install specific directories under a different prefix, etc.) The names of the components correspond to the directories into which they will be installed. Refer to libvips/libvips#3931, #265, #338 Closes #756
DRC 6a522fcd 2024-05-02T14:07:45 jpegtran -drop: Ensure all quant tables defined It is possible to craft a malformed JPEG image in which all of the scans contain fewer components than the number of components specified in the Start Of Frame (SOF) segment. Attempting to use such an image as either an input image or a drop image with 'jpegtran -drop' caused a NULL dereference and subsequent segfault in transupp.c:adjust_quant(), so this commit adds appropriate checks to guard against that. Since the issue involved an interface that is only exposed on the jpegtran command line, it did not represent a security risk. 'jpegtran -drop' could not ever be used successfully with images such as the ones described above. This commit simply makes jpegtran fail gracefully rather than crash. Fixes #758
DRC 2dfe6c0f 2024-03-18T14:51:04 CI: Work around segfaults in ASan/MSan jobs Referring to actions/runner-images#9491, the sanitizers in LLVM 14 that ships with Ubuntu 22.04 are incompatible with high-entropy address space layout randomization (ASLR), which is enabled in the GitHub runners via their use of a newer kernel than ubuntu 22.04 uses.
DRC 710865cf 2024-03-18T12:56:42 Build: Don't explicitly set CMP0065 to NEW This is no longer necessary, because of 1644bdb7d2fac66cd0ce25adef7754e008b5bc1e.
DRC fe218ca1 2024-03-18T11:27:30 Build: Handle CMAKE_C_COMPILER_ID=AppleClang Because of 1644bdb7d2fac66cd0ce25adef7754e008b5bc1e, we are now effectively using the NEW behavior for all CMake policies introduced in all CMake versions up to and including CMake 3.28. The NEW behavior for CMP0025, introduced in CMake 3.0, sets CMAKE_C_COMPILER_ID to "AppleClang" instead of "Clang" when using Apple's variant of Clang (in Xcode), so we need to match all values of CMAKE_C_COMPILER_ID that contain "Clang". This fixes three issues: - -O2 was not replaced with -O3 in CMAKE_C_FLAGS_RELWITHDEBINFO. This was a minor issue, since -O3 is now the default in CMAKE_C_FLAGS_RELEASE, and we use CMAKE_BUILD_TYPE=Release in our official builds. - The build system erroneously set the default value of FLOATTEST8 and FLOATTEST12 to no-fp-contract when compiling for PowerPC or Arm using Apple Clang 14+ (effectively reverting 5b2beb4bc4f41dd9dd2a905cb931b8d5054d909b.) Because Clang 14+ now enables -ffp-contract=on by default, this issue caused floating point test failures unless FLOATTEST8 and FLOATTEST12 were overridden. - The build system set MD5_PPM_3x2_FLOAT_FP_CONTRACT as appropriate for GCC, not as appropriate for Clang (effectively reverting 47656a082091f9c9efda054674522513f4768c6c.) This also caused floating point test failures. Fixes #753 Closes #755
DRC dfde1f85 2024-03-08T12:09:23 Fix (and test) more Clang 14 compiler warnings -Woverlength-strings, -Wshift-negative-value, -Wsign-compare
DRC 905ec0fa 2024-03-08T10:29:27 Avoid tautological comparisons Several TurboJPEG functions store their return value in an unsigned long long intermediate and compare it against the maximum value of unsigned long or size_t in order to avoid integer overflow. However, such comparisons are tautological (always true, i.e. redundant) unless the size of unsigned long or size_t is less than the size of unsigned long long. Explicitly guarding the comparisons with #if avoids compiler warnings with -Wtautological-constant-in-range-compare in Clang and also makes it clear to the reader that the comparisons are only intended for 32-bit code. Refer to #752
DRC 34c05585 2024-03-06T15:12:31 Fix warnings with -Wmissing-variable-declarations
DRC 7bb958b7 2024-03-04T12:12:03 12-bit: Don't gen opt Huff tbls if tbls supplied (regression introduced by e8b40f3c2ba187ba95c13c3e8ce21c8534256df7) The documented behavior of the libjpeg API is to compute optimal Huffman tables when generating 12-bit lossy Huffman-coded JPEG images, unless the calling application supplies its own Huffman tables. However, e8b40f3c2ba187ba95c13c3e8ce21c8534256df7 and 96bc40c1b36775afdbad4ae05a6b3f48e2eebeb9 modified jinit_c_master_control() so that it always set cinfo->optimize_coding to TRUE when generarating 12-bit lossy Huffman-coded JPEG images, which prevented calling applications from supplying custom Huffman tables for such images. This commit modifies jinit_c_master_control() so that it only overrides cinfo->optimize_coding when generating 12-bit lossy Huffman-coded JPEG images if all Huffman table slots are empty or all slots contain default Huffman tables. Determining whether the latter is true requires using memcmp() to compare the allocated Huffman tables with the default Huffman tables, because: - The documented behavior of jpeg_set_defaults() is to initialize any empty Huffman table slot with the default Huffman table corresponding to that slot, regardless of the data precision. There is also no requirement that the data precision be specified prior to calling jpeg_set_defaults(). Thus, there is no reliable way to prevent jpeg_set_defaults() from initializing empty Huffman table slots with default Huffman tables, which are useless for 12-bit data precision. - There is no requirement that custom Huffman tables be defined prior to calling jpeg_set_defaults(). A calling application could call jpeg_set_defaults() and modify the values in the default Huffman tables rather than allocating new tables. Thus, there is no reliable way to detect whether the allocated Huffman tables contain default values without comparing the tables with the default Huffman tables. Fortunately, comparing the allocated Huffman tables with the default Huffman tables is the last stop on the logic train, so it won't happen unless cinfo->data_precision == 12, cinfo->arith_code == FALSE, cinfo->optimize_coding == FALSE, and one or more Huffman tables are allocated. (If the compressor object is reused, this ensures that the full comparison will be performed at most once.) Custom Huffman tables will be flagged as non-default when the first non-default value is encountered, and the worst case (comparing 400 bytes) is very fast on modern CPUs anyhow. Fixes #751
DRC 3202feb0 2024-02-29T16:10:20 x86-64 SIMD: Support CET if C compiler enables it - Detect at configure time, via the __CET__ C preprocessor macro, whether the C compiler will include either indirect branch tracking (IBT) or shadow stack support, and define a NASM macro (__CET__) if so. - Modify the x86-64 SIMD code so that it includes appropriate endbr64 instructions (to support IBT) and an appropriate .note.gnu.property section (to support both IBT and shadow stack) when __CET__ is defined. Closes #350
DRC 13355475 2024-02-29T12:18:49 x86 SIMD: Capitalize all instruction-like macros (to improve code readability)
DRC 26fc07c8 2024-02-08T12:03:37 Build: Set MSVC run-time lib based on IDE config
Alyssa Ross b6ee1016 2024-01-29T17:18:38 Build: Fix tests w/ emulators that don't check CWD While QEMU will run executables from the current working directory, other emulators may not. It is more reliable to pass the full executable path to the emulator. The add_test(NAME ... COMMAND ...) syntax automatically invokes the emulator (e.g. the command specified in CMAKE_CROSSCOMPILING_EMULATOR) and passes the full executable path to it, as long as the first COMMAND argument is the name of a target. This cleans up the CMake code somewhat as well, since it is no longer necessary to manually invoke CMAKE_CROSSCOMPILING_EMULATOR. Closes #747
DRC d59b1a3b 2024-01-30T15:40:51 Build: Reformat lines longer than 80 columns ... ... to ensure that no function argument starts beyond the 80th column.
DRC 36c51dd3 2024-01-26T15:55:19 GitHub: Update checkout, AWS credentials actions ... to silence deprecation warning regarding Node.js 12 and 16 actions.
DRC 9ef0d03e 2024-01-26T10:50:26 LICENSE.md: zlib License clarifications Disclaimer: I am not a lawyer, nor do I play one on TV. Referring to #744, mentioning the zlib License as a license that applies to libjpeg-turbo is confusing, and it isn't actually necessary, since the IJG License subsumes the terms of the zlib License in the context of the libjpeg API library and associated programs. This was presumably understood to be the case by Miyasaka-san when he chose the zlib License for the first libjpeg SIMD extensions. The libjpeg/SIMD web site (https://cetus.sakura.ne.jp/softlab/jpeg-x86simd/jpegsimd.html) states (translated from Japanese): "The terms of use of this SIMD enhanced version of IJG JPEG software are subject to the terms of use of the original version of IJG JPEG software." Detailed analysis of the zlib License terms: This software is provided 'as-is', without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this software. This text is an almost literal subset of the warranty disclaimer text in the IJG License. The IJG License states everything above with only slight differences in wording, and it further clarifies that the user assumes all risk as to the software's quality and accuracy and that vendors of commercial products based on the software must assume all warranty and liability claims. Permission is granted to anyone to use this software for any purpose, including commercial applications, and to alter it and redistribute it freely, subject to the following restrictions: This is semantically the same as the permission text in the IJG License, since "use, copy, modify, and distribute this software (or portions thereof) for any purpose, without fee" covers "use" for "any purpose, including commercial applications" as well as alteration and redistribution. 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required. The IJG License requirement that "If any part of the source code for this software is distributed, then this README file must be included, with this copyright and no-warranty notice unaltered; and any additions, deletions, or changes to the original files must be clearly indicated in accompanying documentation" (Clause 1), as well as the requirement that "If only executable code is distributed, then the accompanying documentation must state that 'this software is based in part on the work of the Independent JPEG Group'" (Clause 2), satisfies the requirement of Clause 1 of the zlib License. 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software. Since Clause 1 of the IJG License applies only to the distribution of source code, the copyright headers in the source code are effectively "accompanying documentation" in that case. This is why we ensure that the copyright headers of individual source files indicate the year(s) in which modifications were made by each contributor. Doing so satisfies the requirements of both Clause 2 of the zlib License and Clause 1 of the IJG License. 3. This notice may not be removed or altered from any source distribution. Clauses 2 and 3 of the zlib License apply only to the source code that bears that license. Thus, as applied to the software as a whole, those requirements of the inbound zlib License are compatible with the outbound IJG License as long as the IJG License does not contradict them (which it doesn't.) NOTE: To be clear, existing source code that bears the zlib License cannot literally be re-licensed under the IJG License, since that would violate Clause 3 of the zlib License. However, when considering the terms under which the overall library is made available, the IJG License effectively subsumes the terms of the zlib License. https://www.gnu.org/licenses/license-compatibility.en.html is a thorough, albeit somewhat GPL-biased, discussion of license compatibility.
DRC 7d67c349 2024-01-26T10:34:04 Build/Win: Report CMAKE_MSVC_RUNTIME_LIBRARY value ... when using CMake 3.15+
DRC 17df25f9 2024-01-25T13:52:58 Build/Win: Eliminate MSVC run-time DLL dependency (regression introduced by 1644bdb7d2fac66cd0ce25adef7754e008b5bc1e) Setting a maximum version in cmake_minimum_required() effectively sets the behavior to NEW for all policies introduced in all CMake versions up to and including that maximum version. The NEW behavior for CMP0091, introduced in CMake 3.15, uses CMake variables to specify the MSVC runtime library against which to link, rather than placing the relevant flags in CMAKE_C_FLAGS*. Thus, replacing /MD with /MT in CMAKE_C_FLAGS* no longer has any effect when using CMake 3.15+.
DRC 289df647 2024-01-23T17:35:53 Build: Add tjdoc target for building TurboJPEG dox
DRC 0ef07927 2024-01-23T10:46:04 Bump copyright year to 2024
DRC 1644bdb7 2024-01-22T14:33:31 BUILD: Silence CMake 3.28.x deprecation warning Closes #740
DRC 5a2353c2 2024-01-22T14:07:55 GNUInstallDirs.cmake: Various improvements - Integrate https://github.com/Kitware/CMake/commit/c07bba27302960fc2f35b6a9e00eab8b32ca9a49#diff-1e2deb5301e9481203102fcddd1b2d0d2bf0ddc1cbb445c7f4b6414a3b869ce8 so that the default man directory is <CMAKE_INSTALL_PREFIX>/share/man on FreeBSD systems. - For good measure, integrate https://github.com/Kitware/CMake/commit/f835f189aeb38a791ad09ba5c2d89300a3fd16f1 so that the default info directory is <CMAKE_INSTALL_PREFIX>/share/info on FreeBSD systems, even though we don't use that directory. - Automatically set the CMake variable type to PATH for any user-specified CMAKE_INSTALL_*DIR variables. Addresses concerns raised in #326, #346, #648 Closes #648
DRC 335ed793 2024-01-19T12:58:13 Assume 3-comp lossls JPEG w/o Adobe marker is RGB libjpeg-turbo always includes Adobe APP14 markers in the lossless JPEG images that it generates, but some compressors (e.g. accusoft PICTools Medical) do not. Fixes #743
DRC fa2b6ea0 2024-01-12T18:21:41 Eliminate duplicate copies of jpeg_nbits_table ef9a4e05ba919494cbebe50e15f332de5ab97e82 (libjpeg-turbo 1.4.x), which was based on https://bug815473.bmoattachments.org/attachment.cgi?id=692126 (https://bugzilla.mozilla.org/show_bug.cgi?id=815473), modified the C baseline Huffman encoder so that it precomputes jpeg_nbits_table, in order to facilitate sharing the table among multiple processes. However, libjpeg-turbo never shared the table, and because the table was implemented as a static array, f3a8684cd1c28e557d394470962a7a224c76ddbc (libjpeg-turbo 1.5.x) and 37bae1a0e977ee1ba769e6f0aa27e519ab6e58c6 (libjpeg-turbo 2.0.x) each introduced a duplicate copy of the table for (respectively) the SSE2 baseline Huffman encoder and the C progressive Huffman encoder. This commit does the following: - Move the duplicated code in jchuff.c and jcphuff.c, originally introduced in 0cfc4c17b740cb2cbb11f9d85c8ab3745d5b913a and 37bae1a0e977ee1ba769e6f0aa27e519ab6e58c6, into a header (jpeg_nbits.h). - Credit the co-author of 0cfc4c17b740cb2cbb11f9d85c8ab3745d5b913a. (Refer to https://sourceforge.net/p/libjpeg-turbo/patches/57). - Modify the SSE2 baseline Huffman encoder so that the C Huffman encoders can share its definition of jpeg_nbits_table. - Move the definition of jpeg_nbits_table into a C source file (jpeg_nbits.c) rather than a header, and define the table only if USE_CLZ_INTRINSIC is undefined and the SSE2 baseline Huffman encoder will not be built. - Apply hidden symbol visibility to the shared definition of jpeg_nbits_table, if the compiler supports the necessary attribute. (In practice, only Visual C++ doesn't.) Closes #114 See also: https://bugzilla.mozilla.org/show_bug.cgi?id=1501523
DRC be96fa0a 2023-12-14T13:18:20 Doc: Lossless JPEG clarifications - Clarify that lossless JPEG is slower than and doesn't compress as well as lossy JPEG. (That should be obvious, because "lossy" literally means that data is thrown away.) - Re-generate TurboJPEG C API documentation using Doxygen 1.9.8. - Clarify that setting the data_precision field in jpeg_compress_struct to 16 requires lossless mode. - Explain what the predictor selection value actually does. (Refer to Recommendation ITU-T T.81 (1992) | ISO/IEC 10918-1:1994, Section H.1.2.1.)
DRC abeca1f0 2023-11-30T09:35:11 Move official releases to GitHub
DRC 3eee0dd7 2023-11-29T10:03:49 ChangeLog.md: "since" = "relative to"
DRC 55d342c7 2023-11-16T15:36:47 TurboJPEG: Expose/extend hidden "max pixels" param TJPARAM_MAXPIXELS was previously hidden and used only for fuzz testing, but it is potentially useful for calling applications as well, particularly if they want to guard against excessive memory consumption by the tj3LoadImage*() functions. The parameter has also been extended to decompression and lossless transformation functions/methods, mainly as a convenience. (It was already possible for calling applications to impose their own JPEG image size limits by reading the JPEG header prior to decompressing or transforming the image.)
DRC 6136a9e2 2023-11-16T14:12:28 Java doc: Terminology tweaks - "function" = "method" - "decompression and transform functions" = "decompression and transform operations" (for consistency with the 2.1.x documentation) - "return an error" = "throw an error" - "ceil()" = "Math.ceil()"
DRC 40419472 2023-11-15T13:42:34 SECURITY.md: Further clarify security adv. policy Security advisories should only be filed against official releases.
DRC 45f018cb 2023-11-15T13:04:12 SECURITY.md: Clarify security advisories policy Unfortunately, most of the GitHub security advisories filed against libjpeg-turbo thus far have been the result of non-exploitable API abuses triggered by randomly-generated test programs and accompanied by wild claims of denials of service with no demonstrable or even probable exploit that might cause such a DoS (assuming a service even existed that used the API in question.) Security advisories remain private unless accepted, and I cannot accept them if they do not describe an actual security issue. Thus, it's best to steer most users toward regular bug reports.
DRC d5183047 2023-11-15T11:06:23 tjexampletest.in: Fix code formatting issue (introduced by 837e471a90e79b37cb4f7ef9950321f48c7c5f41)
DRC 8a4ce73c 2023-11-15T10:54:58 tj3Transform(): Range check transform operations
DRC a27bad65 2023-11-14T14:47:40 tj3Transform(): Ensure JPEG hdr successfully read Because of the TurboJPEG 3 API overhaul, the legacy decompression and lossless transformation functions now wrap the new TurboJPEG 3 functions. For performance reasons, we don't want to read the JPEG header more than once during the same operation, so the wrapped functions do not read the header if it has already been read by a wrapper function. Initially the TurboJPEG 3 functions used a state variable to track whether the header had already been read, but b94041390c477a02b3cab79d0cc0ef321dc889e8 made this more robust by using the libjpeg global decompression state instead. If a wrapper function has already read the JPEG header successfully, then the global decompression state will be DSTATE_READY, and the logic introduced in b94041390c477a02b3cab79d0cc0ef321dc889e8 will prevent the header from being read again. A subtle issue arises because tj3DecompressHeader() does not call jpeg_abort_decompress() if jpeg_read_header() fails. (That is arguably a bug, but it has existed since the very first implementation of the function.) Depending on the nature of the failure, this can cause tj3DecompressHeader() to return an error code and leave the libjpeg global decompression state set to DSTATE_INHEADER. If a misbehaved application ignored the error and subsequently called a TurboJPEG decompression or lossless transformation function, then the function would fail to read the JPEG header because the global decompression state was greater than DSTATE_START. In the case of the decompression functions, this was innocuous, because jpeg_calc_output_dimensions() and jpeg_start_decompress() both sanity check the global decompression state. However, it was possible for a misbehaved application to call tj3DecompressHeader() with junk data, ignore the return value, and pass the same junk data into tj3Transform(). Because tj3DecompressHeader() left the global decompression state set to DSTATE_INHEADER, tj3Transform() failed to detect the junk data (because it didn't try to read the JPEG header), and it called jtransform_request_workspace() with dinfo->image_width and dinfo->image_height still initialized to 0. Because jtransform_request_workspace() does not sanity check the decompression state, a division-by-zero error occurred with certain combinations of transform options in which TJXOPT_TRIM or TJXOPT_CROP was specified. However, it should be noted that TJXOPT_TRIM and TJXOPT_CROP cannot be expected to work properly without foreknowledge of the JPEG source image dimensions, which cannot be gained except by calling tj3DecompressHeader() successfully. Thus, a calling application is inviting trouble if it does not check the return value of tj3DecompressHeader() and sanity check the JPEG source image dimensions before calling tj3Transform(). This commit softens the failure, but the failure is still due to improper API usage.
DRC 837e471a 2023-11-14T11:17:25 tjexample.c: Fix error when decompressing (regression introduced by 300a344d653d4a8779706e42828d945c6a53ff9d) 300a344d653d4a8779706e42828d945c6a53ff9d fixed the recompression code path but also broke the pure decompression code path, because the fix caused the TurboJPEG decompression instance to be destroyed before tj3SaveImage() could use it. Furthermore, the fix in 300a344d653d4a8779706e42828d945c6a53ff9d prevented pixel density information from being transferred from the input image to the output image when recompressing. This commit does the following: - Modify tjexample.c so that a single TurboJPEG instance is initialized for lossless transformation and shared by all code paths. In addition to fixing both of the aforementioned issues, this makes the code more readable. - Extend tjexampletest to test the recompression code path, thus ensuring that the issues fixed by this commit and 300a344d653d4a8779706e42828d945c6a53ff9d are not reintroduced. - Modify tjexample.c to remove redundant fclose(), tj3Destroy(), and tj3Free() calls.
DRC df9dbff8 2023-11-11T12:25:03 TurboJPEG: New param to limit virt array mem usage This corresponds to max_memory_to_use in the jpeg_memory_mgr struct in the libjpeg API, except that the TurboJPEG parameter is specified in megabytes. Because this is 2023 and computers with less than 1 MB of memory are not a thing (at least not within the scope of libjpeg-turbo support), it isn't useful to allow a limit less than 1 MB to be specified. Furthermore, because TurboJPEG parameters are signed integers, if we allowed the memory limit to be specified in bytes, then it would be impossible to specify a limit larger than 2 GB on 64-bit machines. Because max_memory_to_use is a long signed integer, effectively we can specify a limit of up to 2 petabytes on 64-bit machines if the TurboJPEG parameter is specified in megabytes. (2 PB should be enough for anybody, right?) This commit also bumps the TurboJPEG API version to 3.0.1. Since the TurboJPEG API version no longer tracks the libjpeg-turbo version, it makes sense to increment the API revision number when adding constants, to increment the minor version number when adding functions, and to increment the major version number for a complete overhaul. This commit also removes the vestigial TJ_NUMPARAM macro, which was never defined because it proved unnecessary. Partially implements #735
DRC bf248a50 2023-11-11T15:14:07 tj3Compress*(): Free virt arrays if mem limit hit This is very subtle, but if a user specifies a libjpeg virtual array memory limit via the JPEGMEM environment variable and one of the tj3Compress*() functions hits that limit, the libjpeg error handler will be invoked in jpeg_start_compress() (more specifically in realize_virt_arrays() in jinit_compress_master()) before the libjpeg global compression state can be incremented. Thus, jpeg_abort_compress() will not be called before the tj3Compress*() function exits, the unrealized virtual arrays will not be freed, and if the TurboJPEG compression instance is reused, those unrealized virtual arrays will count against the specified memory limit. This could cause subsequent compression operations that require smaller virtual arrays (or even no virtual arrays at all) to fail when they would otherwise succeed. In reality, the vast majority of calling programs would abort and free the TurboJPEG compression instance if one of the tj3Compress*() functions failed, but TJBench is a rare exception. This issue does not bear documenting because of its subtlety and rarity and because JPEGMEM is not a documented feature of the TurboJPEG API. Note that the issue does not exist in the tj3Encode*() and tj3Decode*() functions, because realize_virt_arrays() is never called in the body of those functions. The issue also does not exist in the tj3Decompress*() and tj3Transform() functions, because those functions ensure that the JPEG header is read (and thus the libjpeg global decompression state is incremented) prior to calling a function that calls realize_virt_arrays() (i.e. jpeg_start_decompress() or jpeg_read_coefficients().) If realize_virt_arrays() failed in the body of jpeg_write_coefficients(), then tj3Transform() would abort without calling jpeg_abort_compress(). However, since jpeg_start_compress() is never called in the body of tj3Transform(), no virtual arrays are ever requested from the compression object, so failing to call jpeg_abort_compress() would be innocuous.
DRC 220bd761 2023-11-10T11:06:16 turbojpeg.h: Fix broken refs in API documentation "TJPARAM_DENSITYUNIT" should be "TJPARAM_DENSITYUNITS" (oops.)
DRC cbdc20fb 2023-11-08T11:43:58 example.c: Remove comments regarding temp files Those comments could be confusing to new libjpeg-turbo users, and the same information already exists in libjpeg.txt and structure.txt.
DRC 7c61794f 2023-11-08T11:16:14 jmemsys.h: Remove unused MS-DOS/Mac mem. mgr. code Those obsolete memory managers have never been included in libjpeg-turbo, nor has libjpeg-turbo ever claimed to support MS-DOS or Mac operating systems prior to OS X 10.4 "Tiger." Note that we retain the ability to use temp files, even though libjpeg-turbo does not use them. This allows downstream implementations to write their own memory managers that use temp files, if necessary.
DRC 78eaf0d4 2023-11-07T13:44:40 tj3*YUV8(): Fix int overflow w/ huge row alignment If the align parameter was set to an unreasonably large value, such as 0x2000000, strides[0] * ph0 and strides[1] * ph1 could have overflowed the int datatype and wrapped around when computing (src|dst)Planes[1] and (src|dst)Planes[2] (respectively.) This would have caused (src|dst)Planes[1] and (src|dst)Planes[2] to point to lower addresses in the YUV buffer than expected, so the worst case would have been a visually incorrect output image, not a buffer overrun or other exploitable issue.
DRC 0e2d289f 2023-11-07T12:38:05 Bump version to 3.0.2 to prepare for new commits
DRC ec32420f 2023-10-11T15:02:33 example.c: Fix 12-bit PPM write w/ big endian CPUs
DRC 5b2beb4b 2023-10-10T16:44:59 Build: Set FLOATTEST* by default for AArch64, PPC Because of 47656a082091f9c9efda054674522513f4768c6c, we can now reliably determine the correct default values for FLOATTEST8 and FLOATTEST12 when using Clang or GCC to build for AArch64 or PowerPC platforms. (Testing confirms that this is the case with GCC 5-13 and Clang 5-14 on Ubuntu/AArch64, GCC 4 on CentOS 7/PPC, and GCC 8-10 and Clang 6-12 on Ubuntu/PPCLE.) Other CPU architectures and compilers can be added on a case-by-case basis as they are tested.
DRC da48edfc 2023-10-09T14:13:55 jchuff.c: Fix uninit read w/ AArch64, WITH_SIMD=0 Because of bf01ed2fbc02c15e86f414ff4946b66b4e5a00f1, the simd field in huff_entropy_encoder (and, by extension, the simd field in savable_state) is only initialized if WITH_SIMD is defined. Due to an oversight, the simd field in savable_state was queried in flush_bits() regardless of whether WITH_SIMD was defined. In most cases, both branches of the query have identical code, and the optimizer removes the branch. However, because the legacy Neon GAS Huffman encoder uses the older bit buffer logic from libjpeg-turbo 2.0.x and prior (refer to 087c29e07f7533ec82fd7eb1dafc84c29e7870ec), the branches do not have identical code when building for AArch64 with NEON_INTRINSICS undefined (which will be the case if WITH_SIMD is undefined.) Thus, if libjpeg-turbo was built for AArch64 with the SIMD extensions disabled at build time, it was possible for the Neon GAS branch in flush_bits() to be taken, which would have set put_bits to a value that is incorrect for the C Huffman encoder. Referring to #728, a user reported that this issue sometimes caused libjpeg-turbo to generate bogus JPEG images if it was built for AArch64 without SIMD extensions and subsequently used through the Qt framework. (It should be noted, however, that disabling the SIMD extensions in AArch64 builds of libjpeg-turbo is inadvisable for performance reasons.) I was unable to reproduce the issue on Linux/AArch64 using libjpeg-turbo alone, despite testing various versions of GCC and Clang and various optimization levels. However, the issue is reproducible using MSan with -O0, so this commit also modifies the GitHub Actions workflow so that compiler optimization is disabled in the linux-msan job. That should prevent the issue or similar issues from re-emerging. Fixes #728
DRC 3d1d68cf 2023-10-04T13:20:38 README.md: Mention 4:4:0 math. incomp. vs. jpeg-6b libjpeg-turbo implements 4:4:0 "fancy" (smooth) upsampling, which is enabled by default when decompressing JPEG images that use 4:4:0 chrominance subsampling. libjpeg did not and does not implement fancy 4:4:0 upsampling.
DRC 2c97a1ff 2023-10-03T12:07:40 GitHub: Use Ubuntu 20.04 runner for x32 build/test The Ubuntu 22.04 kernel no longer supports the x32 ABI.
DRC 47656a08 2023-10-02T18:03:50 Test: Fix float test errors w/ Clang & fp-contract The MD5 sums associated with FLOATTEST8=fp-contract and FLOATTEST12=fp-contract are appropriate for GCC (tested v5 through v13) with -ffp-contract=fast, which is the default when compiling for an architecture that has fused multiply-add (FMA) instructions. However, different MD5 sums are needed for Clang (tested v5 through v14) with -ffp-contract=on, which is now the default in Clang 14 when compiling for an architecture that has FMA instructions. Refer to #705, #709, #710
DRC c0412b56 2023-09-14T17:19:36 ChangeLog.md: List CVE ID fixed by ccaba5d7 Closes #724
DRC f3c7116e 2023-09-11T12:46:13 jpeglib.h: Document that JCS_RGB565 is decomp-only Closes #723
DRC 9b704f96 2023-08-15T11:03:57 Fix block smoothing w/vert.-subsampled prog. JPEGs The 5x5 interblock smoothing implementation, introduced in libjpeg-turbo 2.1, improperly extended the logic from the traditional 3x3 smoothing implementation. Both implementations point prev_block_row and next_block_row to the current block row when processing, respectively, the first and the last block row in the image: if (block_row > 0 || cinfo->output_iMCU_row > 0) prev_block_row = buffer[block_row - 1] + cinfo->master->first_MCU_col[ci]; else prev_block_row = buffer_ptr; if (block_row < block_rows - 1 || cinfo->output_iMCU_row < last_iMCU_row) next_block_row = buffer[block_row + 1] + cinfo->master->first_MCU_col[ci]; else next_block_row = buffer_ptr; 6d91e950c871103a11bac2f10c63bf998796c719 naively extended that logic to accommodate a 5x5 smoothing window: if (block_row > 1 || cinfo->output_iMCU_row > 1) prev_prev_block_row = buffer[block_row - 2] + cinfo->master->first_MCU_col[ci]; else prev_prev_block_row = prev_block_row; if (block_row < block_rows - 2 || cinfo->output_iMCU_row + 1 < last_iMCU_row) next_next_block_row = buffer[block_row + 2] + cinfo->master->first_MCU_col[ci]; else next_next_block_row = next_block_row; However, this new logic was only correct if block_rows == 1, so the values of prev_prev_block_row and next_next_block_row were incorrect when processing, respectively, the second and second to last iMCU rows in a vertically-subsampled progressive JPEG image. The intent was to: - point prev_block_row to the current block row when processing the first block row in the image, - point prev_prev_block_row to prev_block_row when processing the first two block rows in the image, - point next_block_row to the current block row when processing the last block row in the image, and - point next_next_block_row to next_block_row when processing the last two block rows in the image. This commit modifies decompress_smooth_data() so that it computes the current block row's position relative to the whole image and sets the block row pointers based on that value. This commit also restores a line of code that was accidentally deleted by 6d91e950c871103a11bac2f10c63bf998796c719: access_rows += compptr->v_samp_factor; /* prior iMCU row too */ access_rows is merely a sanity check that tells the access_virt_barray() method to generate an error if accessing the specified number of rows would cause a buffer overrun. Essentially, it is a belt-and-suspenders measure to ensure that j*init_d_coef_controller() allocated enough rows for the full-image virtual array. Thus, excluding that line of code did not cause an observable issue. This commit also documents dbae59281fdc6b3a6304a40134e8576d50d662c0 in the change log. Fixes #721
DRC 7b844bfd 2023-07-28T11:46:10 x86-64 SIMD: Use std stack frame/prologue/epilogue This allows debuggers and profilers to reliably capture backtraces from within the x86-64 SIMD functions. In places where rbp was previously used to access temporary variables (after stack alignment), we now use r15 and save/restore it accordingly. The total amount of work is approximately the same, because the previous code pushed the pre-alignment stack pointer to the aligned stack. The new prologue and epilogue actually have fewer instructions. Also note that the {un}collect_args macros now use rbp instead of rax to access arguments passed on the stack, so we save a few instructions there as well. Based on: https://github.com/alk/libjpeg-turbo/commit/debcc7c3b436467aea8d02c66a514c5099d0ad37 Closes #707 Closes #708
DRC e17fa3a2 2023-07-27T13:11:39 Bump version to 3.0.1 to prepare for new commits
DRC dbae5928 2023-08-03T14:42:30 Fix interblock smoothing with narrow prog. JPEGs Due to an oversight, the assignment of DC05, DC10, DC15, DC20, and DC25 (the right edge coefficients in the 5x5 interblock smoothing window) in decompress_smooth_data() was incorrect for images exactly two MCU blocks wide. For such images, DC04, DC09, DC14, DC19, and DC24 were assigned values based on the last MCU column, but DC05, DC10, DC15, DC20, and DC25 were assigned values based on the first MCU column (because block_num + 1 was never less than last_block_column.) This commit modifies jdcoefct.c so that, for images at least two MCU blocks wide, DC05, DC10, DC15, DC20, and DC25 are assigned the same values as DC04, DC09, DC14, DC19, and DC24 (respectively.) DC05, DC10, DC15, DC20, and DC25 are then immediately overwritten for images more than two MCU blocks wide. Since this issue was minor and not likely obvious to an end user, the fix is undocumented. Fixes #700
DRC 300a344d 2023-07-30T11:02:29 tjexample.c: Fix error when recompressing (regression introduced by fc01f4673b71c0b833c59c21e8c4478a9c4bcf21) Because of the TurboJPEG 3 API overhaul, the pure compression code path in tjexample.c now creates a TurboJPEG compression instance prior to calling tj3LoadImage(). However, due to an oversight, a compression instance was no longer created in the recompression code path. Thus, that code path attempted to reuse the TurboJPEG decompression instance, which caused an error ("tj3Set(): TJPARAM_QUALITY is not applicable to decompression instances.") This commit modifies tjexample.c so that the recompression code path creates a new TurboJPEG compression instance if one has not already been created. Fixes #716
DRC ebca79d5 2023-07-25T16:46:07 xform fuzz: Test optimized baseline entropy coding Because of d011622f4b5b2c3f0141e93fc3e1da6169915c18, optimized baseline entropy coding wasn't actually being tested.
DRC 63bd7188 2023-07-25T10:01:42 Build: Unset FLOATTEST* by default for non-x86 Because libjpeg-turbo 3.0.x now supports multiple data precisions in the same build, the regression test system can test the 8-bit and 12-bit floating point DCT/IDCT algorithms separately. The expected MD5 sums for those tests are communicated to the test system using the FLOATTEST8 and FLOATTEST12 CMake variables. Whereas it is possible to intelligently set a default value for FLOATTEST8 when building for x86[-64] and a default value for FLOATTEST12 when building for x86-64, it is not possible with other architectures. (Refer to #705, #709, and #710.) Clang 14, for example, now enables FMA (fused multiply-add) instructions by default on architectures that support them, but with AArch64 builds, the results are not the same as when using GCC/AArch64 with FMA instructions enabled. Thus, setting FLOATTEST12=fp-contract doesn't make the tests pass. It was already impossible to intelligently set a default for FLOATTEST8 with i386 builds, but referring to #710, that appears to be the case with other non-x86-64 builds as well. Back in 1991, when Tom Lane first released libjpeg, some CPUs had floating point units and some didn't. It could take minutes to compress or decompress a 1-megapixel JPEG image using the "slow" integer DCT/IDCT algorithms, and the floating point algorithms were significantly faster on systems that had an FPU. On systems without FPUs, floating point math was emulated and really slow, so Tom also developed "fast" integer DCT/IDCT algorithms to speed up JPEG performance, at the expense of accuracy, on those systems. Because of libjpeg-turbo's SIMD extensions, the floating point algorithms are now significantly slower than the "slow" integer algorithms without being significantly more accurate, and the "fast" integer algorithms fail the ISO/ITU-T conformance tests without being any faster than the "slow" integer algorithms on x86 systems. Thus, the floating point and "fast" integer algorithms are considered legacy features. In order for the floating point regression tests to be useful, the results of the tests must be validated against an independent metric. (In other words, it wouldn't be useful to use the floating point DCT/IDCT algorithms to determine the expected results of the floating point DCT/IDCT algorithms.) In the past, I attempted without success to develop a low-level floating point test that would run at configure time and determine the appropriate default value of FLOATTEST*. Barring that approach, the only other possibilities would be: 1. Develop a test framework that compares the floating point results with a margin of error, as TJUnitTest does. However, that effort isn't justified unless it could also benefit non-legacy features. 2. Compare the floating point results against an expected MD5 sum, as we currently do. However, as previously described, it isn't possible in most cases to determine an appropriate default value for the expected MD5 sum. For the moment, it makes the most sense to disable the 8-bit floating point tests by default except with x86[-64] builds and to disable the 12-bit floating point tests by default except with x86-64 builds. That means that the floating point algorithms will still be regression tested when performing x86[-64] builds, but other types of builds will have to opt in to the same regression tests. Since the floating point DCT/IDCT algorithms are unlikely to change ever again (the only reason they still exist at all is to maintain backward compatibility with libjpeg), this seems like a reasonable tradeoff.
DRC d6914b6b 2023-07-24T16:41:18 CMakeLists.txt: Fix comment buglet
DRC e429e379 2023-07-06T16:58:20 tjunittest.c: Use _getpid() on Windows
DRC 035ea386 2023-07-06T12:04:22 Build: Fix regression test concurrency issues - The example-*bit-*-decompress test must run after the example-*bit-*-compress test, since the latter generates testout*-example.jpg. - Add -static to the filenames of all output files generated by the "static" regression tests, to avoid conflicts with the "shared" regression tests. - Add the PID to the filenames of all files generated by the tjunittest packed-pixel image I/O tests. - Check the return value of MD5File() in tjunittest to avoid a segfault if the file doesn't exist. (Prior to the fix described above, that could occur if two instances of tjunittest ran concurrently from the same directory with the same -bmp and -precision arguments.) Fixes #705
DRC d011622f 2023-07-06T10:29:27 Restore xform fuzzer behavior from before c8d52f1c The intent was for the final transform operation to be the same as the first transform operation but without TJXOPT_COPYNONE or TJPARAM_NOREALLOC. Unrolling the transform operations in c8d52f1c4c7480277b91420c27b2548d4c8e9043 accidentally changed that.
DRC 89528757 2023-07-05T15:35:21 xform fuzz: Use src subsamp to calc dst buf size Referring to https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60379 there are some specially-crafted malformed JPEG images that, when transformed to grayscale, will exceed the worst-case transformed grayscale JPEG image size. This is similar in nature to the issue fixed by c8d52f1c4c7480277b91420c27b2548d4c8e9043, except that in this case, the issue occurs regardless of the amount of metadata in the source image. Also, the tj3Transform() function, the Java_org_libjpegturbo_turbojpeg_TJTransformer_transform() JNI function, and TJBench were behaving correctly in this case, because the TurboJPEG API documentation specifies that the source image's subsampling type should be used when computing the worst-case transformed JPEG image size. (However, only the Java API documentation specified that. Oops. The C API documentation now does as well.) The documented usage mitigates the issue, and only the transform fuzzer did not adhere to that. Thus, this was an issue with the fuzzer itself rather than an issue with the library.
DRC 30c21e55 2023-07-05T13:51:35 OSS-Fuzz: Ignore tj3DecompressHeader() return val Unlike its predecessors, tj3DecompressHeader() does not fail if passed a JPEG image with an unknown subsampling type. This led me to believe that it was OK for the fuzzers to abort if tj3DecompressHeader() returned an error. However, there are apparently some malformed JPEG images that can expose issues in libjpeg-turbo while also causing tj3DecompressHeader() to complain about header errors. Thus, it is best to ignore the return value of tj3DecompressHeader(), as the fuzzers in libjpeg-turbo 2.1.x and prior did.
DRC 926f1f3d 2023-07-05T12:41:32 README.md: Include GitHub Sponsors link/button Closes #706
DRC 762f8b4f 2023-07-05T10:55:07 Doc: Mention that we are a JPEG ref implementation
DRC 6c87537f 2023-07-03T10:25:24 AppVeyor: Use SignPath release cert/only sign tags The SignPath release certificate for our project is not yet available as of this writing, but this commit prepares the CI system to use the release certificate whenever it becomes available. It also restricts signing only to tags, which correspond to official releases. (That mimics our traditional policy of not signing pre-release builds.)
DRC 02b074fd 2023-07-01T08:09:23 xform fuzz: Use only xform opts to set entropy alg This is subtle, but the tj3DecompressHeader() function sets the values of TJPARAM_ARITHMETIC, TJPARAM_OPTIMIZE, and TJPARAM_PROGRESSIVE. Unless we unset those values, the entropy algorithm used in the transformed JPEG image will be determined by the union of the parameter values and the transform options, which isn't what we want.
DRC e0c53aa3 2023-06-30T17:58:45 jchuff.c: Test for out-of-range coefficients Restore two coefficient range checks from libjpeg to the C baseline Huffman encoder. This fixes an issue (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60253) whereby the encoder could read from uninitialized memory when attempting to transform a specially-crafted malformed arithmetic-coded JPEG source image into a baseline Huffman-coded JPEG destination image with default Huffman tables. More specifically, the out-of-range coefficients caused r to equal 256, which overflowed the actbl->ehufsi[] array. Because the overflow was contained within the huff_entropy_encoder structure, this issue was not exploitable (nor was it observable at all on x86 or Arm CPUs unless JSIMD_NOHUFFENC=1 or JSIMD_FORCENONE=1 was set in the environment or unless libjpeg-turbo was built with WITH_SIMD=0.) The fix is performance-neutral (+/- 1-2%) for x86-64 code and causes a 0-4% (avg. 1-2%, +/- 1-2%) compression regression for i386 code on Intel CPUs when the C baseline Huffman encoder is used (JSIMD_NOHUFFENC=1). The fix is performance-neutral (+/- 1-2%) on Intel CPUs when all of the libjpeg-turbo SIMD extensions are disabled (JSIMD_FORCENONE=1). The fix causes a 0-2% (avg. <1%, +/- 1%) compression regression for PowerPC code.
DRC 4a831d6e 2023-06-30T10:15:40 turbojpeg.h: Make customFilter() proto match doc Closes #704
DRC 240a5a5c 2023-06-29T17:45:07 ChangeLog.md: Fix typo
DRC 71738892 2023-06-29T16:31:37 djpeg: Fix -map option with 12-bit data precision
DRC bf9f319c 2023-06-29T16:07:42 Disallow color quantization with lossless decomp Color quantization is a legacy feature that serves little or no purpose with lossless JPEG images. 9f756bc67a84d4566bf74a0c2432aa55da404021 eliminated interaction issues between the lossless decompressor and the color quantizers related to out-of-range 12-bit samples, but referring to #701, other interaction issues apparently still exist. Such issues are likely, given the fact that the color quantizers were not designed with lossless decompression in mind. This commit reverts 9f756bc67a84d4566bf74a0c2432aa55da404021, since the issues it fixed are no longer relevant because of this commit and 2192560d74e6e6cf99dd05928885573be00a8208. Fixed #672 Fixes #673 Fixes #674 Fixes #676 Fixes #677 Fixes #678 Fixes #679 Fixes #681 Fixes #683 Fixes #701
DRC c8d52f1c 2023-06-26T11:53:03 tj3Transform: Calc dst buf size from xformed dims When used with TJPARAM_NOREALLOC and with TJXOP_TRANSPOSE, TJXOP_TRANSVERSE, TJXOP_ROT90, or TJXOP_ROT270, tj3Transform() incorrectly based the destination buffer size for a transform on the source image dimensions rather than the transformed image dimensions. This was apparently a long-standing bug that had existed in the tj*Transform() function since its inception. As initially implemented in the evolving libjpeg-turbo v1.2 code base, tjTransform() required dstSizes[i] to be set regardless of whether TJFLAG_NOREALLOC (the predecessor to TJPARAM_NOREALLOC) was set. ff78e37595c8462f64fd100f928aa1d08539527e, which was introduced later in the evolving libjpeg-turbo v1.2 code base, removed that requirement and planted the seed for the bug. However, the bug was not activated until 9b49f0e4c77c727648c6d3a4915eefdf5436de4a was introduced still later in the evolving libjpeg-turbo v1.2 code base, adding a subsampling type argument to the (new at the time) tjBufSize() function and thus making the width and height arguments no longer commutative. The bug opened up the possibility that a JPEG source image could cause tj3Transform() to overflow the destination buffer for a transform if all of the following were true: - The JPEG source image used 4:2:2, 4:4:0, 4:1:1, or 4:4:1 subsampling. (These are the only subsampling types for which the width and height arguments to tj3JPEGBufSize() are not commutative.) - The width and height of the JPEG source image were such that tj3JPEGBufSize(height, width, subsamplingType) returned a smaller value than tj3JPEGBufSize(width, height, subsamplingType). - The JPEG source image contained enough metadata that the size of the transformed image was larger than tj3JPEGBufSize(height, width, subsamplingType). - TJPARAM_NOREALLOC was set. - TJXOP_TRANSPOSE, TJXOP_TRANSVERSE, TJXOP_ROT90, or TJXOP_ROT270 was used. - TJXOPT_COPYNONE was not set. - TJXOPT_CROP was not set. - The calling program allocated tj3JPEGBufSize(height, width, subsamplingType) bytes for the destination buffer, as the API documentation instructs. The API documentation cautions that JPEG source images containing a large amount of extraneous metadata (EXIF, IPTC, ICC, etc.) cannot reliably be transformed if TJPARAM_NOREALLOC is set and TJXOPT_COPYNONE is not set. Irrespective of the bug, there are still cases in which a JPEG source image with a large amount of metadata can, when transformed, exceed the worst-case transformed JPEG image size. For instance, if you try to losslessly crop a JPEG image with 3 kB of EXIF data to 16x16 pixels, then you are guaranteed to exceed the worst-case 16x16 JPEG image size unless you discard the EXIF data. Even without the bug, tj3Transform() will still fail with "Buffer passed to JPEG library is too small" when attempting to transform JPEG source images that meet the aforementioned criteria. The bug is that the function segfaults rather than failing gracefully, but the chances of that occurring in a real-world application are very slim. Any real-world application developers who attempted to transform arbitrary JPEG source images with TJPARAM_NOREALLOC set would very quickly realize that they cannot reliably do that without also setting TJXOPT_COPYNONE. Thus, I posit that the actual risk posed by this bug is low. Applications such as web browsers that are the most exposed to security risks from arbitrary JPEG source images do not use the TurboJPEG lossless transform feature. (None of those applications even use the TurboJPEG API, to the best of my knowledge, and the public libjpeg API has no equivalent transform function.) Our only command-line interface to the tj3Transform() function, TJBench, was not exposed to the bug because it had a compatible bug whereby it allocated the JPEG destination buffer to the same size that tj3Transform() erroneously expected. The TurboJPEG Java API was also not exposed to the bug because of a similar compatible bug in the Java_org_libjpegturbo_turbojpeg_TJTransformer_transform() JNI function. (This commit fixes both compatible bugs.) In short, best practices for tj3Transform() are to use TJPARAM_NOREALLOC only with JPEG source images that are known to be free of metadata (such as images generated by tj3Compress*()) or to use TJXOPT_COPYNONE along with TJPARAM_NOREALLOC. Still, however, the function shouldn't segfault as long as the calling program allocates the suggested amount of space for the JPEG destination buffer. Usability notes: tj3Transform() could hypothetically require dstSizes[i] to be set regardless of the value of TJPARAM_NOREALLOC, but there are usability pitfalls either way. The main pitfall I sought to avoid with ff78e37595c8462f64fd100f928aa1d08539527e was a calling program failing to set dstSizes[i] at all, thus leaving its value undefined. It could be argued that requiring dstSizes[i] to be set in all cases is more consistent, but it could also be argued that not requiring it to be set when TJPARAM_NOREALLOC is set is more user-proof. tj3Transform() could also hypothetically set TJXOPT_COPYNONE automatically when TJPARAM_NOREALLOC is set, but that could lead to user confusion. Ultimately, I would like to address these issues in TurboJPEG v4 by using managed buffer objects, but that would be an extensive overhaul.
DRC 6b9e3b04 2023-06-19T11:11:16 README.md: Include link to project home page (for compliance with SignPath's Code of Conduct for Open Source projects)
DRC 41b18e8d 2023-06-18T12:50:47 AppVeyor: Only add installers to zip file Oops
DRC a403ad5b 2023-06-16T15:20:29 AppVeyor: Integrate with SignPath.io
DRC c27695a1 2023-06-16T11:20:15 Fix build warnings/errs w/ -DNO_GETENV/-DNO_PUTENV - strtest.c: Fix unused variable warnings if both -DNO_GETENV and -DNO_PUTENV are specified or if only -DNO_GETENV is specified. - jinclude.h: Fix build error if only -DNO_GETENV is specified. Fixes #697
DRC 65a85ce3 2023-06-16T11:16:08 GitHub: Fix x32 build 1f55ae7b0fa3acc348a630171617d0e56d922b68 accidentally overrode the value of CMAKE_C_FLAGS, thus eliminating the -mx32 flag that was necessary to enable x32.
DRC 0e9683c4 2023-06-12T14:36:18 Bump version to 3.0.0
DRC 6b506ed3 2023-06-01T13:11:14 tjexample.c: Prevent integer overflow Because width, height, and tjPixelSize[] are signed integers, signed integer overflow will occur if width * height * tjPixelSize[pixelFormat] > INT_MAX, which would cause an incorrect value to be passed to tj3Alloc(). This commit modifies tjexample.c in the following ways: - Implicitly promote width, height, and tjPixelSize[pixelFormat] to size_t before multiplying them. - Use malloc() rather than tj3Alloc() to allocate the uncompressed image buffer. (tj3Alloc() is only necessary for JPEG buffers that will potentially be reallocated by the TurboJPEG API library.) - If size_t is 32-bit, throw an error if width * height * tjPixelSize[pixelFormat] would overflow the data type. Since tjexample is not installed or packaged, the worst case for this issue was that a downstream application might interpret tjexample.c literally and introduce a similar overflow issue into its own code. However, it's worth noting that such issues could also be introduced when using malloc().
DRC 2192560d 2023-05-31T13:02:42 Disallow merged upsampling with lossless decomp Colorspace conversion is explicitly not supported with lossless JPEG images. Merged upsampling implies YCbCr-to-RGB colorspace conversion, so allowing it with lossless decompression was an oversight. 9f756bc67a84d4566bf74a0c2432aa55da404021 eliminated interaction issues between the lossless decompressor and the merged upsampler related to out-of-range 12-bit samples, but referring to #690, other interaction issues apparently still exist. Such issues are likely, given the fact that the merged upsampler was never designed with lossless decompression in mind. This commit also extends the decompress fuzzer so that it catches the issue reported in #690. Fixes #690 Redundantly fixes #670 Redundantly fixes #675
kmx.io     mail     discord kmxgit v0.4.0