kc3-lang/libjpeg-turbo/fuzz/compress12_lossless.cc

• Show log

  Commit

• Hash : fc01f467
  Author :
  Date : 2023-01-05T06:36:46
  

  TurboJPEG 3 API overhaul
  
  (ChangeLog update forthcoming)
  
  - Prefix all function names with "tj3" and remove version suffixes from
    function names.  (Future API overhauls will increment the prefix to
    "tj4", etc., thus retaining backward API/ABI compatibility without
    versioning each individual function.)
  
  - Replace stateless boolean flags (including TJ*FLAG_ARITHMETIC and
    TJ*FLAG_LOSSLESS, which were never released) with stateful integer
    parameters, the value of which persists between function calls.
    * Use parameters for the JPEG quality and subsampling as well, in
      order to eliminate the awkwardness of specifying function arguments
      that weren't relevant for lossless compression.
    * tj3DecompressHeader() now stores all relevant information about the
      JPEG image, including the width, height, subsampling type, entropy
      coding type, etc. in parameters rather than returning that
      information in its arguments.
    * TJ*FLAG_LIMITSCANS has been reimplemented as an integer parameter
      (TJ*PARAM_SCANLIMIT) that allows the number of scans to be
      specified.
  
  - Use the const keyword for all pointer arguments to unmodified
    buffers, as well as for both dimensions of 2D pointers.  Addresses
    #395.
  
  - Use size_t rather than unsigned long to represent buffer sizes, since
    unsigned long is a 32-bit type on Windows.  Addresses #24.
  
  - Return 0 from all buffer size functions if an error occurs, rather
    than awkwardly trying to return -1 in an unsigned data type.
  
  - Implement 12-bit and 16-bit data precision using dedicated
    compression, decompression, and image I/O functions/methods.
    * Suffix the names of all data-precision-specific functions with 8,
      12, or 16.
    * Because the YUV functions are intended to be used for video, they
      are currently only implemented with 8-bit data precision, but they
      can be expanded to 12-bit data precision in the future, if
      necessary.
    * Extend TJUnitTest and TJBench to test 12-bit and 16-bit data
      precision, using a new -precision option.
    * Add appropriate regression tests for all of the above to the 'test'
      target.
    * Extend tjbenchtest to test 12-bit and 16-bit data precision, and
      add separate 'tjtest12' and 'tjtest16' targets.
    * BufferedImage I/O in the Java API is currently limited to 8-bit
      data precision, since the BufferedImage class does not
      straightforwardly support higher data precisions.
    * Extend the PPM reader to convert 12-bit and 16-bit PBMPLUS files
      to grayscale or CMYK pixels, as it already does for 8-bit files.
  
  - Properly accommodate lossless JPEG using dedicated parameters
    (TJ*PARAM_LOSSLESS, TJ*PARAM_LOSSLESSPSV, and TJ*PARAM_LOSSLESSPT),
    rather than using a flag and awkwardly repurposing the JPEG quality.
    Update TJBench to properly reflect whether a JPEG image is lossless.
  
  - Re-organize the TJBench usage screen.
  
  - Update the Java docs using Java 11, to improve the formatting and
    eliminate HTML frames.
  
  - Use the accurate integer DCT algorithm by default for both
    compression and decompression, since the "fast" algorithm is a legacy
    feature, it does not pass the ISO compliance tests, and it is not
    actually faster on modern x86 CPUs.
    * Remove the -accuratedct option from TJBench and TJExample.
  
  - Re-implement the 'tjtest' target using a CMake script that enables
    the appropriate tests, depending on the data precision and whether or
    not the Java API is part of the build.
  
  - Consolidate the C and Java versions of tjbenchtest into one script.
  
  - Consolidate the C and Java versions of tjexampletest into one script.
  
  - Combine all initialization functions into a single function
    (tj3Init()) that accepts an integer parameter specifying the
    subsystems to initialize.
  
  - Enable decompression scaling explicitly, using a new function/method
    (tj3SetScalingFactor()/TJDecompressor.setScalingFactor()), rather
    than implicitly using awkward "desired width"/"desired height"
    parameters.
  
  - Introduce a new macro/constant (TJUNSCALED/TJ.UNSCALED) that maps to
    a scaling factor of 1/1.
  
  - Implement partial image decompression, using a new function/method
    (tj3SetCroppingRegion()/TJDecompressor.setCroppingRegion()) and
    TJBench option (-crop).  Extend tjbenchtest to test the new feature.
    Addresses #1.
  
  - Allow the JPEG colorspace to be specified explicitly when
    compressing, using a new parameter (TJ*PARAM_COLORSPACE).  This
    allows JPEG images with the RGB and CMYK colorspaces to be created.
  
  - Remove the error/difference image feature from TJBench.  Identical
    images to the ones that TJBench created can be generated using
    ImageMagick with
    'magick composite <original_image> <output_image> -compose difference <diff_image>'
  
  - Handle JPEG images with unknown subsampling types.  TJ*PARAM_SUBSAMP
    is set to TJ*SAMP_UNKNOWN (== -1) for such images, but they can still
    be decompressed fully into packed-pixel images or losslessly
    transformed (with the exception of lossless cropping.)  They cannot
    be partially decompressed or decompressed into planar YUV images.
    Note also that TJBench, due to its lack of support for imperfect
    transforms, requires that the subsampling type be known when
    rotating, flipping, or transversely transposing an image.  Addresses
    #436
  
  - The Java version of TJBench now has identical functionality to the C
    version.  This was accomplished by (somewhat hackishly) calling the
    TurboJPEG C image I/O functions through JNI and copying the pixels
    between the C heap and the Java heap.
  
  - Add parameters (TJ*PARAM_RESTARTROWS and TJ*PARAM_RESTARTBLOCKS) and
    a TJBench option (-restart) to allow the restart marker interval to
    be specified when compressing.  Eliminate the undocumented TJ_RESTART
    environment variable.
  
  - Add a parameter (TJ*PARAM_OPTIMIZE), a transform option
    (TJ*OPT_OPTIMIZE), and a TJBench option (-optimize) to allow
    optimized baseline Huffman coding to be specified when compressing.
    Eliminate the undocumented TJ_OPTIMIZE environment variable.
  
  - Add parameters (TJ*PARAM_XDENSITY, TJ*PARAM_DENSITY, and
    TJ*DENSITYUNITS) to allow the pixel density to be specified when
    compressing or saving a Windows BMP image and to be queried when
    decompressing or loading a Windows BMP image.  Addresses #77.
  
  - Refactor the fuzz targets to use the new API.
    * Extend decompression coverage to 12-bit and 16-bit data precision.
    * Replace the awkward cjpeg12 and cjpeg16 targets with proper
      TurboJPEG-based compress12, compress12-lossless, and
      compress16-lossless targets
  
  - Fix innocuous UBSan warnings uncovered by the new fuzzers.
  
  - Implement previous versions of the TurboJPEG API by wrapping the new
    functions (tested by running the 2.1.x versions of TJBench, via
    tjbenchtest, and TJUnitTest against the new implementation.)
    * Remove all JNI functions for deprecated Java methods and implement
      the deprecated methods using pure Java wrappers.  It should be
      understood that backward API compatibility in Java applies only to
      the Java classes and that one cannot mix and match a JAR file from
      one version of libjpeg-turbo with a JNI library from another
      version.
  
  - tj3Destroy() now silently accepts a NULL handle.
  
  - tj3Alloc() and tj3Free() now return/accept void pointers, as malloc()
    and free() do.
  
  - The image I/O functions now accept a TurboJPEG instance handle, which
    is used to transmit/receive parameters and to receive error
    information.
  
  Closes #517
  

Download

• fuzz/compress12_lossless.cc

• /*
   * Copyright (C)2021-2023 D. R. Commander.  All Rights Reserved.
   *
   * Redistribution and use in source and binary forms, with or without
   * modification, are permitted provided that the following conditions are met:
   *
   * - Redistributions of source code must retain the above copyright notice,
   *   this list of conditions and the following disclaimer.
   * - Redistributions in binary form must reproduce the above copyright notice,
   *   this list of conditions and the following disclaimer in the documentation
   *   and/or other materials provided with the distribution.
   * - Neither the name of the libjpeg-turbo Project nor the names of its
   *   contributors may be used to endorse or promote products derived from this
   *   software without specific prior written permission.
   *
   * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS",
   * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
   * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
   * ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
   * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
   * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
   * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
   * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
   * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
   * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
   * POSSIBILITY OF SUCH DAMAGE.
   */
  
  #include <turbojpeg.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <stdint.h>
  #include <string.h>
  #include <unistd.h>
  
  
  #define NUMTESTS  7
  
  
  struct test {
    enum TJPF pf;
    int psv, pt;
  };
  
  
  extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
  {
    tjhandle handle = NULL;
    short *srcBuf = NULL;
    unsigned char *dstBuf = NULL;
    int width = 0, height = 0, fd = -1, i, ti;
    char filename[FILENAME_MAX] = { 0 };
    struct test tests[NUMTESTS] = {
      { TJPF_RGB, 1, 0 },
      { TJPF_BGR, 2, 2 },
      { TJPF_RGBX, 3, 4 },
      { TJPF_BGRA, 4, 7 },
      { TJPF_XRGB, 5, 5 },
      { TJPF_GRAY, 6, 3 },
      { TJPF_CMYK, 7, 0 }
    };
  #if defined(__has_feature) && __has_feature(memory_sanitizer)
    char env[18] = "JSIMD_FORCENONE=1";
  
    /* The libjpeg-turbo SIMD extensions produce false positives with
       MemorySanitizer. */
    putenv(env);
  #endif
  
    snprintf(filename, FILENAME_MAX, "/tmp/libjpeg-turbo_compress_fuzz.XXXXXX");
    if ((fd = mkstemp(filename)) < 0 || write(fd, data, size) < 0)
      goto bailout;
  
    if ((handle = tj3Init(TJINIT_COMPRESS)) == NULL)
      goto bailout;
  
    for (ti = 0; ti < NUMTESTS; ti++) {
      int sum = 0, pf = tests[ti].pf;
      size_t dstSize = 0, maxBufSize;
  
      /* Test non-default compression options on specific iterations. */
      tj3Set(handle, TJPARAM_BOTTOMUP, ti == 0);
      tj3Set(handle, TJPARAM_NOREALLOC, ti != 2);
      tj3Set(handle, TJPARAM_RESTARTROWS, ti == 0 || ti == 6 ? 1 : 0);
  
      tj3Set(handle, TJPARAM_MAXPIXELS, 1048576);
      /* tj3LoadImage12() will refuse to load images larger than 1 Megapixel, so
         we don't need to check the width and height here. */
      if ((srcBuf = tj3LoadImage12(handle, filename, &width, 1, &height,
                                   &pf)) == NULL)
        continue;
  
      maxBufSize = tj3JPEGBufSize(width, height, TJSAMP_444);
      if (tj3Get(handle, TJPARAM_NOREALLOC)) {
        if ((dstBuf = (unsigned char *)malloc(maxBufSize)) == NULL)
          goto bailout;
      } else
        dstBuf = NULL;
  
      tj3Set(handle, TJPARAM_LOSSLESS, 1);
      tj3Set(handle, TJPARAM_LOSSLESSPSV, tests[ti].psv);
      tj3Set(handle, TJPARAM_LOSSLESSPT, tests[ti].pt);
      if (tj3Compress12(handle, srcBuf, width, 0, height, pf, &dstBuf,
                        &dstSize) == 0) {
        /* Touch all of the output pixels in order to catch uninitialized reads
           when using MemorySanitizer. */
        for (i = 0; i < dstSize; i++)
          sum += dstBuf[i];
      }
  
      free(dstBuf);
      dstBuf = NULL;
      tj3Free(srcBuf);
      srcBuf = NULL;
  
      /* Prevent the code above from being optimized out.  This test should never
         be true, but the compiler doesn't know that. */
      if (sum > 255 * maxBufSize)
        goto bailout;
    }
  
  bailout:
    free(dstBuf);
    tj3Free(srcBuf);
    if (fd >= 0) {
      close(fd);
      if (strlen(filename) > 0) unlink(filename);
    }
    tj3Destroy(handle);
    return 0;
  }
  

Download