Changes in TIFF v4.0.9 — LibTIFF 4.7.2 documentation

# Changes in TIFF v4.0.9

References
Current Version | v4.0.9 (tag Release-v4-0-9)
Previous Version | v4.0.8
Master Download Site | https://download.osgeo.org/libtiff/
Master HTTP Site #1 | http://www.simplesystems.org/libtiff/
Master HTTP Site #2 | http://libtiff.maptools.org/

This document describes the changes made to the software between the previous and current versions (see above). If you don't find something listed here, then it was not done in this timeframe, or it was not considered important enough to be mentioned. The following information is located here:

## Major changes

- None

## Software configuration changes

- `test/Makefile.am`: Add some tests for tiff2bw.

- `.appveyor.yml`, `.travis.yml`, `build/travis-ci`: apply patches `0001-ci-Travis-script-improvements.patch` and `0002-ci-Invoke-helper-script-via-shell.patch` by Roger Leigh (sent to mailing list)

- `.travis.yml`, `build/travis-ci`: new files from `0001-ci-Add-Travis-support-for-Linux-builds-with-Autoconf.patch` by Roger Leigh (sent to mailing list on 2017-06-08). This patch adds support for the Travis-CI service.

- `.appveyor.yml`: new file from `0002-ci-Add-AppVeyor-support.patch` by Roger Leigh (sent to mailing list on 2017-06-08). This patch adds a `.appveyor.yml` file to the top-level. This allows one to opt in to having a branch built on Windows with Cygwin, MinGW and MSVC automatically when a branch is pushed to GitHub, GitLab, BitBucket or any other supported git hosting service.

- `CMakeLists.txt`, `test/CMakeLists.txt`, `test/TiffTestCommon.cmake`: apply patch `0001-cmake-Improve-Cygwin-and-MingGW-test-support.patch` from Roger Leigh (sent to mailing list on 2017-06-08). This patch makes the CMake build system support running the tests with MinGW or Cygwin.

- `test/tiffcp-lzw-compat.sh`, `test/images/quad-lzw-compat.tiff`: new files to test old-style LZW decompression

- `test/common.sh`, `Makefile.am`, `CMakeList.txt`: updated with above

- `test/Makefile.am`: add missing reference to images/quad-lzw-compat.tiff to fix `make distcheck`. Patch by Roger Leigh

- `nmake.opt`: support a `DEBUG=1` option, so as to adjust `OPTFLAGS` and use `/MDd` runtime in debug mode.

## Library changes

- `libtiff/tif_color.c`: `TIFFYCbCrToRGBInit()`: stricter clamping to avoid `int32` overflow in `TIFFYCbCrtoRGB()`. Fixes OSS-Fuzz #1844. Credit to OSS Fuzz

- `libtiff/tif_getimage.c`: `initYCbCrConversion()`: stricter validation for `refBlackWhite` coefficients values. To avoid invalid `float->int32` conversion (when `refBlackWhite[0] == 2147483648.f`) Fixes OSS-Fuzz #1907. Credit to OSS Fuzz

- `libtiff/tif_dirinfo.c`, `tif_dirread.c`: add `_TIFFCheckFieldIsValidForCodec()`, and use it in `TIFFReadDirectory()` so as to ignore fields whose tag is a codec-specified tag but this codec is not enabled. This avoids `TIFFGetField()` to behave differently depending on whether the codec is enabled or not, and thus can avoid stack based buffer overflows in a number of TIFF utilities such as tiffsplit, tiffcmp, thumbnail, etc. Patch derived from `0063-Handle-properly-CODEC-specific-tags.patch` (MapTools bugzilla #2580) by Raphaël Hertzog. Fixes: MapTools bugzilla #2580, MapTools bugzilla #2693, MapTools bugzilla #2625 (CVE-2016-10095), MapTools bugzilla #2564 (CVE-2015-7554), MapTools bugzilla #2561 (CVE-2016-5318), MapTools bugzilla #2499 (CVE-2014-8128), MapTools bugzilla #2441, MapTools bugzilla #2433.

- `libtiff/tif_swab.c`: if `DISABLE_CHECK_TIFFSWABMACROS` is defined, do not do the `#ifdef TIFFSwabXXX` checks. Make it easier for GDAL to rename the symbols of its internal libtiff copy.

- `libtiff/tif_dirread.c`: fix regression of libtiff 4.0.8 in `ChopUpSingleUncompressedStrip()` regarding update of newly single-strip uncompressed files whose bytecount is 0. Before the change of 2016-12-03, the condition `bytecount==0` used to trigger an early exit/disabling of strip chop. Re-introduce that in update mode. Otherwise this cause later incorrect setting for the value of `StripByteCounts`/`StripOffsets`. (GDAL trac #6924).

- `libtiff/tif_dirread.c`: `TIFFFetchStripThing()`: limit the number of items read in `StripOffsets`/`StripByteCounts` tags to the number of strips to avoid excessive memory allocation. Fixes OSS-Fuzz #2215. Credit to OSS Fuzz

- `libtiff/tif_getimage.c`: avoid many (harmless) unsigned int overflows.

- `libtiff/tif_fax3.c`: avoid unsigned int overflow in `Fax3Encode2DRow()`. Could potentially be a bug with huge rows.

- `libtiff/tif_jpeg.c`: avoid (harmless) unsigned int overflow on tiled images.

- `libtiff/tif_dirread.c`: avoid unsigned int overflow in `EstimateStripByteCounts()` and `BYTECOUNTLOOKSBAD` when file is too short.

- `libtiff/tif_predict.c`: decorate legitimate functions where unsigned int overflow occur with `TIFF_NOSANITIZE_UNSIGNED_INT_OVERFLOW`

- `libtiff/tif_dirread.c`: avoid unsigned int overflow in `EstimateStripByteCounts()`

- `libtiff/tiffiop.h`: add `TIFF_NOSANITIZE_UNSIGNED_INT_OVERFLOW` macro to disable CLang warnings raised by `-fsanitize=undefined,unsigned-integer-overflow`

- `libtiff/tif_jpeg.c`: add anti-denial of service measure to avoid excessive CPU consumption on progressive JPEGs with a huge number of scans. See http://www.libjpeg-turbo.org/pmwiki/uploads/About/TwoIssueswiththeJPEGStandard.pdf.

  Note

  Only affects libtiff since 2014-12-29 where support of non-baseline JPEG was added.

- `libtiff/tif_jpeg.c`: error out at decoding time if anticipated libjpeg memory allocation is above 100 MB. libjpeg in case of multiple scans, which is allowed even in baseline JPEG, if components are spread over several scans and not interleavedin a single one, needs to allocate memory (or backing store) for the whole strip/tile. See http://www.libjpeg-turbo.org/pmwiki/uploads/About/TwoIssueswiththeJPEGStandard.pdf. This limitation may be overridden by setting the `LIBTIFF_ALLOW_LARGE_LIBJPEG_MEM_ALLOC` environment variable, or recompiling libtiff with a custom value of `TIFF_LIBJPEG_LARGEST_MEM_ALLOC` macro.

- `libtiff/tif_jbig.c`: fix memory leak in error code path of `JBIGDecode()`. Fixes MapTools bugzilla #2706. Reported by team OWL337

- `libtiff/tif_dirread.c`: in `TIFFReadDirEntryFloat()`, check that a double value can fit in a float before casting. Patch by Nicolas RUFF

- `libtiff/tiffiop.h`, `libtiff/tif_jpeg.c`, `libtiff/tif_jpeg_12.c`, `libtiff/tif_read.c`: make `TIFFReadScanline()` works in `CHUNKY_STRIP_READ_SUPPORT` mode with JPEG stream with multiple scans. Also make configurable through a `LIBTIFF_JPEG_MAX_ALLOWED_SCAN_NUMBER` environment variable the maximum number of scans allowed. Defaults to 100.

- `libtiff/tif_read.c`: `TIFFFillTile()`: add limitation to the number of bytes read in case td_stripbytecount[strip] is bigger than reasonable, so as to avoid excessive memory allocation (similarly to what was done for `TIFFFileStrip()` on 2017-05-10)

- `libtiff/tif_getimage.c`: use `_TIFFReadEncodedStripAndAllocBuffer()`. Fixes MapTools bugzilla #2708 and OSS-Fuzz #2433. Credit to OSS Fuzz

- `libtiff/tif_read.c`, tiffiop.h: add a `_TIFFReadEncodedStripAndAllocBuffer()` function, variant of `TIFFReadEncodedStrip()` that allocates the decoded buffer only after a first successful `TIFFFillStrip()`. This avoids excessive memory allocation on corrupted files.

- `libtiff/tif_dirwrite.c`: in `TIFFWriteDirectoryTagCheckedXXXX()` functions associated with LONG8/SLONG8 data type, replace assertion that the file is BigTIFF, by a non-fatal error. Fixes MapTools bugzilla #2712 Reported by team OWL337

- `libtiff/tif_read.c`: `TIFFStartTile()`: set tif_rawcc to tif_rawdataloaded when it is set. Similarly to `TIFFStartStrip()`. This issue was revealed by the change of 2017-06-30 in `TIFFFileTile()`, limiting the number of bytes read. But it could probably have been hit too in CHUNKY_STRIP_READ_SUPPORT mode previously ? Fixes OSS-Fuzz #2454 Credit to OSS Fuzz

- `libtiff/tif_error.c, tif_warning.c`: correctly use va_list when both an old-style and new-style warning/error handlers are installed. Patch by Paavo Helde (sent on the mailing list)

- `libtiff/tif_getimage.c`: use `_TIFFReadTileAndAllocBuffer()`. Fixes OSS-Fuzz #2470 Credit to OSS Fuzz.

- `libtiff/tif_read.c`, tiffiop.h: add a `_TIFFReadEncodedTileAndAllocBuffer()` and `_TIFFReadTileAndAllocBuffer()` variants of `TIFFReadEncodedTile()` and `TIFFReadTile()` that allocates the decoded buffer only after a first successful `TIFFFillTile()`. This avoids excessive memory allocation on corrupted files.

- `libtiff/tif_pixarlog.c`: avoid excessive memory allocation on decoding when RowsPerStrip tag is not defined (and thus td_rowsperstrip == UINT_MAX) Fixes OSS-Fuzz #2554 Credit to OSS Fuzz

- `libtiff/tif_lzw.c`: fix 4.0.8 regression in the decoding of old-style LZW compressed files.

- `libtiff/tif_lzw.c`: fix potential out-of-buffer read on 1-byte LZW strips. Crashing issue only on memory mapped files, where the strip offset is the last byte of the file, and the file size is a multiple of one page size on the CPU architecture (typically 4096). Credit to myself :-)

- `libtiff/tif_dir.c`: avoid potential null pointer dereference in `_TIFFVGetField()` on corrupted TIFFTAG_NUMBEROFINKS tag instance. Fixes MapTools bugzilla #2713

- `tools/tiff2pdf.c`: prevent heap buffer overflow write in "Raw" mode on `PlanarConfig=Contig` input images. Fixes MapTools bugzilla #2715 Reported by team OWL337

- `libtiff/tif_read.c`: `TIFFFillStrip()` / `TIFFFillTile()`. Complementary fix for MapTools bugzilla #2708 in the `isMapped()` case, so as to avoid excessive memory allocation when we need a temporary buffer but the file is truncated.

- `libtiff/tif_read.c`: `TIFFFillStrip()` / `TIFFFillTile()`. Complementary fix for MapTools bugzilla #2708 in the `isMapped()` case, so as to avoid excessive memory allocation when we need a temporary buffer but the file is truncated.

- `libtiff/tif_read.c`: in `TIFFFetchStripThing()`, only grow the arrays that hold StripOffsets/StripByteCounts, when they are smaller than the expected number of striles, up to 1 million striles, and error out beyond. Can be tweaked by setting the environment variable `LIBTIFF_STRILE_ARRAY_MAX_RESIZE_COUNT`. This partially goes against a change added on 2002-12-17 to accept those arrays of wrong sizes, but is needed to avoid denial of services. Fixes OSS-Fuzz #2350 Credit to OSS Fuzz

- `libtiff/tif_read.c`: in `TIFFFetchStripThing()`, only grow the arrays that hold `StripOffsets`/`StripByteCounts`, when they are smaller than the expected number of striles, up to 1 million striles, and error out beyond. Can be tweaked by setting the environment variable `LIBTIFF_STRILE_ARRAY_MAX_RESIZE_COUNT`. This partially goes against a change added on 2002-12-17 to accept those arrays of wrong sizes, but is needed to avoid denial of services. Fixes OSS-Fuzz #2350 Credit to OSS Fuzz

- `libtiff/tif_read.c`: add protection against excessive memory allocation attempts in `TIFFReadDirEntryArray()` on short files. Effective for mmap'ed case. And non-mmap'ed case, but restricted to 64bit builds. Fixes MapTools bugzilla #2675

- `libtiff/tif_read.c`: add protection against excessive memory allocation attempts in `TIFFReadDirEntryArray()` on short files. Effective for mmap'ed case. And non-mmap'ed case, but restricted to 64bit builds. Fixes MapTools bugzilla #2675

- `libtiff/tif_luv.c`: `LogLuvInitState()`: avoid excessive memory allocation when `RowsPerStrip` tag is missing. Fixes OSS-Fuzz #2683 Credit to OSS-Fuzz

- `libtiff/tif_getimage.c`: `gtTileContig()` and `gtTileSeparate()`: properly break from loops on error when `stoponerr` is set, instead of going on iterating on row based loop.

- `libtiff/tif_getimage.c`: fix fromskew computation when to-be-skipped pixel number is not a multiple of the horizontal subsampling, and also in some other cases. Impact `putcontig8bitYCbCr44tile`, `putcontig8bitYCbCr42tile`, `putcontig8bitYCbCr41tile`, `putcontig8bitYCbCr21tile` and `putcontig8bitYCbCr12tile`. Fixes MapTools bugzilla #2637 (discovered by Agostino Sarubbo) and OSS-Fuzz #2691 (credit to OSS Fuzz)

- `libtiff/tif_luv.c`: further reduce memory requirements for temporary buffer when `RowsPerStrip >= image_length` in `LogLuvInitState()` and `LogL16InitState()`. Fixes OSS-Fuzz #2700 Credit to OSS Fuzz

- `libtiff/tif_dirwrite.c`: replace assertion related to not finding the `SubIFD` tag by runtime check (in `TIFFWriteDirectorySec()`) Fixes MapTools bugzilla #2727 Reported by team OWL337

- `libtiff/tif_dirwrite.c`: replace assertion to tag value not fitting on `uint32` when selecting the value of `SubIFD` tag by runtime check (in `TIFFWriteDirectoryTagSubifd()`). Fixes MapTools bugzilla #2728 Reported by team OWL337

- `libtiff/tif_jpeg.c`: accept reading the last strip of a JPEG compressed file if the codestream height is larger than the truncated height of the strip. Emit a warning in this situation since this is non compliant.

- `libtiff/tiffiop.h`, `tif_aux.c`: redirect `SeekOK()` macro to a `_TIFFSeekoK()` function that checks if the offset is not bigger than `INT64_MAX`, so as to avoid a `-1` error return code of `TIFFSeekFile()` to match a required seek to `UINT64_MAX`/`-1`. Fixes MapTools bugzilla #2726 Adapted from proposal by Nicolas Ruff.

- `libtiff/tif_dirread.c`: add `NULL` check to avoid likely false positive null-pointer dereference warning by CLang Static Analyzer.

- `libtiff/libtiff.def`: add `TIFFReadRGBAStripExt()` and `TIFFReadRGBATileExt()` Fixes MapTools bugzilla #2735

- `libtiff/tif_jpeg.c`: add compatibility with libjpeg-turbo 1.5.2 that honours `max_memory_to_use > 0`. Cf https://github.com/libjpeg-turbo/libjpeg-turbo/issues/162.

- `libtiff/tif_getimage.c`: avoid floating point division by zero in `initCIELabConversion()` Fixes OSS-Fuzz #3733 Credit to OSS Fuzz

## Tools changes

- `tools/tiff2pdf.c`: prevent heap buffer overflow write in "Raw" mode on `PlanarConfig=Contig` input images. Fixes MapTools bugzilla #2715 Reported by team OWL337

- `tools/tiffset.c`: fix setting a single value for the `ExtraSamples` tag (and other tags with variable number of values). So `tiffset -s ExtraSamples 1 X`. This only worked when setting 2 or more values, but not just one.

- `tools/fax2tiff.c` (`_FAX_Client_Data`): Pass `FAX_Client_Data` as the client data. This client data is not used at all at the moment, but it makes the most sense. Issue that the value of `client_data.fd` was passed where a pointer is expected was reported via email by Gerald Schade on Sun, 29 Oct 2017.

- `tools/tiff2pdf.c` (`t2p_sample_realize_palette`): Fix possible arithmetic overflow in bounds checking code and eliminate comparison between signed and unsigned type.

- `tools/tiff2bw.c` (`main()`): Free memory allocated in the tiff2bw program. This is in response to the report associated with CVE-2017-16232 but does not solve the extremely high memory usage with the associated POC file.

## Contributed software changes

None