Validating TIFFs with embedded color profiles

About ICC profile validation

In the last days we enhanced our baseline TIFF conformance checking tool "checkit_tiff" with support to validate TIFFs containing embedded ICC profiles.

There are two specifications of ICC profiles. The first and older one is from the year 2001 and available at http://www.color.org/ICC_Minor_Revision_for_Web.pdf. The second one (and current version) is  http://www.color.org/specification/ICC1v43_2010-12.pdf.

The ICC profile standards are very complex and do not really fit in the design of the checkit_tiff configuration files, so we decided to check only the headers of the embedded ICC profiles for now.

Because the differences between the headers of both standards are very marginal at first sight, this could be simple enough to additionally implement it in checkit_tiff.

For more detailed checks we would need a plugin system to delegate validation of embedded standards (as ICC, XMP and so on) to more highly specialized validators.

Ooops, our tool finds more broken TIFFs again

To test the validation, we ran checkit_tiff against some old tiffs from our digitization collection. As you can see in the following picture, the checkit_tiff tool detects a discrepancy between the size of the embedded ICC-profile as reported in TIFF-tag (34675) and the size reported by the ICC profile itself:

After some analysis we made an interesting observation. The given TIFF reports a size of 13691 bytes (in hex: 0x357b), but the ICC itself (tag "profile size") reports a size of 669051 bytes (in hex: 0xa357b). We thought we had a bug in our TIFF tag or ICC header decoding, but other tools (exiftool or tiffdump) also reported this difference.

As we saw in previous TIFFs, some TIFF implementations are wrong and often some information were missing because of off-by-one errors. In this case, it seems that the software "Omniscan 12.8 Build2476" does not correctly fill the Tiff-tag 34675 for ICC-profile, leaving out the most significant byte (here: '0xa').

We need your help

Anyway. Writing a conformance checker is a tough job and sometimes we introduce bugs in this kind of software as well. Therefore, please have a look at the code of checkit_tiff, test it and send us bug reports if you find any bugs.
The code can be found on https://github.com/SLUB-digitalpreservation/checkit_tiff.

Here are some open questions about ICC profiles that we have encountered:

• We found some TIFFs which reported ICC version 4.2.0, but on http://www.color.org we only found versions 4.3.0 and 2.4.0. Could you help us to find out the differences in the headers for the different versions?

• The meaning of the field "preferred CMM type" is unclear. We assume that the list of allowed strings is part of the document http://www.color.org/registry/signature/TagRegistry-2016-05.pdf. But we have also found a TIFF where the string 'Lino' is set for this ICC header field. Are we interpreting the document in a wrong way, or has the tag been set to an incorrect value?

Konfigurierbarer TIFF-Validator in Arbeit…

Die bestehenden TIFF - Validatoren/Extraktoren, wie JHove und Co. sind zwar ganz nett, allerdings ist es schwierig unsere Policy abzufragen, sprich, welche Tags mit welchen Werten erlauben wir, welche sind Pflicht.

Dies war die Geburtsstunde für einen konfigurierbaren Validator. Die Regeln werden in einer einfachen Abfragesprache formuliert, die der Validator übersetzt und danach das TIFF entsprechend auswertet.

Es liegt noch etliche Arbeit vor uns, und einiges an Funktionalität fehlt noch. Dennoch möchten wir Euch auf das Tool hinweisen, damit ihr uns vlt. schon frühzeitig Rückmeldungen geben könnt.

Den Quellcode gibt es hier: https://github.com/SLUB-digitalpreservation/fixit_tiff/tree/master/checkit, der Code steht unter der gleichen Lizenz, wie der der LibTIFF (sh. http://www.libtiff.org/)

Hier ein Beispiel für eine einfache Abfrage:

# tag; required; values
#
# works as whitelist
# * tag: any tag number in TIFF
# * required: mandatory | optional | depends( $tag.$value ) 
# * values: range($start, $end) | logical_or( $a, …) | only($a) |any

# This defines a sample config for baseline tiffs
# Remember, because it works as whitelist for each required tag we need an
# entry here

#####
# Required Baseline Tags
#####

# 256 0100 ImageWidth The number of columns in the image, i.e., the number of pixels per row.
256; mandatory; range(1, 4294967295)

# 257 0101 ImageLength The number of rows of pixels in the image.
257; mandatory; range(1, 4294967295)

# 258 0102 BitsPerSample Number of bits per component.
##########################258; mandatory; logical_or(8,16)
# Bitonal is optional, grey & RGB is mandatory. If 262 AND 258 exist, then the values need to be in the specified range.
258; depends(262.2); only(8,8,8)
258; depends(258.any); logical_or(4, 8)

# 259 0103 Compression Compression scheme used on the image data 
### (1 means no compression)
259; mandatory; only(1)

# 262 0106 PhotometricInterpretation The color space of the image data.
### 2 means RGB, 0 and 1 means bilevel or grayscale, 0 is unusual, 3 is Palette Color (FORBIDDEN), 4 is Transparency Mask (FORBIDDEN)
262; mandatory; range(0, 2)

# 273 0111 StripOffsets For each strip, the byte offset of that strip.
273; mandatory; any

# 277 0115 SamplesPerPixel The number of components per pixel.
### if RGB then 3 else 1
### Even though Baseline TIFF allows for SamplesPerPixel>3, we do NOT allow this for long term archival.
277; depends(262.2); only(3)
277; depends(262.1); only(1)
277; depends(262.0); only(1)

# 278 0116 RowsPerStrip The number of rows per strip.
278; mandatory; range(1, 4294967295)

# 279 0117 StripByteCounts For each strip, the number of bytes in the strip after compression.
279; mandatory; range(1, 4294967295)

# 282 011A XResolution The number of pixels per ResolutionUnit in the ImageWidth direction.
282; mandatory; range(300, 1200)

# 283 011B YResolution The number of pixels per ResolutionUnit in the ImageLength direction.
283; mandatory; range(300, 1200)

# 296 0128 ResolutionUnit The unit of measurement for XResolution and YResolution. 1 = No absolute unit of measurement. 2 = Inch. 3 = Centimeter. Default: 2
296; mandatory; only(2)