Understanding Image ArchivingDuring scanning, image scanners running with ScanTools Plus can capture image clips of OMR grids, bar codes, grayboxes, and whole sheet sides. These image clips can be viewed during post-scan editing of the data record and, optionally, they can be archived for later access by your image archiving system. Archiving is configured as part of your application definition process.
There are two ways to archive images in ScanTools Plus:
-
Post-scan image archiving is a manual process. After the forms are scanned, you click a button in the Scanning Module to process the images into an archive format.
Post-scan image archiving can sometimes take longer than scanning, due to:
- Time required to get and decompress the scanned image and the background form image. This varies based on the compression rate of these images.
- Time required to combine the two images. This should not vary except by image area clipped.
- Time required to compress and write the compressed image. This can vary greatly based on the chosen compression type.
- Compressibility of the image.
- Performance of the PC being used for archiving.
- Speed of the full storage path where the image is to be stored.
- Network traffic/server load if the results are written to network storage.
Before you can perform post-scan image archiving, you must create and save a ScanTools application with all appropriate output fields and grid/graybox/bar code definitions. The following items are used to manage post-scan archiving:
If using a background form image, and your form has the required form ID mark, you can use the OMR/Merge Horizontal Adjustment in the Application Settings dialog box to improve the alignment of the background form image to the scanned image.
For each sheet of each document, select parameters on the Document and Sheet Properties dialog box - Clips Tab to manage whether to save images as bitone or grayscale, what compression type to use, whether to take individual clips or whole sheet clips, and so forth.
If you have the graphic .tif file of the Background Form Image, select the image on the Document and Sheet Properties dialog box - Background Tab to merge it with the individual clip so you can view the scanned data in the context of the form.
For each field for which you want an image clip file produced, check Archive this Clip in the Archive section of the Clip Attributes dialog box. Then, enter a meaningful and unique File Name Suffix. The name you enter is used as part of the image clip file name. If you are using a background form image (see Image Clips dialog box above) and you want it merged with the individual grid clip, check Combine with Form. If you selected the Sparse (individual clips) method in the Image Clips dialog box, then select the Clip Grayscale (grayscale/bitone choice) and Clip Compression type for each individual grid. Using this method lets you have different clip parameters for separate clips.
NOTE: When archiving linked grids, only the last grid in the link is included in the image clip unless you take corrective action. Correct this by expanding the clip region (drag the edge of the clip region while pressing the CTRL key) to include all grids that are linked before you click Append. If you forget to do this before clicking Append, you will need to redefine the linked grids being sure to expand the clip region before clicking Append.
Archive indices use is optional. If you use it, use the Archive Indices dialog box to select all fields you want available as index entries in the archive index file for each document. These field names and data will then be entered into the archive index file. If you do not use archive indices, the index file will not contain field names or data.
NOTE: We recommend that you do not re-archive data files if the application has been modified after scanning.
-
ScanTools Plus includes a utility called iNAME that has scan-time image archiving. During scanning, iNAME image archiving is performed automatically using the parameters stored in the .ini file you created through the iNAME configuration process.
- Scan-Time Archiving - Image files are created during scanning. There is no need to archive as a separate action after scanning.
- Full Sheet Image Archiving - A single sheet with no form ID marks and/or no timing marks can be archived. To do this, your application must be defined as a single document with a single sheet. Each side of the sheet can be archived as a separate file. For details, see Archiving Single Sheets without Form ID Marks or Timing Marks.
- Integrated iNAME Configuration - iNAME image archiving can be an integral part of your ScanTools application. Image format parameters can be defined and iNAME Configuration used to define images to archive, archive directories and file names, and archive indexing. All of this is accomplished during application definition. For details, see Application Definition for Scan-Time Archiving with iNAME.
- Archive Directory Path - An image file destination location, using a Base path name and a Variable path name, can be user-defined. For details, see iNAME Configuration - Paths and Indexes Screen.
- Archive Filenames - Image filenames, using a Base filename and an optional Variable filename, can be user-defined. For details, see iNAME Configuration - Paths and Indexes Screen.
- Index Entries -
- Images to Archive - You may select which grids or whole sheet images to archive, based on all available in the ScanTools application. For details, see iNAME Configuration - Images To Archive Definition Screen.
- iNAME Merge - This post-scan process can be set up during iNAME configuration to run automatically, after scanning is completed. It lets you merge a background form image into the individual image clip files that were generated during scanning. It can perform any image rotation that may be needed. The image clip files generated during scanning do not contain the background form image, and are not rotated at scan time. It also supports image scaling to 200, 240, 300, 400, or 600 DPI. iNAME Merge can also be run manually, after scanning is completed. For details, see Using iNAME Merge.
- Application Migration - If you are using an application that was created in an earlier version of ScanTools software
Before you can perform scan-time image archiving, you must create and save a ScanTools application with all appropriate output field and grid, graybox, and bar code definitions, plus the additional application settings, image clipping definitions, and archiving configuration settings required for iNAME image archiving. The following items are used to manage iNAME image archiving:
In the Application Settings dialog box, for all documents in the application:
- Select Enable iNAME for Before User Exits or After User Exits if your application includes a user exit. Either selection is acceptable if your application does not include a user exit.
- Optionally, select Archive Image Scaling and/or OMR/Merge Horizontal Adjustment.
- Check the Release for Scanning check box when the application is completely defined and is ready to be used for scanning.
In the Document and Sheet Properties Dialog - Clips Tab, for each sheet of each document:
- Select Whole Sheet Capture Options.
- Select a Save As option (specifies bitone and compression)
This sets the image format parameters for all whole sheet clips generated by iNAME to the same values. If your archiving requirement calls for individual clips of grids, bar codes, or grayboxes, iNAME Configuration allows you to specify the use of Document Image Format Parameters (i.e. use the same image format parameters specified for whole sheet clips) or Grid Image Format Parameters when selecting an image to archive. If you are using Grid Image Format Parameters for your individual clips, change the Clipping Mode to Sparse on the Image Clips dialog box and then, on the Clip Attributes dialog box associated with each grid for which iNAME will generate an image clip, select the desired image format parameters.
The background form image files that are used by iNAME Merge are identified in the Background Top/Bottom Form Image group boxes. You can specify these files in the Add/Replace Background Form Images for Sheet # dialog box.
CAUTION: Sparse clipping of software bar codes and, with iNAME, of grayboxes, is not compatible with use of absolute (.001 in or .01 mm) or Hybrid units of measurement. See Grid Measurement dialog box - Unit of Measurement.
In the Document and Sheet Properties dialog box - Background Tab, enter the names and paths to the background form images that will be used to merge with the scanned data.
In the Clip Attributes dialog box - Archive group box, for each field you want an image clip file produced:
- To prevent excessive disk space and processing time from being used by scan-time archiving during scanning, do NOT select Archive this Clip. It is only used with post-scan archiving.
- It is not necessary to select Combine with Form when using the post-scan iNAME Merge function to merge a background form image with an image clip, if the background form image is specified in the Add/Replace Background Form Images for Sheet # dialog box.
- Enter a unique File NameSuffix. The name you enter is then available as an Image to Archive when you use iNAME configuration.
If you cleared (or did not check) Whole sheet in the Document and Sheet Properties Dialog - Background Tab, you can also change Grayscale, Compression, and other related image format parameters in the separate Clip Attributes dialog boxes of each grid you image clip.
NOTE: When archiving linked grids, only the last grid in the link is included in the image clip unless you expand the clip region (CTRL + drag the edge of the clip region) to include all grids that are linked before you click Append.
In the iNAME Configurationdialog box, for all documents in the application:
- Select images to archive, at scan-time, from the list of available images in your application. This includes image clips for whole sheets and for individual grids.
- Define the order in which images are archived.
- In the Image To Archive Information dialog box (click the image, then click Edit), specify whether to use the document image format parameters from the Document and Sheet Properties dialog box, Clips tab (Whole Sheet setting) or the grid image format parameters from the Clip Attributes dialog box(individual clip settings).
In the iNAME Configuration - Paths and Indexes dialog box, for all documents in the application:
- Define a Base Directory where your archived images are stored. This is required.
- Define Variable Directory paths where your archived images are stored. This can be a subdirectory of the Base Directory or you can append to the Base Directory name. This is required.
- Define a Base Filename for your image files. This is optional.
- Define Variable Filenames for your image files. This is required.
- Select the Index file format you want to use.
- Select the Image file type you want to create, either a Single Image or Multi-image file.
- Define how iNAME image archiving works with user exits, if any.
- Select whether or not to run iNAME Merge automatically, after scanning. If you select it, background form images are merged as soon as scanning is complete. If you do not select it, you can merge background form images later, manually.
- If needed, define a Power User proxy to enable iNAME image archiving and iNAME Merge to perform tasks that your application user has insufficient privileges to perform.
- Create any number of archive indexes to enable your archiving system to most effectively access the image files.
If you are using iNAME image archiving and edit correction is required, use post-scan edit correction. If you want to view the image clip of a failing field during post-scan editing, you must select the same edit profile during scanning that you will use for post-scan edit correction. Image clips of edit failures are created during scanning, using the edit profile selected at scan time. A different edit profile can be selected for post-scan editing; however, there may not be image clips displayed in the post-scan editor, depending on how the edit profiles differ. A field could have passed editing during scanning but failed using a different edit profile in post-scan editing, therefore the image clip was not captured during scanning.
CAUTION: Sparse clipping of software bar codes and (with iNAME) of grayboxes is not compatible with use of absolute (.001 in or .01 mm) or Hybrid units of measurement.
During scanning, all image clips are stored in a temporary folder. This folder is automatically deleted when scanning is completed. From the temporary folder, the image clips are moved into the specified folder you defined in the iNAME configuration.
CAUTION: Do not open either the temporary folder or final destination folders (using Windows Explorer) until scanning is complete.
If a path or filename was defined to use the contents of a field (Field Data), and during scanning a multiple character was encountered in that field, the corresponding name generated by iNAME image archiving will contain an alphabetic character in the same position as the multiple that was found in the field. A multiple usually generates an asterisk (*), however, asterisks cannot be used in filenames. For example:
The filename is defined to use the contents of the Name field.
The name scanned is Smi*h, with a multiple in the 4th position.
iNAME produces a file of SmiXh, where X is any alphabetic character.
If a path or filename was defined for a specific number of characters and the path or filename generated from the field selected was not as long, the name generated will have underscores (__) added to the end to fill in the length defined. In the same manner, if the field defined to generate a path or filename had no data (was blank), underscores are generated for all characters in the field. During post-scan editing you can change the underscores to meaningful characters.
If field data is corrected during editing, file/directory names and/or indexes that use this field data are also updated.
NOTE: You cannot perform Online edit correction using iNAME scan-time archiving. You can only implement archiving during post-scan editing. The image clips taken during scanning are used for post-scan editing. To ensure you have the image clips you need for post-scan editing, select the edit profile during scanning and use the same edit profile later during post-scan editing. If you use a different edit profile than the one used during scanning, you may not have the image clips to view in the post-scan editor. During post-scan editing, iNAME applies any edit changes to the directory, file names, and indexes which use field data to reference the image clips.
Regardless of which archiving method you choose, you also must decide what type of clips you want:
-
Whole sheet clipping might be done in cases where it is important to capture every possible comment that might be made on a page, or where the paper needs to be reconstituted from the clip and the background form image. When you use whole sheet clipping, ScanTools clips the whole page instead of a single grid. If the data file must be edited after scanning, the entire page is visible. Because the smaller clips defined for each grid exist in the clip file as "phantom" clips, the editor is still able to highlight specific areas on the form.
-
Sparse clipping means only specified grid image clips are saved. In the case of an OMR document with no edit failures and no archiving, no image clips are saved. Sparse clipping can be helpful because it is fast—no time is wasted compressing and saving area on the form that you do not want. Sparse clipping also allows you to clip different grids at different pixel depths, using different compression methods. This allows you to clip a portion of a form in grayscale for KFI, while you clip another portion in bitone for input to an archiving system.
NOTES:
- iNAME archiving allows you to select both whole sheet and sparse clips. Post-scan archiving requires you to choose either whole sheet clips or sparse clips; ScanTools Plus does not support archiving both clip types post-scan.
- If you use iNAME archiving and intend to archive both clip types, there is a specific way you must set the whole page clip properties. See How do I…
Image archiving produces two output file types:
-
Each clip (whole sheet or sparse (grid)) generates its own image file. Scan-time archiving allows a greater choice of assigning file names. Post-scan archiving uses the six-digit document serial number assigned during scanning, plus the suffix defined in the application definition. The archive clips have a .tif file extension for Packbits RLE, CCITT G3 Fax, and CCITT G4 Fax compression. They have a .jpg file extension for JPEG compression.
You can write archive clips in any of the supported compression formats and bit depths. If you do not use a background form .tif file (see What about background images, below for details) , the clip file is written with the same bit depth as defined in the ScanTools Plus application.
When you use a background form .tif file, the compression (bit depth) you specified for the clip is also applied to the background form image, as follows:
- Bitone clip: The background form is converted to a bitone image and a single bitone image is written to disk.
- 4-bit clip (only supported with RLE compression): The background form is mapped into 16 gray tones, resulting in a 4-bit grayscale .tif file.
- 8-bit non-JPEG clip: 240 palette entries are reserved for grayscale data from the clip, and 16 palette entries are reserved for the most popular colors from the form, resulting in an 8-bit palleted color .tif file.
- JPEG clip: The clip file is a 24-bit color JPG file.
For details on clip settings, see Document and Sheet Properties Dialog - Clips Tab or Clip Attributes Dialog.
-
ScanTools generates a single index file containing the index values for all the image clip files generated. The index file is not used by ScanTools but is created for the archiving system that will be used to store and access the image files. The index file format is selected for:
- Post-scan archiving in the Archive Options dialog box.
- Scan-time archiving in iNAME Configuration.
NOTE: The file name in the archive index will be set to *NOT TAKEN*** if Conditional Clipping for Image Archiving was enabled for a graybox and the graybox clip region did not contain data.
The index file has one line for each archive clip taken. Each line contains the following data:
- Document name.
- File suffix of the associated clip (grid).
- Serial number of the associated document.
- Name of the associated image file. When using iNAME scan-time archiving, the full path name is included.
- Pairs of index fields and the associated data. Index/data pairs are delimited by the tilde (~) character.
Each field in the index information is delimited by the tab character.
Each line is delimited by a carriage return line feed.
Though this index file format is unlikely to be directly usable by any archiving software as it stands, it is designed so that it can be easily manipulated into some other format. It contains data sufficient for an application to easily extract all the clips for either a document, a serial number, or a clip suffix.
The following is a sample ScanTools index (.ndx) file created during post-scan archiving.
GPAS(tab)PH(tab)000001(tab)000001PH.TIF(tab)Name~John Doe(tab)ID~6128920316
GPAS(tab)SIG(tab)000001(tab)000001SIG.TIF(tab)Name~John Doe(tab)ID~6128920316
GPAS(tab)PH(tab)000002(tab)000002PH.TIF(tab)Name~Mary Richards
GPAS(tab)SIG(tab)000002(tab)000002SIG.TIF(tab)Name~Mary Richards
GPAS(tab)PH(tab)000003(tab)000003PH.TIF(tab)Name~Jay Smith(tab)ID~6126832231
GPAS(tab)SIG(tab)000003(tab)000003SIG.TIF(tab)Name~Jay Smith(tab)ID~6126832231
This example is for 3 serial numbers that are a mix of the "Single Sided GPAS" and "Double Sided GPAS" documents in sample application 701. The single sided form is indexed by name and ID number while the double sided form is indexed by name only. Each document archives a photo, designated by suffix PH, and a signature, designated by suffix SIG.
When you scan forms printed in "dropout" colors, much of the visual formatting provided by the titles, captions, lines, and boxes on the form is electronically removed to improve data collection speed and accuracy. Since this same visual formatting provides context to the information on archived form images, ScanTools software enables you to merge the data on the image clip with a background form image, either during post-scan archiving or using iNAME. This image is a .tif file that should be supplied by the vendor who designed your form (although ScanTools also provides a way for you to create a background image by scanning a sample form). It usually contains just the parts of the image that drop out during scanning.
If you do not choose to Run iNAME Merge After Scanning when you configured iNAME, you can run it manually after scanning is completed and the image clip files are in the appropriate directory. iNAME Merge enables you to perform any combination of the following:
- Merge a background form image into the image clip files that were generated during scanning.
- Rotate an image clip for better viewing.
- Scale images to match your application.
See Using iNAME Merge for details on merging background images when archiving.
Background form images are also used during application definition. They appear in the grid definition area of the Application Module to make it easier for you to locate and define grids.