How to update fields and metadata using Overlay

DISCO provides you with the capability to edit documents you have already ingested into a review database, by utilizing overlay files. An overlay is a CSV or DAT file that, when applied to a review database via the overlay process in DISCO, can change the metadata, image, OCR text, or native file of a document.  

Preparing your overlay file 

1. Organize the files and images you would like to update in the database into a folder. 
2. Create your overlay file as a CSV or DAT file with individual columns for each field you want to update.
3. The overlay file must have at least two columns:
   • A unique ID to match the new images to the field in DISCO. You can use either the DISCO Document ID or Instance ID field 
   • A column for each field and or tag you want to update.

To begin the overlay

1. In the DISCO main menu, click Overlay.
   
2. Click New Overlay.
   
3. On the Name and description page, enter a name and an optional description for the overlay, and then click Next. The overlay name will be displayed on the card.
   
4. On the File selection page, navigate to and select the overlay file. Ensure that the referenced files are contained in a single folder and that any file paths are relative to the overlay file. Relative paths mean the file paths for referenced files begin with a .\ or with the folder name of the parent folder.
   
5. Click Next to proceed to the Field Mapping page. Here, you can select the delimiters used between fields in your overlay file. The three types of delimiters are:
   • Field separator - A character separating the columns.
   • Field quote - The character used to “quote” or group multiple items together, like a phrase. The field quote is particularly useful when overlaying a custodian in last name, first name format (e.g., “Jones, George”), where the comma is also used as a field separator.

   • Multi-value delimiter - A character used to delimit multiple values within a column. In the following example, commas and semicolons are the delimiters: 

     ,“Kinsley, Dave”;”Slovacek, Jerry”,

     If you do not have a multi-value delimiter, leave the field blank

6. DISCO will pre-populate the delimiters based on the selected overlay file’s extension. These values can be changed by selecting a different value from the dropdown lists or typing in a new one.
7. You will now see the section where you can map the fields in your overlay file to the corresponding system fields in DISCO. This is how you tell DISCO what to overlay where. In DISCO, we store individual copies of every file uploaded, also known as instances. 
   We combine the data from duplicate instances into review documents. Some fields are stored on individual instances, while others are stored on the document level. When overlaying, the identifier you select may change the behavior of the overlay. See Overlayable fields for more information.
8. You must first establish a method of relating the documents in the overlay file to the documents in the review database. This is often referred to as a control key. When selected, the Overlay file field identifier dropdown will display all of the column names in your overlay file. Select the column name containing the values DISCO will use to identify the documents in the database. DISCO will read the first 25 rows of the column you select and auto-populate the corresponding DISCO identifier type: Instance ID, Document ID, or Bates number. The DISCO field identifier can be manually changed from what is auto-populated.
   
   The following fields must use the Instance ID as the DISCO identifier to be updated:
   1. Bates Numbers: BeginBates, EndBates
   2. Dates: LastAccessedDate, LastAccessedTime, LastModifiedDate, LastModifiedTime, LastPrintedDate, LastPrintedTime, CreateDate, CreateTime, ReceivedDate, ReceivedTime
   3. Dedupe fields: Custodian, FileName, OriginalFilepath
   4. Email Metadata: EmailInReplyTo, EmailMessageID, EmailPrConversationIndex, EmailRead, EmailReferences, EmailTransportHeaders
9. To add tags, map your columns with tags to the Tags field in DISCO or select an existing tag name to map to existing tags in your review database. 
10. To update the image for a document, map the column in your overlay file that contains the relative path to your image files to the ImagePath field in DISCO.
11. Next, map any desired remaining fields in your overlay file to corresponding DISCO fields. Not all overlay-file fields must be mapped and not all DISCO fields must be used. DISCO automatically attempts to match the DISCO field name to the corresponding overlay-file field name. 
    To map or change the mapping of the column in your overlay file, click on the corresponding pre-mapped DISCO field or unmapped placeholder. If a matching DISCO field already exists, select it from the dropdown list. If a matching DISCO field does not already exist, you can create a new custom field in the database by typing the name of the new custom field or selecting a custom field name supplied in the overlay file.

12. 

    By default, DISCO skips blank values in the overlay file and overwrites the values already in the database with the values provided in the overlay file. To override this behavior for an overlay field, select the APPEND or WRITE BLANKS checkboxes.

    • APPEND - The default behavior in DISCO is to replace (overwrite) existing information with the information contained in the overlay. Selecting the APPEND checkbox appends the overlay information to the existing information in the field so that the original information is still accessible.
    • WRITE BLANKS - The default behavior in DISCO is to skip blank values in the overlay file. Selecting the WRITE BLANKS checkbox overwrites a blank value over the existing information, deleting it.
      

• If the field mapping tool does not properly display your data load-file fields or displays an error message, there is something wrong with your load file’s delimiters. Please ensure your overlay file is a valid text delimited UTF-8 or ANSI text file.

1. Once you’ve finished mapping all the fields, click Next.
2. On the Overlay options page, select the format you would like dates to appear.
3. Review everything on the Summary page and ensure it is correct. If everything looks correct and you’re ready to proceed, click Overlay.
   

Monitoring the status of an overlay

On the Overlay page of DISCO, you can follow the status of the overlay using the appropriate overlay session card.

From the overlay card, you can:

• See the major details of the overlay
• Navigate directly to the documents updated by the overlay
• Download and read the validation report, if the file failed validation
• Download and read the overlay exceptions report, if there were any exceptions

Learn more about how to overlay specific types of data in these articles.

• How to update images using overlay
• How to update native files or text using overlay
• How to update or create tags using overlay