Key Auto Review Terms
- Suggestion – This is a Yes/No field that indicates whether Auto Review suggests a tag for a document based on the tag description provided.
- Suggestion Explanation – This is the suggestion reason or rationale Auto Review provides for its tagging suggestions.
- Case Background – This is overall information your provide Auto Review about the case that is relevant for all tags, like parties, claims, and common terms.
- Tag Description – These are the instructions you give Auto Review detailing when to apply a tag and what information to consider.
- Failed Document – This is a document that was unable to be reviewed by Auto Review. Failed documents require manual review.
- Skipped Document – This is a document that could not be reviewed due to a lack of text. Skipped documents are tracked separately from failed documents, but still require manual review.
Running an Auto Review job
To run Auto Review, you will need to define your review universe, enter and refine your case background and tag descriptions, and kick off the review job.
Start and name your Auto Review job
Go to Menu > Workflow > Auto Review to open up Auto Review.
Click Start new review job on the top right to access the New review job screen.
Enter a unique name for your review job. This name can be changed after the job is run.
Define your review universe
First, create a folder containing only the documents to be reviewed. Select that folder using the folder picker. Documents in subfolders will not be included.
Enter your case background and tag descriptions
Next, enter your case background and tag descriptions. This is what gives Auto Review the context it needs to understand the matter and evaluate each document. Entering and iterating the tag descriptions is a way of training Auto Review, like drafting a protocol and providing feedback to a human reviewer. (See Auto Review best practices for writing prompts and sampling for tips on drafting case background and tag description entries.) Enter your case background, which is general context that applies to every tag in your Auto Review job.
Then select up to 10 tags to review. The tag selector only populates existing tags, so you will need to create any new tags before this step. Once you are done selecting tags, enter tag descriptions for each tag. There is a 15,000 character aggregate limit for case background and tag descriptions per review job to ensure high performance.
If you have already run Auto Review using the same browser you are using now, the case background, tags, and the tag descriptions will be auto-filled from the previous run. You can update any pre-filled tags or descriptions as needed.
Information considered by Auto Review
When generating tag suggestions, Auto Review considers (1) the text of the document being Auto Reviewed; (2) certain metadata fields; and (3) the case background and tag descriptions you input when starting an Auto Review job.
You can see the text of the document by clicking to the "Text" view at the top of the document viewer.
The following metadata fields are sent to Auto Review. Just like a human reviewer, Auto Review may consider these fields if they contain particularly relevant information related to what you are looking for.
For any document:
- Filename
- Family date
- Last modified date
- File type
- Custodian
- File extension
- File path
If an email:
- Email subject
- Sender, recipient, cc, bcc
- Sent date
If a chat:
- Sender, recipients
- Channel name
- Sent date
- Message type
Note: As a general best practice, we recommend searching or filtering on date fields in order to implement date restrictions, such as in ESI agreements, rather than relying only on Auto Review. This is because Auto Review is optimized for understanding language, not computing math or timelines, and may occasionally make mistakes when comparing dates.
Run your review
Finally, click the Review documents button at the top right of the Auto Review New review job screen.
To prevent you from incurring costs from misclicks, a pop-up will appear asking you to confirm that you wish to run the Auto Review job.
When the Pending count reaches 0, and the status icon changes from In Progress to Complete, all documents have been reviewed. If you wish to cancel an in-progress job, you can do so by clicking Stop this review. You will be billed only for the documents Auto Review reviewed (not for those it failed or skipped). You can see the case background and tag descriptions used for this particular Auto Review job by clicking on Instructions.
See this article for information on viewing your results: "Viewing and understanding Auto Review results and metrics."
Unable to run Auto Review job
If you are unable to run Auto Review, you will see an error indicator at the very top of the New review job pane or inline where an error has occurred.
Example top-line error:
Example inline error:
Errors include the following:
- The documents to be reviewed are in ECA;
- No folder has been selected;
- No tags have been selected;
- No tag descriptions have been entered; or
- You do not have database permissions to run Auto Review.