By organizing documents into review stages, you can control user access to each stage, customize the review decisions, and monitor the progress of each stage.
- Ingesting, creating tags, and organizing documents
- Creating stages
- Selecting a document source
- Organizing documents
- Ranking documents
- Creating a decision pane
- Adding and assigning reviewers
- Checking out batches
- Tracking progress
- Quality control
- Using documents
Ingesting, creating tags, and organizing documents
Obtain the opposition’s production and send the data to DISCO for processing and ingest. During ingest, DISCO strategically deduplicates your documents, extracts OCR data, and identifies family and conversation relationships among all applicable documents.
Next, create tags for various issues that you expect to find in the documents.
When your documents are ready for review, create your review stages.
To create a review stage:
- On the DISCO home page, in the main menu, click Manage Review Stages.
- On the Review Stages page, click Create New.
- In the New Stage box, enter a name for the new review stage and then click Create.
Common review stages include:
- Issue Review – All or some documents that need to be reviewed for issues. If you receive documents as a rolling production, where data comes to you in stages rather than in one piece, set up your stage groupings by ingest session and group by Bates number.
- Escalation Review – Funnel documents from the issue review into an escalated review stage for attorneys with more familiarity with the facts and issues of the case.
Selecting a document source
The default source for documents is all documents. Should you choose to focus your stage on a subset of all documents, you can populate the stage with search results as sources. You may have up to 20 discrete sources per stage each containing a distinct search string. For example, you can use a search string that limits the documents in a stage by a date range, to a particular set or custodians, or to a set of search terms. To make sure that all family members are included in a batch, when setting up the stage, toggle Keep families together to YES. If you would like to include conversations and group by them in your batch, toggle Keep conversations together to YES.
To select a particular set of documents to review, click Include only documents from specific sources. Navigate to the location of the new source or locate the source by searching:
- An ad hoc search (with the help of traditional Search Builder and Search Examples)
- A search from your Search History
- Any Saved Search
Here you can also rename a source.
There are three organization options for documents:
- Group by – You may group by Custodian, Document type, Ingest Session, Stage Source, or No groups. You can restrict access level permission control for groups.
- Order by – If you are using groups, order adjusts the sort order within each group.
If you choose to NOT keep families or conversations together in your batch, you may select one of the Order by options: Tag Prediction, BatesNumber, Date, Family Date, or Path.
If you chose to keep conversations together, the default sort order is Conversation Date.
Note: When keeping conversations together, documents not part of a conversation will be ordered by their family date, if in a family, or by their document date if not.
- Batches contain – Select the maximum batch size. Batches represent the maximum number of documents that may be checked out in a set of documents. Users can check out multiple batches, but if the number of documents remaining in a group is less than the maximum batch size, the batch will only contain the available number of documents. Five hundred documents is a common batch size. Batches are checked out by reviewers but may be re-assigned to a different reviewer by an administrator.
Ranking places the visible instance of deduplicated documents in the highest ranked group. Ranking is important for getting the most context out of your documents you are reviewing. For example, if you group your documents by Custodian (as displayed in the image above), you will want the most important Custodian listed at the top. This way, if you have documents across multiple custodians, your documents will get grouped into review stages so that you get the most context around those documents.
You can rank your groups by dragging and dropping the rows.
When you have finished, click Save and next.
Creating a decision pane
Review decisions are the decisions that you want reviewers to consider when reviewing the documents within a stage. The reviewer uses the decision pane to notate the decision information for each document they review.
Construct your decision pane by clicking the buttons under Review Components. Once you click on a component, it will appear in the decision pane editor on the right. You can include a label or instructions in the text box. Select the tags you’d like the reviewer to choose from and those tags will be populated in the editor as radio buttons. As you build a decision pane, the decision pane editor displays exactly what the reviewer will see as they review the stage’s documents.
When you are done building your decision pane, click Done.
Adding and assigning reviewers
Once you've set up your stages and created the decision pane, you can create users and assign them to each stage.
To add and assign reviewers:
- On the DISCO home page, in the main menu, click Review Team.
- On the Review Team page, click in the Assign or create a user... text box and select a user from the list or click New user to create a new user. Then, choose a role for the user in the Select a role dropdown (custom roles are available as well):
- Admin: The admin role may complete all actions in DISCO.
- Reviewer: The reviewer may complete searches, complete batch actions such as mass tag and exhibit sets, and produce documents from DISCO as well as review documents inside and outside of Review Stages. Reviewers, however, do not have access to setup stages or to manage stages.
- Restricted Reviewer: The restricted reviewer does not have access to the usual DISCO features of search, batch actions, and productions. They only have access to the stages for which they have been assigned. In the document viewer, restricted reviewers will have access to view related documents (such as families, conversations, and similar documents). However, if a related document is not in one of their assigned batches, the restricted reviewer will not be able to perform any action on the document; they will only be able to view these related documents for purposes of context in their review.
When you add a restricted reviewer to a stage, they will receive a welcome email instructing them how to log in to the database and start reviewing, and they can then check out batches and review documents.
Checking out batches
A batch is a collection of the next available documents or documents from a group (The default is Next available).
To check out a batch:
- On the DISCO home page, in the main menu, click Check Out Batches, and then click the stage name.
- On the My Batches page, next to Grab next batch from, either leave the default Next available or select a custodian from the dropdown, and then click Check out.
- On the batch review screen, click a document to open it. The decision pane you created earlier will be on the left side of the document viewer.
- When you have finished reviewing the document, click Mark Reviewed & Next. The next document in the batch will open.
When a document has been reviewed, a green check mark will display in the REVIEWED column on the batch review screen. Also, a progress bar shows how many documents have successfully been reviewed.
- When the batch has been reviewed, click Check-in.
*IMPORTANT NOTE: When you check out a batch, DISCO uses "just-in-time batching," which means that all of the documents in a stage are not put into batches until the reviewer requests them. This means that any change in strategy in your review does not revolve around lots of deconstruction of batches that have not yet been reviewed.
*IMPORTANT NOTE: For databases created BEFORE 02/07/17, a document enters stage complete after being marked Reviewed, whether the document was checked back in or not. For databases created ON or AFTER 02/07/17, a document will enter stage complete ONLY IF the document has been marked Reviewed and is currently in a batch that is checked in.
DISCO’s review stage metrics further assist you in the management process by providing you with a deeper understanding about the pace of review, tag rates, and overall findings. Such insights can not only help with staffing and budgetary considerations but also identify and correct anomalies early in the review process.
To view progress information, on the Review Stages page, click Metrics.
Review metrics are unique to each review stage. To see the metrics for a particular review stage, click the review stage card on the left and then click Metrics.
The metrics are broken down into three areas:
- Pace: Displays the team’s overall review pace along with the median pace of active reviewers. DISCO will estimate how long the review will take based on current review pace.
- People: DISCO provides charts that show reviewer pace by day and tagging rate by reviewer. These charts provide insight into the overall review, and highlight outliers among your review team.
- Findings: View the percentage distribution of tags applied within the stage. Double click on any specific tag to review the associated documents.
All reviewer progress, regardless of their role, is tracked when using Review Stages. However, regular reviewers and admins may access documents in DISCO through the search. This activity outside of Review Stages is not tracked on the metrics page.
For more information, see Metrics for review stages.
To analyze the accuracy of your reviewers, you can quality control the different stages:
- Create either the tag "QC approved” or the “QC defect”. See Working with tags for more information.
- Run a search using the syntax sample() to review a subset of the documents that have passed through the first stage.
For example, running sample(20%,"stagecomplete(FPR)") will return a randomized sample of 20% of the documents that have completed First Pass Review.
- To drill in deeper and see the documents tagged by a specific reviewer, run the search sample(10, tag(by email@example.com)).
- (Optional) Mass tag your search results with a tag such as “QC Sample 1” to save and access your quality control batch at a later time.
- Review the sample results and tag them as necessary to identify any problem documents.
- Determine the extensiveness of quality issues and plan for resolution.
- Review and tag the documents with the correct tags. If there is an extensive problem, such as a general misunderstanding of tagging instructions, the review manager should craft a new stage with a document source search that captures the affected documents to be reviewed again or reviewed by higher level reviewer.
Once the review is complete, members of the case team can run a search on any desired issue tag, review the documents in DISCO, and use them outside of DISCO by adding some or all of the appropriately tagged documents to Exhibit Sets. Alternatively, you can batch print documents directly from DISCO.