While artificial intelligence has altered the way nearly every industry operates, humans have a vital role in overcoming AI’s limitations in eDiscovery document review.
Artificial intelligence (AI) and machine learning technologies (considered a subset of AI) quickly have become a large part of our everyday lives, often in ways we don’t even realize. For example, the social media advertisement that perfectly applies to you is powered by AI, and there are even machines that can predict and trade in the stock market. These technologies are increasingly essential in many scenarios, but the importance of human experience and expertise must not be overlooked. Often, the most efficient and effective results are achieved through a hybrid of machine learning and human intelligence.
This is certainly the case during the legal document review stage of electronic discovery, or eDiscovery. Review is a crucial step of the legal process that involves mining ever-increasing mountains of data for evidence relevant to a lawsuit, investigation, audit or regulatory review. The data first must be identified and collected, then culled and reviewed before being presented as evidence in court or produced to a regulatory agency. In more and more such reviews today, machine learning technologies are implemented to significantly increase both the speed and accuracy of that process.
Take the example of a review of 100,000 documents. Humans can manually review about 500 or so documents on a good day. So, a case involving 100,000 documents would take around 1,600 hours, or about two weeks for 20 people. By comparison, if that same team reviewed a random sampling of only about 10,000 of those documents, machine learning can take the knowledge gained from that and apply it to the remaining 90,000 documents in just a few minutes. While that process also involves various tweaks and checks to ensure accuracy, even so, both the time required and associated costs are much less. Study after study has shown that, when done correctly, the results are as good or better than manual review.
But there’s a catch (of course, isn’t there always?). Machine learning is only as smart as the people operating it. It’s possible that AI will operate without any human guidance in the future with respect to some of these tasks, but in the meantime, AI’s success, at least in legal document review, is entirely reliant upon the validity of what it’s fed.
Artificial intelligence in eDiscovery
So, how does AI work in this process, exactly? Artificial intelligence is the overall term used to describe computers performing tasks that would typically require human intelligence. The eDiscovery process, though, mostly involves machine learning, which is a subset of AI in which the system makes predictions based on human training. Essentially, with careful teaching from a person on how to think, the computer can make decisions about large sets of data in a much shorter timeframe and with higher accuracy than if humans were acting alone.
The type of machine learning used in eDiscovery, most often called technology assisted review (TAR), starts with humans reviewing thousands of documents and identifying them as either relevant or non-relevant. The software learns from this group of human-reviewed documents and uses the information to code the rest of the documents, which can be thousands or even millions of documents. While there are usually many other factors that go into the document review process, and there are several methods that can be used, the basic elements of TAR remain. The software relies on the quality of that initial human input and could miss important data if the user makes errors or doesn’t review enough documents or a sufficient variety of document types, for instance.
Another advancement called emotionally intelligent AI searches for indications of emotion, such as positive or negative sentiment, intent, rationalization, opportunity and pressure. More truly AI in nature, this technology picks up on these emotional aspects through language, punctuation, phrasing and other clues. It also searches for abnormal behaviors, such as frequent communication outside of business hours or regular contact between two people (especially if they don’t often work together or are discussing sensitive topics) — which can be helpful in identifying fraud or theft. While this branch is relatively new to legal document reviews, it is quickly proving to be an effective tool, especially in investigatory matters.
Involving the experts
Because of the impact that humans’ decisions have on TAR’s accuracy, it’s essential that the human reviewers are experts on the document review processes, well-versed in the subject matter and details of the case, and familiar with the intricacies of the TAR process itself. Even though most document review attorneys needn’t learn in full depth the technical aspects of TAR technologies (a different set of experts usually runs that process), some basic TAR skills often separate good reviewers from great ones. A deep understanding of every aspect of document review – from the review process and special considerations one must take in that process to enable TAR usage to the technologies themselves — leads not only to more accurate results, but also reduced costs and a much faster timeline.
Additionally, while AI has come a long way in the last couple of decades, there are still shortcomings that need human attention. TAR software cannot easily review graphics (such as tables and charts), complex Excel spreadsheets or other specialized file types. While advanced recognition technologies that identify features such as skin tone do exist, they don’t easily integrate into document review platforms, and false positives are still common.
Whether to bring knowledge of certain legal topics or to review various types of complex data that AI can’t yet understand, human experts are still needed for successful document review.
Benefits of a dedicated team
In traditional manual reviews, law firms and eDiscovery companies have relied heavily on temporary staffing of review teams. But as we’ve discussed, TAR reviews tend to be most accurate when built by highly skilled and well-trained reviewers. In my experience, especially when it comes to implementing machine learning into this process, document review is most efficient and effective when handled by a small, consistent team of attorneys with extensive knowledge and experience. Working with a dedicated group increases the project’s quality while decreasing time spent, costs incurred, mistakes caused by inexperience, and risk of security or confidentiality issues.
Businesses that commonly face litigation, such as insurance companies, especially benefit from a constant, dedicated team that understands the company’s daily business and even inner workings. As that core group reviews documents across multiple legal events for the same company, the reviewers become familiar with recurring jargon, nicknames, abbreviations, products and services, or other language specific to the business or industry. They can then leverage all of that knowledge to make the machine learning processes better and faster with each new matter. They’ll also develop relationships with key players, such as company executives, general counsel, outside counsel and paralegals, which allows for quick resolution of any issues that may arise.
AI still has limitations
While AI has come a long way, it still doesn’t capture all the characteristics that humans offer. For example, it’s currently impossible for technology to have background knowledge or experience in legal fields such as intellectual property, contract disputes or class actions, and it certainly doesn’t have the problem-solving capabilities of a human.
Because of these limitations, courts will likely continue requiring human involvement throughout document review for the foreseeable future — especially considering the massive consequences legal proceedings have on individuals, companies, governments and nations. This requirement typically works in each party’s favor, as it’s much easier to justify the use of TAR when it’s backed up by a highly experienced and certified team of professionals. For now, humans will carry on as AI’s teacher and its quality control instrument during legal document review.
It’s important to remember that artificial intelligence is a tool to streamline the eDiscovery process. In the same way that building a table is much more difficult without a hammer, AI without a human as its guiding tool is much less successful in accomplishing a specific goal. Without the right combination of technology and people, eDiscovery would take longer, be more labor-intensive and produce poorer quality overall results. With this in mind, a hybrid approach to document review is the most effective way to produce relevant data for any legal event.
About Brian Schrader
Brian Schrader, Esq., is President & CEO of BIA (www.biaprotect.com), a leader in reliable, innovative and cost-effective eDiscovery services. With early career experience in information management, computer technology and the law, Brian co-founded BIA in 2002 and has since developed the firm’s reputation as an industry pioneer and a trusted partner for corporations and law firms around the world. He can be reached at bschrader@biaprotect.com.
Leave a Reply