Completing the First Batch of Harvey’s Legal Engineering Course: What I Learnt So Far
The use of AI in our daily tasks has manifestly shifted from ordinary text and visual generation, recommendation, naturally responsive conversation and support to more crucial professional tasks, such as data reviews and analysis, automation, etc., and having an insider perspective on how an AI model processes information, arrives at its outputs, and the probable constraints that can affect its accuracy is a huge leap, especially for legal practitioners. No more backdoor projections as to its capabilities. And more importantly, the myths were demystified. “AI will not replace lawyers." At least not in the way it's popularly envisaged.
Taking the Harvey pre-course for legal engineering “Level 1 Foundation” has offered a more practical insight into the daily interactions of lawyers with AI models in managing legal workloads such as contract reviews and redlining, document summary, research and policy drafting, generation of adversarial questions, legalese translations, and the ethical roles and responsibilities of lawyers as a centralized agent in ensuring the literal control of AI risks and outputs.
The pre-course, split into two batches, “AI for Legal Basics” and “AI Analysis for Legal Workflows” proffers authoritative guides and solutions on professional AI use. The former batch borders structurally on the foundational knowledge of AI: what it means, the misconceptions and legal implications, and why it's relevant for legal professionals. It also discusses ethical uses and methods to protect clients' confidentiality and ensure data disclosure limits, as well as common risks and how to effectively mitigate them. Lastly, it reveals prompt structuring as the methodical compass for ensuring AI accuracy, and the evaluation of its outputs in case of errors.
The latter further showcases the use of AI in analysing and extracting large-scale data and documents. Contextually, it explains how an AI model receives and processes information within a context window, the batching of large documents, and the need for accurate column design while using Harvey's review tables. With practical examples, it reveals structured prompts for executing document analysis and extraction to ensure consistent output across document sets and how to verify them against the source documents.
More importantly, it emphasized the treatment of AI models as mere assistants, with the lawyers still retaining the supervisory role over each prompt and output. Each response, regardless of its apparent completeness, requires verification. As such, in each module, strategic steps for verification were provided as a guide to ensure accuracy.
My biggest takeaways however, were the underlying causes of AI output errors, hallucinations, and the introduction to the term "false negatives." The latter simply occurs when AI incorrectly indicates data, clause, etc. as absent or missing wherein it's present in the source document. Most of these aforementioned errors occur when the prompt is improperly defined in terms of contexts, boundaries, and missing fields.
Also, the context window’s inability to process large amounts of information in a document set, or the input of hundreds of documents into the AI model with vague prompts and no proper batching, results in generic and speculative answers. However, if all is procedurally done and the necessary checklist is ticked before running the prompts across all the documents, Harvey could analyse hundreds of documents accurately, leaving room for minimal errors.
These were all mind-boggling but were ridiculously simplified for learning. Having prior knowledge about how AI works also makes it enjoyable.
Conclusively, AI is reputedly good at recognizing existing patterns and modeling after those patterns to generate new content. Hence, it'll be immensely productive for lawyers in handling analytical works, research, drafting, and summaries. However, in all cases, it lacks judgement. As such, its outputs should at best be treated by lawyers as drafts requiring further reviews and verification regardless of its apparent completeness. Also, due to this defect, it's not outrightly a good fit for analysing or reviewing novel laws or high-stakes decisions about rights or financial outcomes.
.jpeg)
