The Ultimate Guide to AI Accuracy: Part 1: Awareness
Imagine you're leading a groundbreaking justice tech initiative aimed at helping small businesses navigate complex legal processes using AI. Your team has developed an AI-powered platform to assist with filing for Chapter 7 bankruptcy, inspired by successful non-profits like Upsolve.
As you prepare to present your findings to potential funders and partners, a crucial question arises: How confident are you in the accuracy of your AI model's legal advice?
This series focuses on awareness as the first crucial step. Even though this series focuses on scalable solutions, awareness and critical thinking cannot be fully automated away.
This guide is part one of a multi-part series, which will share a few strategies to improve awareness, such as critical thinking about failure modes (this post), risk assessment and transparency, and prompt engineering best practices. Part two and three will address grounding and verification techniques respectively.
Strategy 1: Critical Thinking for AI Accuracy
Accuracy in legal technology isn't just a technical concern—it's an ethical and legal imperative. Social impact leaders often fall into the trap of passive AI adoption, treating AI outputs as infallible legal advice. This approach can lead to severe consequences, especially for vulnerable populations navigating complex legal systems.
Our strategy advocates for cultivating an "active driving" mindset towards AI accuracy in justice tech. This approach treats AI as a powerful tool that requires constant steering and critical assessment, particularly in legal contexts.
To implement this strategy effectively, consider these key steps:
Step 1: Educate yourself on common failure modes and adopt healthy skepticism
Step 2: Foster a culture of continuous learning, especially tracking mistakes and issues in your own work and those reported publicly. Example trackers include Law360 tracker on AI court orders on the use of AI and research updates by Stanford Law’s RegLab Lab.
Step 3: Postmortems. Reflect on how you would avoid the mistakes you've found in your own work and in others.
Failure Modes in AI Accuracy
(1) Hallucination: AI generates false or nonsensical information presented as fact.
Type 1: Incorrect information (e.g., fake or misplaced citations; outdated information; factual errors; fabricated content; temporal confusion)
Example: An AI cites a non-existent Supreme Court case, "Smith v. United States (2024)," to support a claim about recent changes in copyright law.
Indicator: Information provided cannot be verified through reputable legal databases or seems anachronistic given the current legal landscape.
Type 2: Incorrect interpretations (ex: of evidence or legal language; contextual misunderstanding - java as coffee vs programming language; Over-prediction of rule violations)
Example: When asked about "grounds for appeal," the AI provides information about landscaping regulations for courthouses instead of legal bases for appealing a decision.
Indicator: The AI's response seems oddly disconnected from the legal context of the query or misapplies legal concepts in an illogical manner.
Type 3: Overconfidence in incorrect information
Example: The AI states with 99% confidence that "All trademark disputes must be settled through binding arbitration," when this is not a universal legal requirement.
Indicator: The AI expresses high certainty about information that contradicts well-established legal principles or current statutes.
(2) Biased outputs
Definition: AI produces results reflecting societal biases present in training data.
Example: An AI tool for predicting recidivism rates consistently assigns higher risk scores to defendants from minority backgrounds, despite similar criminal histories to other defendants.
Indicator: Consistent patterns of unfavorable outcomes for specific demographic groups across multiple cases or queries.
(3) Sycophantic behavior
Definition: AI excessively agrees with or flatters the user, compromising objectivity.
Example: When a user suggests a questionable legal strategy, the AI responds with, "That's a brilliant approach! Your innovative thinking will revolutionize legal practice."
Indicator: The AI consistently provides overly positive feedback without offering critical analysis or pointing out potential flaws in user suggestions.
The next post in the series will round this out with failure mode #4 and #5 and an example application.
_________________________
Stay tuned to discover how you can transform your internal processes to scale faster and better, becoming a trusted strategic advisor
I'd be curious to hear if you've experienced similar operational challenges. If so feel free to share in the comments or reach out to me directly.
PS -- want to get more involved with LexLab? Fill out this form here