Paxton AI proudly presents the Paxton AI Citator, now patent pending. Citators are essential tools for legal professionals to check the standing and precedential value of case law. These tools track whether a case has been overruled, affirmed, questioned, or cited by later cases. Traditionally, citators like LexisNexis’s Shepard's, Thomson Reuters’s KeyCite, and Bloomberg’s BCite have been managed by human editors and reliant on extensive legal databases. Human-based citators have been essential for legal research, yet they can be expensive, time-consuming, and prone to error.
Why Rethink Traditional Citators?
Despite their ubiquity in legal practice, traditional citators such as Shepard's, KeyCite, and BCite are expensive and error-prone. They depend on vast editorial teams to manually update a case's legal status. The UCLA Law Library advises caution, noting the inherent fallibility of these systems:
Each citator service is quite good, but is important to keep in mind that these citators are fallible. Each is only as good as its algorithm and its editors. UCLA Law Library Guide
A study by the College of Willaim & Mary Law School found significant discrepancies in how different citators handle negative citing relationships. Shepard’s and KeyCite missed or mislabeled about one-third of negative citations, while BCite’s error rate exceeded two-thirds.
The study stated:
"I looked at 357 citing relationships that have at least one negative label from a citator. Out of these, all three citators (KeyCite, BCite and Shepard’s) agree that there was negative treatment only 53 times. This means that in 85% of these citing relationships, the three citators do not agree on whether there was negative treatment. Even when they all agree there was negative treatment, their descriptions often conflict."
Technological Advancements with the Paxton AI Citator
Leveraging artificial intelligence, the Paxton AI Citator addresses these limitations. Evaluated against the Stanford Casehold dataset of 2,400 examples testing whether a case was overturned or upheld, our citator achieved a remarkable 94% accuracy rate (Stanford Casehold Benchmark).
The tables below present performance metrics for the Paxton AI Citator across different datasets and case types. To understand these metrics, let's break down what precision, recall, and F1-score mean in this context:
- Precision measures the accuracy of positive predictions. For example, a precision of 0.90 for overruling cases means that when the AI predicts a case was overruled, it's correct 90% of the time.
- Recall measures the ability to find all positive instances. A recall of 0.99 for overruling cases means the AI correctly identified 99% of all cases that were in fact overruled.
- F1-Score is the harmonic mean of precision and recall, providing a single score that balances both metrics. An F1-score closer to 1 indicates better overall performance.
Paxton AI Citator Performance on the Stanford Casehold Benchmark
Our internal benchmarks further emphasize its robustness. Paxton consistently determined the standing of cases across our extensive database of attorney-reviewed cases. The Paxton AI Citator demonstrated substantial improvements over existing tools in handling complex legal narratives. We are releasing a subset of this data - achieving an accuracy rate of 86%.
Paxton AI Citator Performance on Paxton AI Internal Benchmarks
The Paxton AI Citator is different
We designed the Paxton AI Citator to redefine legal research. This powerful tool identifies and groups cases based on similar legal principles and does so even when cases do not directly cite each other. This innovative approach ensures comprehensive coverage and relevancy, dramatically increasing the efficiency of legal research by providing legal professionals with swift access to pertinent case sets.
The Paxton AI Citator will also improve consistency over human-based citators. Our Citator standardizes case analysis based on sophisticated AI reasoning, ensuring each case is evaluated under consistent criteria. This method significantly reduces the variability and subjective interpretations common in human review, offering a more reliable and predictable research tool.
Moreover, our Citator goes beyond traditional citation indicators by offering detailed explanations for each citation. This feature enhances the understanding of a case's relevance and legal standing, making the results easier to validate and increasing transparency within legal research processes.
Industry Benefits
Paxton AI's Citator provides a standardized review analysis that boosts efficiency and accuracy. Our tool automates the complex tasks of analyzing and synthesizing cases and their relationships. This automation saves time and cuts costs, making comprehensive legal research tools accessible to a broader audience.
The Paxton AI Citator significantly lowers costs by offering the capabilities of a traditional law library without the need for expensive annual subscriptions. It also enhances accessibility for solo practitioners and small law firms, who can now conduct detailed legal research anytime, anywhere.
The integration of AI ensures consistent, reliable results free from human bias and variability. Our tool streamlines the validation process and promotes transparency in legal research by offering comprehensive explanations for each case status determination, not just status indicators. We hope this tool will bring new efficiency and accuracy to legal research, benefitting the entire legal community.
Walkthrough: Using the Paxton AI Citator
Users can access the new Paxton AI Citator by starting in the Case Law module in Paxton. (Screenshot 1). Users can ask a question, for example, “explain the evolution of abortion rights from the 1950s to the present.” (Screenshot 2).
Once they submit a query, Paxton will synthesize the relevant case law and respond. (Screenshot 3). If users are interested in the status of a particular case, they can navigate to the “References” dropdown. From there, users can click on “Check case status” to access the citator tool.
After a user selects “Check case status,” Paxton will begin to analyze the relevant case. Paxton reviews the case itself, every case that cites to the case in question, and cases that are semantically similar to the case in question. (Screenshot 5). Once the Paxton AI Citator’s analysis is complete, users can navigate to the Paxton Citator tab to see an in depth analysis of all of the Important Cases, where Paxton highlights significant case relationships. Paxton explains why it highlights each case as an important case. For example, if a user runs the citator on Roe v. Wade, Paxton will return Dobbs v. Women’s Health Organization as an important case. Paxton recognizes that Dobbs overturned Roe, stating that the relationship “can only be accurately categorized as ‘overturned.’”
.png)
Releasing sample cases & legal benchmark
In the spirit of transparency, we are releasing a sample of cases that show how the Paxton AI Citator performs along with an explanation of our assessment and a human validated review of each case in the benchmark test set.
For each case in the test set, Paxton reviewed all citing cases to determine the status of the test case. Paxton then provided a conclusion on whether the case was likely good or whether researchers should proceed with caution.
On this set, the Paxton AI Citator correctly classified the relationship between citing and cited cases on 86% of all test cases, and accurately identified 83% of all cautionary cases.
While Paxton was not perfect, neither are human-based citators. Human-based citators have been around for decades, but Paxton AI’s Citator, powered by advanced AI technology, will improve in leaps and bounds in the coming years. We expect the Paxton AI Citator to provide significant value to our users and anyone seeking to conduct thorough legal research. We are excited about the potential for continuous improvement that an AI-based citator can deliver.
Exclusive Trial Offer
We invite all legal professionals to trial the Paxton AI Citator with complimentary 7-day access (subscription starts at $99 per month per user). Visit Paxton AI to start your trial and witness firsthand the precision and efficiency of AI-driven legal research.
Conclusion
The Paxton AI Citator stands out as a pioneering advancement in legal technology. By integrating cutting-edge and testing with rigorous legal benchmarks, it offers a more consistent, reliable, and accessible means of legal citation analysis. With the Paxton AI Citator, legal professionals can expect a more efficient, accurate, and transparent research experience, enabling them to make well-informed decisions faster.
Embrace innovation and enhance your legal research capabilities with the patent-pending Paxton AI Citator.
