In legal AI, a fluent answer is not necessarily a reliable one.
A model can produce text that is convincing and apparently well-grounded while relying on the wrong provision, overlooking an exception or introducing a detail that is not supported by the law. For applications dealing with legislation and regulation, the quality of the answer therefore depends on more than the language model generating it.
A system first needs to identify the right sources and account for the way legislation is structured and interconnected. Once an answer has been generated, its claims should also be traceable to the legal text that supports them.
Two recent research projects at Aptus.AI address different parts of this problem. The first, JuriFindIT[1], published in the Findings of EACL 2026[2], focuses on retrieving relevant provisions from Italian legislation. A follow-up study, to appear in the main track of EMNLP 2027[3], moves further along the same path, from finding the right provisions to generating answers supported by explicit textual evidence.
Together, the two projects explore a broader direction for legal AI: building systems in which the path from a question to its legal basis can be examined, rather than focusing exclusively on the quality of the generated answer.
Finding the right law comes first
Statutory retrieval is not simply a matter of finding the provision with the closest wording to a query. The answer to a legal issue may depend on several provisions. An article may rely on definitions introduced elsewhere, contain references to other rules or belong to a broader regulatory structure that affects how it should be interpreted. Relevant information can therefore be distributed across documents that are textually different but legally connected.
JuriFindIT was developed as a benchmark for studying this problem in Italian law. It covers civil law, criminal law, anti-money laundering and counter-terrorist financing, and privacy, combining more than 23,000 statutory articles with questions written and annotated by legal experts.
One distinctive aspect of the dataset is that legislative provisions are not represented only as independent pieces of text. References between articles and the hierarchical organisation of legal documents are also encoded, making it possible to study retrieval methods that take these relationships into account. The experiments conducted with JuriFindIT show that models specifically trained for statutory retrieval can improve the identification of relevant legislation, and that information about the structure of the legal corpus can provide an additional signal.
This first stage has consequences for the entire system. The material selected by the retriever defines what the components that follow can use as evidence. A relevant provision that is missed at this point may never enter the reasoning process at all.
From relevant articles to actual evidence
Finding the correct article is still different from identifying the legal basis for a particular answer.
A provision can be long, while only a paragraph or clause may be relevant to the issue at hand. In more complex cases, the answer depends on several passages spread across different sources. Our follow-up research extends JuriFindIT to this finer level of information. Its expert-authored questions are complemented with answers written by Aptus.AI legal specialists and with annotations identifying the specific passages of legislation supporting those answers. The evidence can correspond to paragraphs, sub-paragraphs, clauses or smaller spans rather than entire articles. This moves the problem from document relevance towards evidence grounding. Instead of knowing only that a given provision is related to a question, the dataset records which parts of that provision provide the basis for the answer.
For professional use, the distinction is significant. A list of relevant documents can support research; a direct connection between a statement and the passage underlying it makes the resulting output much easier to review.
The annotations also reveal substantial differences across areas of law. Civil and criminal law questions in the dataset often rely on relatively compact evidence. Privacy and anti-money-laundering questions tend to require a broader and more fragmented set of provisions, sometimes combining definitions, obligations, exceptions and procedural requirements. Legal question answering is therefore not a uniform task. The difficulty changes considerably depending on how the relevant normative information is distributed.
Adding verification to generation
Once the relevant material has been retrieved, a language model can generate an answer from it. Access to the right sources, however, does not guarantee that every statement in the output will be supported.
Generated answers may add details that are absent from the provided legislation, overlook qualifications or combine sound conclusions with unsupported secondary claims. The second study therefore introduces a verification stage. A separate AI component reviews the generated answer against the retrieved legal material, checking its claims for support and consistency. When problems are detected, its feedback is used to revise the answer.
Expert evaluation indicates that this kind of evidence-based revision can improve answer quality. At the same time, the experiments expose a clear dependency: when the retrieval stage fails to provide the relevant provisions, the verification process loses its foundation and can even lead to worse revisions. This is an important constraint on increasingly elaborate reasoning and self-correction mechanisms. They can improve how available evidence is used, but they cannot reliably replace evidence that is missing.
Retrieval, generation and verification are therefore better understood as connected parts of the same system rather than as independent capabilities.
The difficult part: pinpointing the evidence
Our studies also examine whether models can identify the exact passages supporting an answer. This proves considerably harder than retrieving broadly relevant documents. A model may correctly identify the applicable article while still selecting passages that are too broad, incomplete or only indirectly related to the conclusion. More capable models perform better at evidence extraction, but the results still leave substantial room for improvement.
The distinction is especially relevant in legal and regulatory work. Linking an answer to a long regulation is useful; pointing to the paragraph that establishes the relevant obligation, exception or condition is much more useful to someone reviewing the result.
Automated verification shows a similar pattern. AI models can already provide meaningful assessments of the central legal content of an answer, while secondary details and qualifications remain harder to evaluate consistently.
These limitations help define the next research problems: not simply retrieving more information or generating more complete answers but improving the precision with which legal conclusions can be connected to their sources.
Conclusion
Progress in foundation models will continue to improve legal AI, but model capability is only one part of the picture. A dependable system has to retrieve the relevant law, represent relationships between provisions and keep the generated answer connected to the evidence on which it relies. The progression from JuriFindIT to evidence-grounded legal question answering follows that chain: from finding the relevant law, to generating an answer from it, to identifying the passages that justify it. This is also where applied work on regulatory AI and research naturally overlap. Retrieval, traceability, verification and evaluation are practical requirements for building useful systems, but they remain open scientific problems as well.
Fluent legal text is now relatively easy for AI systems to produce. Making the legal basis of that text auditable, precise and verifiable is a harder problem, and a more consequential one.
[1] https://huggingface.co/jurifindit/DAR-legal-it
[2] Dalla Noce, N., Colla, D., Farhang Doust, S., De Mattei, L., & Bacciu, D. (2026). JuriFindIT: an Italian legal retrieval dataset. In Findings of the Association for Computational Linguistics: EACL 2026 (pp. 4223–4241). Association for Computational Linguistics. doi: 10.18653/v1/2026.findings-eacl.221
[3] Dalla Noce, N., Bacciu D., Farhang Doust, S., De Mattei, L. (2027). JustifIT-QA: Evidence-Grounded Legal Question Answering in Italian. To appear in the Proceedins of EMNLP 2027. Association for Computational Linguistics.
