AI in legal services: reliability and confidentiality

Short answer: For lawyers and advisers, legal AI usually does not fall under the high-risk regime, but it is under pressure from three demands: reliability (no fabricated case law), confidentiality (professional secrecy and the GDPR) and transparency. Only when AI supports judges or judicial authorities in fact-finding or applying the law does it tip towards high-risk under Annex III.

Reliability and hallucinations

Generative AI can produce convincing but incorrect statements, citations and case law. In a legal context that is not a fringe issue but a core risk: a fabricated judgment in a court document harms both the client and the professional. The AI Act does not enforce this directly for ordinary advisory work, but professional rules and the duty of care do. Every AI output should be verified by a human against the source.

Professional secrecy and confidentiality

Professional secrecy and the GDPR quickly clash with cloud-based AI. Feeding client files into an external model is data processing and possibly a transfer to a third party. That requires a legal basis, a processor agreement and safeguards that the input is not used to train the model or made accessible to others. Without those safeguards the use can breach professional secrecy, regardless of the AI Act.

Transparency

Article 50 transparency requires that users know when they are interacting with an AI system and that generated content is recognisable as such. For legal services this means openness towards clients and the court: an AI-generated draft or a chatbot must be identifiable. Concealing it undermines both the transparency duty and trust.

When it is high-risk

Annex III lists AI systems intended to assist a judicial authority in researching and interpreting facts and the law, or in dispute resolution, as high-risk. That touches adjudication and ADR, not the ordinary advisory work of a firm. Anyone supplying or deploying such systems falls under the full high-risk obligations. The same sensitivity around profiling appears in AI in retail.

What to do

• Verify every output against the primary source; never trust citations or case law blindly.
• Protect the file: use only models with contractual safeguards against training and third-party access.
• Handle the GDPR: legal basis, processor agreement and data minimisation.
• Be transparent with client and court about AI use.
• Test for high-risk if you supply systems to courts or dispute resolution.

For lawyers the biggest pitfall is not the AI Act but their own duty of care: reliability and confidentiality weigh more heavily than the letter of the regulation.