Judge flags apparent AI-made legal citations in social-media lawsuit

A dismissed case has become another warning about AI in court filings

The legal profession is still trying to determine where generative AI fits into daily practice, but one line is already clear: fabricated citations can quickly become a credibility crisis. A new example has emerged from litigation tied to an “Are We Dating the Same Guy” Facebook group, where a judge said a brief appeared to contain errors and non-existent quotations associated with AI misuse.

According to the source material, the plaintiff, Nikko D’Ambrosio, had sued 27 women, one man, and several platforms after users in a Chicago-based Facebook group described him with terms including “clingy” and “psycho.” The case against Meta was dismissed after the judge raised concerns about the quality and authenticity of supporting citations.

Judge David Hamilton wrote that the filing included no citation to the legislative findings it claimed to rely on and referenced fictitious quotations. The judge said those mistakes “bear the hallmarks of the misuse of generative artificial intelligence,” placing the dispute squarely in a fast-growing debate over AI-generated legal work.

The court’s problem was not AI itself, but unverifiable output

Generative AI tools are increasingly used for drafting, summarizing, and research support across many professions. In law, their appeal is obvious: they can quickly produce outlines, identify issues, and generate formal-looking prose. But the same systems are also known to invent case law, mangle quotations, or produce confident-looking references that collapse under basic verification.

This case highlights the central failure mode. A court does not care whether a mistake came from a rushed associate, a flawed search process, or an AI model. It cares whether the filing is accurate. When citations are fabricated, the problem becomes more than sloppiness. It goes to the integrity of the judicial process, because legal arguments depend on traceable authority.

That is why recent AI-in-law controversies have attracted such attention. The technology can save time, but it can also create a dangerous illusion of completeness. A brief may look polished and well sourced while containing authorities that do not exist or statements not found in the cited material.

A broader institutional problem

The D’Ambrosio matter also shows how AI issues are spilling into cases that already sit at the intersection of platform governance, online speech, and reputational harm. Social-media litigation often carries public interest because it tests the boundaries between user speech, moderation, and liability. When AI-generated legal defects enter that setting, they shift attention away from the underlying claims and toward basic filing reliability.

For courts, this creates an operational challenge as much as an ethical one. Judges and clerks must spend time verifying material that should have been checked before submission. For opposing parties, fabricated citations can mean needless cost responding to arguments built on nonexistent authority. And for clients, the reputational and substantive damage can be immediate if a case is weakened by preventable errors.

The episode also reinforces a point that many courts have already begun to make through standing orders and warnings: if lawyers use AI, they remain responsible for every line they file. Delegating part of the drafting process to a model does not delegate professional accountability.

Trust will depend on verification, not novelty

Generative AI is likely to remain part of legal workflows because the productivity incentives are too strong to ignore. But the profession’s adoption path is narrowing toward a simple rule: use may be tolerated or even useful, while unverified output is not.

This latest incident matters because it is not a speculative risk. The judge directly connected the filing’s fictitious quotations and citation failures to the hallmarks of AI misuse. That language will likely be cited in future arguments over sanctions, disclosure, and professional responsibility.

For the legal system, the lesson is becoming repetitive but sharper each time it appears. AI can draft. It cannot be trusted on its own to certify. Any lawyer treating generated text as final work product is not just taking a technical risk, but a courtroom one.

This article is based on reporting by 404 Media. Read the original article.