Misinformation expert cites bogus studies — likely due to AI — in court case: court docs

An expert in misinformation has been accused of using artificial intelligence to craft an expert declaration in a court case — and cited a study that doesn't exist.

Communication professor Jeff Hancock is the founding director of Stanford’s Social Media Lab stands accused of using AI to craft an expert declaration in a Minnesota court case.

The effort was about a 2023 state law that criminalizes using deepfakes to influence an election, The Stanford Daily noted Monday. Hancock handed in a 12-page declaration defending the law with 15 legal citations. Two of those couldn't be found, however.

The reason: ChatGPT appeared to make them up.

"No article by the title exists," court documents allege. "The publication exists, but the cited pages belong to unrelated articles. Likely, the study was a 'hallucination' generated by an AI large language model like ChatGPT. A part-fabricated declaration is unreliable."

The lawsuit is between Republican Minnesota state Rep. Mary Franson and a conservative social media satirist named Christopher Kohls against the state. The latter claimed that the state law limited free speech when the use of AI media could be used to explain false information.

ALSO READ: FBI uncovers deceptive AI deepfakes in 2024 election's final hours

Hancock was given $600 an hour for writing up his comments and was required to swear under penalty of perjury that what he said in the document was "true and correct."

A Nov. 16 filing cited errors Hancock made and requested the judge exclude it from the state's case.

“The citation bears the hallmarks of being an artificial intelligence (AI) ‘hallucination,’ suggesting that at least the citation was generated by a large language model like ChatGPT,” lawyer Frank Berdnarz wrote. “The existence of a fictional citation Hancock (or his assistants) didn’t even bother to click calls into question the quality and veracity of the entire declaration.”

Read the full report here.