Deloitte to partially refund Australian government for report with apparent AI-generated errors

Home > World > World

print dictionary print

Deloitte to partially refund Australian government for report with apparent AI-generated errors

The logo of Deloitte is displayed at their office in Melbourne, Australia, on Oct. 7. [AP/YONHAP]

The logo of Deloitte is displayed at their office in Melbourne, Australia, on Oct. 7. [AP/YONHAP]

 
Deloitte Australia will partially refund the 440,000 Australian dollars ($290,000) paid by the Australian government for a report that was littered with apparent AI-generated errors, including a fabricated quote from a federal court judgment and references to nonexistent academic research papers.
 
The financial services firm’s report to the Department of Employment and Workplace Relations was initially published on the department's website in July. A revised version was published on Friday after Chris Rudge, a Sydney University researcher of health and welfare law, said he alerted the media that the report was “full of fabricated references.”
 

Related Article

Deloitte had reviewed the 237-page report and “confirmed some footnotes and references were incorrect,” the department said in a statement Tuesday.
 
“Deloitte had agreed to repay the final installment under its contract,” the department said. The amount will be made public after the refund is reimbursed.
 
Asked to comment on the report’s inaccuracies, Deloitte told The Associated Press in a statement that the “matter has been resolved directly with the client.”
 
Deloitte did not respond when asked if the errors were generated by AI.
 
A tendency for generative AI systems to fabricate information is known as hallucination.
 
The report reviewed the use of automated penalties in departmental IT systems within Australia's welfare system. The department said the “substance” of the report had been maintained and there were no changes to its recommendations.
 
The revised version included a disclosure that a generative AI language system, Azure OpenAI, was used in writing the report.
 
Quotes attributed to a federal court judge were removed, as well as references to nonexistent reports attributed to law and software engineering experts.
 
Rudge said he found up to 20 errors in the first version of the report.
 
The first error that jumped out at him wrongly stated that Lisa Burton Crawford, a Sydney University professor of public and constitutional law, had written a nonexistent book with a title suggesting it was outside her field of expertise.
 
“I instantaneously knew it was either hallucinated by AI or the world’s best-kept secret because I’d never heard of the book and it sounded preposterous,” Rudge said.
 
Work by his academic colleagues had been used as “tokens of legitimacy,” cited by the report’s authors but not read, Rudge said, adding that he considered misquoting a judge was a more serious error in a report that was effectively an audit of the department’s legal compliance.
 
“They’ve totally misquoted a court case, then made up a quotation from a judge, and I thought, well, hang on: that’s actually a bit bigger than academics’ egos. That’s about misstating the law to the Australian government in a report that they rely on. So I thought it was important to stand up for diligence,” Rudge said.
 
Sen. Barbara Pocock, the Australian Greens party’s spokesperson on the public sector, said Deloitte should refund the entire 440,000 Australian dollars.
 
Deloitte “misused AI and used it very inappropriately: misquoted a judge, used references that are nonexistent," Pocock told Australian Broadcasting Corp. "I mean, the kinds of things that a first-year university student would be in deep trouble for.”

AP
Log in to Twitter or Facebook account to connect
with the Korea JoongAng Daily
help-image Social comment?
s
lock icon

To write comments, please log in to one of the accounts.

Standards Board Policy (0/250자)