Have the IRS’s chatbot and live chat applications provided effective and efficient service to help taxpayers meet their tax obligations? It’s a question that’s difficult to answer because the data the IRS currently collects to measure chatbot performance is unreliable, limiting the agency’s ability to determine whether the applications are operating effectively, the Treasury Inspector General for Tax Administration says.
Specifically, the IRS hasn’t analyzed its chat applications and doesn’t have any performance measures, TIGTA says in a report released last month.
The Automated Collection System function within the Small Business/Self-Employed Division was the first IRS function to pilot the use of chat applications to interact with taxpayers. The ACS function primarily resolves taxpayer inquiries related to balances due or delinquent returns. The IRS’s goal in using chat applications is to direct taxpayers to available online or self-help tools in lieu of calling the IRS’s toll-free telephone line.
These applications allow a live assistor to exchange text messages, files, web pages, and other information, in real-time when addressing taxpayer inquiries. In addition to chat applications, the ACS function also deployed voice bots that allow taxpayers to verbally navigate through an interactive voice response system via telephone. Both live chats and voice bots aim to improve customer service.
The ACS chat applications provide statistical reports that track chat metrics, such as the number of live chats, average handle and wait times, number of concurrent chats, and total chatbot sessions. Although ACS management explained that they use these reports to review changes in the metrics, TIGTA says they don’t have any performance measures for the application or ensure the accuracy of the report data.
A review of statistical reports found outliers and discrepancies that made the reliability of these reports questionable, TIGTA says. For example, the reports showed a live assistor handling as many as 603 live chats at the same time, which the IRS agreed was incorrect because the system limits the number of concurrent chats to three.
TIGTA also reviewed 40 live assistors and assessed the transcripts of their live chats. The IRS watchdog identified 24 live assistors (60%) who worked multiple live chats concurrently.
“There is an inconsistency between what management believes is permitted and what live assistors believe they are allowed to work,” TIGTA says. “Conducting multiple chats concurrently can be problematic for several reasons, including decreased customer service and increased risk of inappropriately disclosing taxpayer information, especially when the live chats are authenticated.”
In addition, ACS management only conducted nonevaluative performance reviews of live assistors because, until June 2025, they considered live chat to be a pilot program. Nonevaluative reviews can’t be used as part of the employee’s performance appraisal. Although there is no agencywide policy that prohibits evaluative performance reviews during a pilot program, ACS management stated that they plan to implement evaluative reviews in April 2026, the TIGTA report says.
Finally, TIGTA identified 29 responses and 44 keywords or questions where the ACS automated chatbot didn’t provide sufficient information or wasn’t able to recognize the keywords or questions typed into the search box.
“We notified ACS management and they subsequently began taking corrective actions to address the deficiencies,” TIGTA says.
TIGTA made nine recommendations to the IRS, including developing a process to validate the accuracy and completeness of the ACS chat applications’ data; establishing standards to assess the effectiveness and efficiency of the ACS chat applications; and implementing guidance for conducting evaluative reviews for live assistors.
IRS management agreed with all nine recommendations and plans to implement corrective actions for five of the recommendations, such as validating the accuracy and completeness of ACS chat applications’ data. The IRS also stated that it has already implemented corrective actions for the remaining four recommendations, such as implementing guidance for managing the ACS live chat application and evaluative reviews for employees working ACS live chats.
“The Automated Collection System chat program provides a way for taxpayers to resolve tax issues online, offering an alternative to calling the IRS, reducing wait times, and supporting more accessible, reliable, and efficient service. This program is part of our broader focus on transforming the taxpayer experience—meeting taxpayers where they are with more accessible, digital-first service options while improving efficiency,” Lia Colbert, commissioner of the IRS Small Business/Self-Employed Division, wrote in response to the report.
“Since its launch, the chat program has grown significantly and now serves millions of taxpayers annually. Chatbots offer automated, around-the-clock access to information and services, while allowing taxpayers to transition to a live assessor when more complex support is required. These channels support key service activities, including installment agreement assistance and other account-related actions.
“We acknowledge TIGTA’s findings on performance measurement, data reliability, and program oversight,” she continued. “The chat program expanded rapidly during a period of workforce changes and increased demand for digital services. While this created challenges, the chat program remains a valuable and widely used service. We agree that stronger performance measures and more reliable data are needed to fully assess and improve performance.”
Photo credit: Gromit702/iStock
Sign in to get access to this free resource, and all of our whitepapers and reports.
Download this content today!
Register Now Already registered? Click here to Log In
Tags: AI chatbot, Artificial Intelligence, chatbot, IRS, Taxes, Technology, TIGTA
