Popular search terms:

MSP #5: False Positive Rates

The IRS’s Fraud Detection Systems Are Marred by High False Positive Rates, Long Processing Times, and Unwieldy Processes Which Continue to Plague the IRS and Harm Legitimate Taxpayers

TAS Recommendations and IRS Responses

1
1.

TAS RECOMMENDATION #5-1

Calculate an “Operational FPR” in addition to the FPR and OPR for non-IDT accounts.

IRS RESPONSE TO RECOMMENDATION: The IRS agrees that further exploration and refinement of our methodology for calculating the False Detection Rate (FDR), which TAS refers to as the False Positive Rate (FPR), and related activity would be beneficial in reflecting the customer experience. To that end, the IRS has started tracking new measures such as the operational FDR (termed the Refile Rate).

CORRECTIVE ACTION: The IRS agrees that further exploration and refinement of our methodology for calculating the False Detection Rate (FDR), which TAS refers to as the False Positive Rate (FPR), and related activity would be beneficial in reflecting the customer experience. To that end, the IRS has started tracking new measures such as the operational FDR (termed the Refile Rate).

TAS RESPONSE: The National Taxpayer Advocate is pleased that the IRS will begin tracking the Operational FPR. This data point is key to understanding how the IRS fraud detection systems are functioning in regards to selecting returns suspected of refund fraud and the time it takes for those returns to be processed through those systems. This information will be yet another data point that can be considered when designing and modifying filters and developing procedures by which selected returns can be released.

ADOPTED, PARTIALLY ADOPTED or NOT ADOPTED: Adopted

OPEN or CLOSED: Closed

DUE DATE FOR ACTION (if left open): N/A

2
2.

TAS RECOMMENDATION #5-2

Develop criteria to be used in measuring OPR for IDT accounts.

IRS RESPONSE TO RECOMMENDATION: We agree on the importance of measuring the taxpayer experience when responding to potential IDT notices and letters. We continually look for opportunities to refine our process and provide additional clarity to taxpayers. In response to the feedback, the IRS has started tracking authentication timeframes. We will use this information to develop criteria for an IDT OPR that will provide insight on the functional impact of false detections.

Update: We continue to seek opportunities to refine our processes and provide additional clarity to taxpayers. IDT determinations are based on letters sent to the taxpayer and a subsequent taxpayer response. Due to the sequence of events needed to determine IDT, Operational Performance Rate (OPR) cannot be calculated in the same manner as Non-IDT. We did, however, develop a method to show how long it takes for false positives to be determined after selection. The attached slide demonstrates this effort. *Please note, the response time in the report includes effects from COVID-19 and not reflective of authentication times under normal conditions.

CORRECTIVE ACTION: We agree on the importance of measuring the taxpayer experience when responding to potential IDT notices and letters. We continually look for opportunities to refine our process and provide additional clarity to taxpayers. In response to the feedback, the IRS has started tracking authentication timeframes. We will use this information to develop criteria for an IDT OPR that will provide insight on the functional impact of false detections.

TAS RESPONSE: To truly evaluate the effectiveness of the IDT refund fraud program and its impact on taxpayers, it is critical that the IRS measure how long it takes taxpayers to authenticate their identity from the time the initial notice requesting authentication is sent to the taxpayer.  The IRS’s agreement to collect this information will help identify to what extent IDT refund fraud processing times are attributable to taxpayers authenticating their identity. In other words, this information will assist in determining if processing times are due to the authentication process or are caused by the release process after  authentication.

Update: RICS began tracking the open TPP inventory by age, as well as the average days to close for taxpayers shown as legitimate. In addition, RICS consistently lowered the False Detection Rate and improved the taxpayer experience by adding an aggressive release plan and improving Models to account for better accuracy.

ADOPTED, PARTIALLY ADOPTED or NOT ADOPTED: Adopted

OPEN or CLOSED: Closed

DUE DATE FOR ACTION (if left open): 

3
3.

TAS RECOMMENDATION #5-3

Conduct a study to determine why it takes some taxpayers longer to authenticate their identities and what barriers they may encounter when attempting to do so.

IRS RESPONSE TO RECOMMENDATION: The IRS agrees on the importance of better understanding the customer experience during the authentication process. We will conduct a study to determine the difference in timeframes that some taxpayers may encounter.

CORRECTIVE ACTION: The IRS agrees on the importance of better understanding the customer experience during the authentication process. We will conduct a study to determine the difference in timeframes that some taxpayers may encounter.

This activity has been on hold since the pandemic. RICS will continue to work with TAS to complete this initiative.

Update: “Attached is the focus group summary which details the methodology, findings and results. With the way it is written and the survey results/summary complete, it seems we may want to close this one out and start a new MSP. However, if the preference is to keep this one open, I changed the date to 1/31/2024, and we will work on the items previously discussed as well as analyzing the feasibility of the remaining recommendations. Please let me know if this date is feasible for the MSP team. We will make adjustments as we go.”

The survey was developed and subsequently disseminated to a test group within the Low-Income Taxpayer Clinic (LITC) in August 2022. RICS/RIVPM analyzed the survey feedback and recommendations from the test group, and met with TAS on September 19, 2022 to discuss the survey results along with additional feedback. The focus group summary is attached, detailing the methodology, findings, and recommendations. Recommendations include:

• Ability to authenticate identity at other federal agency offices (for example, at a US Post Office)
• Additional Tax Experience Days
• QR codes for additional information and for information in other languages
• Shorter letters and plain language.
• ID theft indicator on transcripts
• Additional focus groups and complete a study that will analyze readability of IRS letters
• Review filter selections
• Additional information on the Where’s My Refund application

TAS RESPONSE: This collaborative study is a significant step towards identifying what barriers taxpayers may face when attempting to authenticate their identity, and what other factors may be responsible for taxpayers’ delayed responses to IRS notices requesting the taxpayer to authenticate.  The results of this study will assist the IRS in measuring the time it takes a refund to be released from the time it is selected into the IDT refund fraud program. TAS looks forward to collaborating with the  IRS on design, implementation, and analysis of the study and the study’s results.

Update: TAS has been working with IRS to administer a taxpayer survey. The survey is awaiting OMB approval.

Update: Survey is in process and should be completed by the end of 2024.

Update: Recommendation 6-3 from the 2023 ID Theft MSP is the same as this recommendation. Closing this one and will continue to work with IRS on 2023 6-3.

ADOPTED, PARTIALLY ADOPTED or NOT ADOPTED: Adopted

OPEN or CLOSED: Closed

DUE DATE FOR ACTION (if left open): N/A

4
4.

TAS RECOMMENDATION #5-4

Design the refund fraud system to consider if applying the third-party information to the return would actually result in a larger refund when there is a mismatch between third-party information and the information on a taxpayer’s return.

IRS RESPONSE TO RECOMMENDATION: ​The IRS is developing a framework to study the information we receive from third parties in order to improve the selection of returns to review. This effort is included in the IRS’s rapid project development that permits appropriate changes to occur more quickly.

CORRECTIVE ACTION: The IRS is developing a framework to study the information we receive from third parties in order to improve the selection of returns to review. This effort is included in the IRS’s rapid project development that permits appropriate changes to occur more quickly.

TAS RESPONSE: This quick implementation of this recommendation will ideally reduce taxpayer burden by preventing taxpayers from being selected into the non-IDT refund fraud program, where a change to the income on the return would result in a larger refund, not a smaller one. In addition to reducing taxpayer burden, this will remove yet another segment of returns that should never have been selected in the first place, thereby allowing the IRS to focus on returns that truly deserve further scrutiny.

ADOPTED, PARTIALLY ADOPTED or NOT ADOPTED: Adopted

OPEN or CLOSED: Closed

DUE DATE FOR ACTION (if left open): N/A

5
5.

TAS RECOMMENDATION #5-5

Request from outside vendors information on ways to improve the FPR, along with proposals to determine the factors that are contributing to high FPRs.

IRS RESPONSE TO RECOMMENDATION: The IRS continues exploring ways to improve the False Detection Rate (FDR), which TAS refers to as the False Positive Rate (FPR). We seek input from stakeholders within the IRS, outside vendors, partners in state governments, and the tax preparation industry. The IRS seeks to strike a balance between protecting revenue and improving the taxpayer experience, and will continue to work with and develop both internal and external partnerships.

CORRECTIVE ACTION: The IRS continues exploring ways to improve the False Detection Rate (FDR), which TAS refers to as the False Positive Rate (FPR). We seek input from stakeholders within the IRS, outside vendors, partners in state governments, and the tax preparation industry. The IRS seeks to strike a balance between protecting revenue and improving the taxpayer experience, and will continue to work with and develop both internal and external partnerships.

TAS RESPONSE: This response communicates a commitment to continually evaluating the programs and how they can be improved in terms of protecting revenue and accuracy. However, the response lacks specificity, making it difficult to evaluate what the IRS has actually agreed to. Although the IRS is working with outside vendors, it is not clear that it is working with these vendors on what is an acceptable FPR and what factors may be contributing to such a high FPR. In fact, the IRS’s refusal to set a target FPR and its acceptance of FPRs over 50 percent for the last three years for both IDT and non-IDT refund fraud seems to indicate that to an extent, the IRS is willing to accept higher FPRs as long as processing times for these returns are relatively quick. However, since two of the non-IDT refund fraud filters exclusively select returns where either the Additional Child Tax Credit (ACTC) or Earned Income Tax Credit (EITC) has been claimed on the return (and where there is either no third-party documentation to support the income reported on the return, or the third-party information does not match the income on the return) they are likely selecting returns filed by low income taxpayers, and a delay of even three weeks in receiving a refund could cause a financial hardship. In order to consider this recommendation as “agreed to,” the IRS would need to provide more details, such as what vendors it is working with, what are the objectives, and what are the timeframes for meeting these objectives.

ADOPTED, PARTIALLY ADOPTED or NOT ADOPTED: Partially Adopted

OPEN or CLOSED: Closed

DUE DATE FOR ACTION (if left open): N/A

6
6.

TAS RECOMMENDATION #5-6

Establish a maximum acceptable FPR goal within industry accepted standards and an actionable timeline to achieve that goal, based on the information and proposals received from outside vendors.

IRS RESPONSE TO RECOMMENDATION: The IRS processes over 150 million returns every year and IRS fraud filter selections have protected about $12 billion per year over the last three years. Although we disagree with setting a target False Detection Rate (FDR) (which TAS refers to as the False Positive Rate (FPR)), we do look to improve selection efforts to reduce false detections. The IRS weighs the cost of lost government revenue, agency integrity, and the taxpayer burden associated with false  positives and  false negatives when configuring its fraud detection systems. Evolving cybercriminal schemes require an agile IRS  anti-fraud strategy. The IRS’ fraud detection strategy has resulted in fewer taxpayers requesting IDT victim assistance. By detecting fraud at the time of tax return submission, the IRS protects the legitimate taxpayer’s account, making it easier for the taxpayer to submit their return and receive their refund.

The IRS will continue to study the FDR and the factors that contribute to selections. We must minimize the burden of false detections while we protect taxpayers and government revenue from the risks posed by third-party data breaches and highly sophisticated cybercriminals.

CORRECTIVE ACTION: N/A

TAS RESPONSE: As stated a number of times before, the National Taxpayer Advocate fully agrees that protecting revenue is a critical component to effective tax administration. When addressing this key component, it is reasonable to have to balance a number of factors, including protecting revenue while mitigating the number of legitimate returns that are selected into the refund fraud program.

Just as the IRS establishes revenue and selection targets, it should also include an FPR target in this analysis, rather than treating high FPRs as a mere consequence of the program that cannot  be addressed or mitigated through adequate planning and design of the refund fraud program. An FPR target does not have to be an exact rate that indicates a success or failure, but rather is one factor to be considered when evaluating the refund fraud program’s overall success and could be established in the form of a small range, rather than an exact percentage. Further, as circumstances change during filing season, there may be a reasonable explanation why an FPR does not fall within the established range. However, by failing to set a target range for FPRs that is based in sound reasoning, the IRS omits an important objective from the overall development and planning of the refund fraud program.

ADOPTED, PARTIALLY ADOPTED or NOT ADOPTED: Not Adopted

OPEN or CLOSED: Closed

DUE DATE FOR ACTION (if left open): N/A