Limits of AI in Complex Elder Benefits Cases

AI is now making real decisions about elder care and benefits, from how many days of rehab a patient gets after surgery to whether someone qualifies for Medicaid, and the failure modes are well documented rather than speculative. This piece walks through how those systems break down for older adults specifically, using the denial data, the appeal statistics, and the regulatory gaps that let bad outcomes stand.

Insurers and administrators are not experimenting with this technology quietly on the margins. Predictive algorithms already approve or deny prior authorization requests at scale, frequently with little meaningful human review at the actual point of decision. Three zones of the elder benefits system now run substantial portions of their decision-making through automated tools: Medicare Advantage prior authorization, Medicaid eligibility determination, and Social Security disability claims processing. More than 31 million people are enrolled in Medicare Advantage plans alone, which means even a small error rate translates into a very large number of real people getting the wrong answer.

Why would an insurer build these systems in the first place? Faster processing and lower administrative overhead explain part of it, but the financial logic goes further than efficiency. CVS projected $77.3 million in savings over three years from its Post-Acute Analytics project. CMS's WISeR Model (Wasteful and Inappropriate Services Reduction), launched in January 2026 across Arizona, New Jersey, Ohio, Oklahoma, Texas, and Washington, pays private companies a share of the Medicare savings generated by "averted expenditures," meaning requests that never resulted in a paid claim. That is worth sitting with for a second: the contractor gets paid more when it says no. Critics have pointed out, reasonably, that this rewards denial as an outcome in itself, not just as a byproduct of accurate review. Running through 2031, WISeR also marks the first time AI-powered prior authorization has been built into traditional Medicare, not just Medicare Advantage. Understanding that incentive structure first is what makes the rest of this piece make sense.

Why pattern-matching against averages fails individual elders

Start with how these tools actually work. nH Predict, the tool built by NaviHealth and used within UnitedHealth Group's Medicare Advantage operations, estimates how many days of nursing-home or rehabilitation care a patient should need by comparing them against a database of "similar" patients. That comparison produces an average. It does not produce a clinical judgment about the specific person sitting in the hospital bed.

Older adults are, almost by definition, a bad fit for averaging. Multiple chronic conditions overlapping at once, recovery trajectories that don't follow a textbook curve, living situations and caregiver availability and social isolation, none of that shows up in a claims database. An 84-year-old recovering from hip surgery who lives alone with no one to help her navigate stairs needs a different care plan than a 71-year-old recovering from the same surgery with a spouse at home. The algorithm sees the same procedure code either way.

Published research has raised a related concern for Medicaid specifically: models trained largely on commercial insurance data may not generalize well to Medicaid's more socioeconomically and racially diverse population, risking biased predictions and unequal care. And there's a deeper problem underneath that one. If a model is trained on historical insurance decisions, and those historical decisions under-authorized care for certain groups of people, the model doesn't correct that pattern. It learns the pattern as normal and repeats it at scale.

Regulators tried to build in a check here: a licensed clinician has to review AI-generated denials. But what happens when that clinician can't actually explain why the algorithm reached its conclusion? Opacity doesn't just sit alongside the pattern-matching problem, it compounds it, because nobody in the review chain, not the reviewing clinician, not the patient, not the patient's doctor, can point to where the model went wrong for this specific person.

Medicaid adds a second layer of instability on top of all this: more than 50 distinct state rule sets, changing on their own schedules. California eliminated its Medicaid asset test starting January 2024, then reinstated it starting January 1, 2026 to address a state budget shortfall. Any AI tool calibrated to the 2024 rule is now handing out wrong answers, and it will keep doing so until someone updates it.

What the denial data actually shows inside Medicare Advantage

The numbers here are not ambiguous. A Senate investigation led by Senator Richard Blumenthal, released in October 2024, found that UnitedHealth's denial rate for post-hospital care rose from 10.9% in 2020 to 22.7% in 2022, tracking closely with its expansion of automated review.

OIG data from June 2024 on long-term care hospital denials puts real numbers next to the three most aggressive adopters of AI in this space: UnitedHealth Group at 71%, Humana at 72%, CVS Health at 80%. Sixteen other Medicare Advantage insurers, combined, denied at 42%. For inpatient rehabilitation facilities, the same pattern holds: United at 66%, Humana at 54%, CVS Health at 51%, versus 41% for everyone else grouped together.

That gap is not statistical noise. It is roughly double, in some categories, the baseline set by insurers who have not automated as heavily. The Senate committee's 2024 report found that AI tools in prior authorization have been accused of producing care denial rates as much as 16 times higher than typical. Humana's denial rate for long-term acute-care hospitals jumped 54% between 2020 and 2022, a steep increase that warrants scrutiny given how aggressively these insurers expanded automated review over the same period.

Physicians have noticed. In a 2024 survey, 61% of physicians reported concern that AI use by health plans is increasing prior authorization denials. When the three insurers furthest above the industry baseline are also the three that leaned hardest into automation, the causal story becomes difficult to wave away as correlation.

The appeal rate gap that lets bad denials stand

Here's where the picture gets genuinely troubling, because the denial numbers alone don't tell the full story. The class action lawsuit Lokken v. A major health insurance company and other named parties. alleges that nH Predict operates with a 90% error rate, a consequence, the complaint argues, of insufficient human review in the coverage denial pipeline. According to that same complaint, patients who actually challenged a denial driven by one algorithmic model won roughly nine times out of ten.

So why isn't this a bigger scandal? Because almost nobody appeals. Only about 0.2% of policyholders ever file a challenge, according to the complaint. The algorithm is losing nearly every fight it picks, and it's picking those fights against a population that, for the most part, never shows up to fight back.

KFF's analysis of federal records across all of Medicare Advantage in 2024 backs this up at a broader scale: patients appealed only about 11.5% of denied prior-authorization requests, and 80.7% of those appeals were overturned. Do the arithmetic on that gap for a second. If four out of five appeals succeed, but fewer than one in eight denials is even appealed, the overwhelming majority of wrongful denials simply stand, uncontested, by default. That gap between the reversal rate and the appeal rate functions as the system's real operating margin.

Why don't more people appeal? Physical and cognitive barriers make multi-step appeals processes hard to navigate for elderly patients in the first place. The denial letter itself often gives no clear explanation of why the algorithm reached its conclusion, and that opacity is not an accident of design, it is functionally what keeps the system from being contested. Then there's the clock: a patient who needs post-acute rehab right now cannot simply wait months for an appeal to work its way through the system. And plenty of people never learn that appeals succeed at these rates to begin with.

The real-world consequences of this dynamic surfaced in Minneapolis in October 2025, when Fairview Health Services cited high denial rates in threatening to stop scheduling appointments for seniors on UnitedHealthcare Medicare Advantage plans. The dispute was ultimately resolved with a one-year agreement in November 2025, but the underlying tension didn't disappear. When providers consider walking away from a payer over denial patterns, the elders caught in the middle, the ones who can't just switch plans mid-year, absorb the disruption.

Where the regulatory guardrails fall short in practice

Regulators have tried to respond. Rules and guidance issued in 2023 and 2024 state that Medicare Advantage organizations cannot make medical necessity decisions using an algorithm or software that fails to account for individual circumstances, and that any denial based on medical necessity has to be reviewed by an actual health care professional.

That sounds like a meaningful safeguard until you ask what the reviewing clinician is actually equipped to do. A licensed reviewer signing off on hundreds of cases a day may not have the time, or in some cases the technical background, to meaningfully interrogate why an algorithm reached a given conclusion. The rule requires a human in the loop. It does not require that human to understand the loop.

Momentum on further protection has stalled rather than built. Proposed 2024 regulations addressing bias and discrimination in Medicare Advantage AI use remained unfinalized under the Trump administration. HHS's Office for Civil Rights did finalize a rule in 2024 clarifying that Section 1557 of the Affordable Care Act's nondiscrimination principles apply to AI and patient care decision-support tools, but clarifying that a rule applies is different from enforcing it. The bipartisan Improving Seniors' Timely Access to Care Act, reintroduced in June 2024, failed to pass in the Senate.

At the state level, Medicaid oversight of AI is patchy at best. KFF's 50-state Medicaid budget survey found that only five states, California, Maryland, Nevada, New Hampshire, and Ohio, had introduced or planned to introduce contract language addressing AI use in managed care organization contracts. Plenty of other states voiced concern, about bias, about improper denials, about privacy risks and inadequate oversight, but had taken no formal action as of the FY 2025-2026 budget cycle.

One might argue the direction of travel matters more than any single rule. The WISeR Model is actively expanding AI-powered prior authorization into traditional Medicare at the exact moment the regulatory framework governing AI in Medicare Advantage remains unfinished. Policy is moving one way. Consumer protection infrastructure is not moving to meet it.

Medicaid eligibility as a moving target AI tools can't reliably track

Medicaid's structure resists standardized AI models almost by design. Managed care versus fee-for-service, optional versus mandatory eligibility pathways, more than 50 separate rule sets across states and territories: this is not a system built for a single generalizable model to learn once and apply everywhere.

Non-MAGI eligibility, the pathway that covers older adults and people with disabilities, is considerably more layered than standard income-based eligibility. Asset tests, spend-down provisions, multiple optional pathways that vary state to state, an AI system would need to track all of it simultaneously, and keep tracking it as states change their own rules on their own timelines.

California is the clearest case study available. Its asset test disappeared in January 2024 and returned on January 1, 2026. For two years, any AI-generated guidance built on the old rule would have been wrong, and wrong in a way that a beneficiary, someone without a law degree or a caseworker on speed dial, would have had almost no way to detect on their own.

The picture is getting more complicated, not less. H.R. 1, the 2025 reconciliation law, turns Medicaid eligibility from a one-time determination into an ongoing compliance obligation. Starting in 2027, many low-income adults will need to document at least 80 hours per month of work, job training, or community engagement just to keep their coverage. Redeterminations move from annual to every six months. The scale of potential coverage loss under these new requirements is projected to be substantial.

Even the AI industry itself recognizes the danger here. The Coalition for Health AI released two Best Practice Guides in May 2026 aimed at helping states deploy AI within Medicaid programs without triggering wrongful terminations. Guides like that don't get written for hypothetical risks. They get written because the risk of AI-driven wrongful termination is already understood to be real and, frankly, imminent. Generative AI compounds the danger further: research on Medicaid AI adoption has flagged serious risks of error in what are, for the person on the other end, genuinely high-stakes communications about whether they keep their health coverage.

How the Social Security disability system compounds these problems for elders seeking benefits

The Social Security Administration's disability claims backlog hit an all-time high of more than 1.26 million pending claims in June 2024. SSA has since reported cutting that backlog by more than 25%, down to 865,000, a level not seen since 2022, while also reducing average initial processing time by 13%, to 209 days, down from 240 days in January 2025.

Those numbers look like progress on their face. But what's driving the improvement? An Urban Institute analysis of SSA data through July 2025 found disability applications had actually decreased by 7%, a pattern that looks less like fewer people needing benefits and more like longer wait times and field office closures discouraging people from applying in the first place. SSA's initial-stage application outcomes have drawn scrutiny. And changes requiring in-person appointments to visit field offices land hardest on low-income clients with limited phone or internet access, which happens to overlap heavily with the elderly population this system exists to serve.

Advocates working directly with claimants have flagged another consequence: consolidation of regional offices has raised concerns about accountability mechanisms within SSA. Critics have noted that SSA's end-of-year report offers little historic data against which to judge how significant the claimed progress actually is. Without a real baseline, the metrics the agency touts are hard to verify on their own terms.

For elderly claimants specifically, the burden compounds. Cognitive and mobility limitations make an increasingly phone- and digital-first system harder to navigate, and a 209-day average wait carries real health consequences when the benefits in question are tied to needed medical care.

What caregivers and seniors need to do when AI-driven decisions go wrong

Start with the single most important fact in this entire piece: 80.7% of Medicare Advantage prior-authorization appeals were overturned in 2024, per KFF. A denial letter is not a verdict. It is an opening position, and the odds favor the person willing to push back.

When a denial arrives, request a written explanation of the reasoning behind it. AI-based denials frequently cite population averages rather than the specific clinical facts of the case in front of them, and naming that gap explicitly is the foundation of a strong appeal. Ask the treating physician to document, in writing, why this particular patient's situation doesn't fit the algorithmic profile used to generate the denial. The algorithm was built to operate without capturing that individualized clinical context. File within the appeal deadline. Medicare Advantage plans operate on defined timeframes, and missing one forfeits the right to challenge the decision at all. For anything time-sensitive, like post-acute rehab that can't simply wait, request an expedited appeal; a standard timeline can make the entire fight moot by the time it resolves.

For Medicaid questions, verify current rules directly with the state agency before acting on anything an AI tool suggests. California's two-year reversal on its asset test is a compelling illustration for why Medicaid guidance from an algorithm has a shelf life measured in months, not years.

For Social Security disability claims, document every contact with the agency, attempted field office visits, phone calls, everything. Access barriers are becoming part of the formal record of how this system actually functions on the ground, and that documentation matters to the advocates working to hold it accountable.

This is where human expertise earns its place. Elder law attorneys, benefits counselors, and enrollment support services bring exactly the individualized context and accountability that these systems are structurally unable to provide, and the appeal reversal numbers suggest the cases most worth fighting are often the very ones AI handles worst: the complex, atypical, doesn't-fit-the-average cases. AI tools built to help beneficiaries discover which programs they actually qualify for, and guide them through enrollment, solve a genuinely different problem than the denial algorithms do. Discovery and navigation are not the same challenge as utilization management, and conflating them is part of how this conversation gets muddled. Pairing that kind of assistive AI guidance with human review for anything complex or contested reflects where this technology actually performs well, rather than where it's currently being deployed to cut costs.

Opacity is the default setting of this entire system, at every layer covered here, from Medicare Advantage to Medicaid to Social Security disability. Pushing back against it, appealing, asking for a plain explanation, insisting on documentation, is not a bureaucratic afterthought. It is the only mechanism currently available that reliably works.