Chatbot, help me understand my benefits

Can AI help us understand our benefits? We explored the possibility.

Understanding your benefits can feel like learning a new language. There are new terms and definitions, a new way to do Math, everything feels backward, and you've got to quickly apply the new concepts as you make decisions for yourself and others. But this time, you can't ask your high school language teacher for help.

If you feel this way, you're not alone. Here is some data on Americans' understanding of their benefits:

  • Three in ten Americans are unsure if they use all their benefits.
  • Six in ten health plan detractors are unsure if they use all their benefits.

Can AI chatbots help us understand our benefits?

Could the aid of a natural language processing tool help clear the confusion? People have used AI chatbots to do all sorts of things, like filling out their NCAA tournament bracket or writing long Twitter threads headlined by fictional stats.* But what if we used AI chatbots to do something productive for our everyday life, like understanding our healthcare benefits? I explored this by posing questions and real-life benefit decision scenarios to OpenAI's ChatGPT and the new AI-powered Bing.

*Many content creators on social media often reference stats such as "97% of people don't know how to use ChatGPT.” We’ve never seen a source. 

I asked and received permission from all interview subjects. 

What's the difference between an HSA and an FSA?

While more than half of Americans confuse health savings accounts (HSAs) with flexible spending accounts (FSAs), both ChatGPT and Bing aced explaining their differences, stating "individuals control an HSA and allow contributions to roll over" and "FSAs are less flexible and owned by an employer" and "HSAs used in conjunction with a high-deductible health insurance plan."

Bing’s chatbot did a great job breaking down the differences between an HSA and an FSA.

In contrast to ChatGPT, Bing cited its sources. This warmed my content marketer's heart, but it's also very helpful for everyday consumers. If they have follow-up questions on the subject, they now have a relevant source for continuing their learning journey.

But what does FSA really stand for?

The FSA acronym is a commonly confused term within the industry. While most commonly referred to as a flexible spending account, it's technically an arrangement. That's how the ultimate authority, the Internal Revenue Service, defines the term.

ChatGPT did a better breakdown of the acronym FSA than I’ve seen from most industry experts.

Both Bing's chatbot and ChatGPT had great answers to this question. But in the end, language evolves and reflects the people speaking it. Like most companies offering an FSA product, First Dollar refers to FSAs as flexible spending accounts because that's what people search for. (This also follows the lead of other government organizations like Healthcare.gov.)

As seen here, "flexible spending accounts" has won the FSA battle. Source: Google Trends

What’s the right health plan for Jenna?

I asked the chatbots to play the role of Jenna Smith: a lawyer, wife of Jason Smith, and mother of Jaylen. While they pointed out they could not take human form, they offered to help Jenna choose the best health plan offered by her employer for her family. I shared the usual plan info with the chatbots to help them make their analysis: in- and out-of-network rates for co-insurance, deductibles, out-of-pocket expenses, and more. 

Bing believed that Plan 2 was the better choice for Jenna’s family.
ChatGPT gave comprehensive reasoning behind why it favored plan #2.

Both AI tools favored the second plan, but I was most impressed by ChatGPT's bullet-by-bullet breakdown on why Jenna Smith should choose the second plan. Choosing the right health plan can feel overwhelming, and these tools help make this process feel more manageable.

What if my son has an existing health condition?

I was curious how the tools would respond to additional variables, such as a family member with a health condition like diabetes.

ChatGPT analyzed the additional information and gave an update on its preferred plan.

Both tools still recommended the second health plan for Jenna, but the recommendation wasn't quite as strong. I felt that was appropriate, knowing that the second plan offered additional opportunities to save for diabetes care through tax-advantaged means.

How do we improve U.S. healthcare?

With all this success, I asked the chatbots to solve the entire system with a sentence.

ChatGPT pointed out there is no single solution.
Bing ignored my one-sentence limit and pointed to a need to shift to focus on preventative measures.

The answers were not as strong as previous responses, but perhaps I got a little too greedy. After all, solving the United States healthcare system is probably not a task that can be solved with a prompt.

Here's a shameless plug.

What's a blog post on a company website without a mention of the product? I was curious if ChatGPT would know how First Dollar can help anyone launch an embedded benefit, and I was pleased with its answer.

Anyone can launch a benefit on their platform through First Dollar.

Conclusion: Exciting new tool for understanding and using benefits.

AI chatbots offer an exciting opportunity for consumers and the organizations that provide them benefits (financial institutions, health plans, and third-party administrators). Here are some questions that both should consider:

  • Benefit providers: Can we guide our customers to leverage these tools while mitigating potential errors? How do we ensure our employees protect customer information by not sharing private data with AI tools?
  • Consumers: What are the best prompts to use for understanding healthcare benefits? What are the flaws of using AI chatbots to understand my benefits?

Ultimately, AI chatbots are new tools that are good at researching and breaking down dense information. That sounds like a great tool to be used for understanding healthcare benefits!

Methodology

All of my conversations were with the Bing chatbot on its "Balanced" conversation style setting. My conversation with ChatGPT was on its free March 14 version, which references GPT-3.5. Bing's chatbot was on its March 20 version, which references GPT-4. I wrote my prompts beforehand to ensure consistency with the two chatbots but adjusted in response to the chatbot's answers (as you would in a normal conversation.)

Josh Hostetler

Josh leads content for First Dollar, a fintech company that builds infrastructure for health spending benefits. Before First Dollar, Josh led course creation at Aceable, taught First Grade, waited tables at Olive Garden, and wore many other hats. He misses the breadsticks.