Author: Randy Maniloff
Putting ChatGPT to the Insurance Coverage Test
The media has been awash in stories about ChatGPT. The artificial intelligence platform – offering answers to questions in human-like conversation -- is being described by some as life changing.
This weekend's Wall Street Journal featured a several-thousand-word essay on the subject by Henry Kissinger, and others, claiming that the “new technology bids to transform the human cognitive process as it has not been shaken up since the invention of printing [in 1455]." School districts around the country have banned ChatGPT, fearful of students using it to do their homework or write term papers.
[First of all, as far as the most important life-changing technology goes, that would be the invention of watermelon without seeds. How in the world did they do that? Everything else is competing for second place. I really hope that the scientist who figured that out is now hard at work on growing fruits and vegetables with a bar code. Everything is fine in the self-serve check-out aisle until you get to the apples. And is a yam the same a sweet potato?]
Anyway, with all of this talk about ChatGPT, I decided to put it to the insurance coverage test. Very simple. I asked it five questions. I chose questions where the answer has been clearly stated in a state supreme court decision. No trick questions. No questions with nuance answers or where the law is not clear, or it depends on the policy language. These are questions that should seemingly be softballs for an artificial intelligence platform being described in the way that ChatGPT is.
The verdict – ChatGPT got 4 out of 5 wrong. But not just wrong – dead wrong. I also added a fun question, and it got that one right.
[To be honest, the real purpose of this exercise was my concern that Insurance Key Issues could be put out of business by some Silicon Valley nerds. But Key Issues will live on – at least until the technology improves.]
Here is the Chat GPT test:
Maniloff: Under Pennsylvania law, is extrinsic evidence permitted to determine an insurer's duty to defend?
ChatGPT: “Yes, extrinsic evidence is permitted to determine an insurer's duty to defend in Pennsylvania. In order to determine whether an insurer has a duty to defend, courts in Pennsylvania will review the language of the insurance policy and the facts of the underlying case, including any extrinsic evidence that is offered."
[This is not correct. See Kvaerner Metals Div. of Kvaerner U.S., Inc. v. Commercial Union Ins. Co., 908 A.2d 888 (Pa. 2006).]
***
Maniloff: Under Washington law, can an insurer seek reimbursement of defense costs following a determination that it had no duty to defend?
ChatGPT: “Yes, an insurer may seek reimbursement of defense costs under Washington law. According to Washington state law, an insurer may be entitled to reimbursement of defense costs if it can prove that it had no duty to defend the insured, and that the insured's claims are not covered by the policy."
[This is not correct. See National Surety Corp. v. Immunex Corp., 297 P.3d 688 (Wash. 2013).]
***
Maniloff: Under Illinois law does the pollution exclusion preclude coverage for carbon monoxide poisoning?
ChatGPT: “Yes, the pollution exclusion in Illinois law typically precludes coverage for carbon monoxide poisoning."
[This is not correct. See Am. States Ins. Co. v. Koloms, 687 N.E.2d 72 (Ill. 1997).]
***
Maniloff: Is coverage owed for pre-tender defense costs in New Jersey?
ChatGPT: “Yes, in New Jersey, insurers are generally required to provide coverage for defense costs incurred prior to a tender of defense. However, the insurer may be able to deny coverage or seek reimbursement for some of these costs if they can show that the costs were unnecessary or unreasonable."
[This is not correct. See SL Indus., Inc. v. Am. Motorists Ins. Co., 607 A.2d 1266 (N.J. 1992).]
***
Maniloff: “Are punitive damages insurable under Oklahoma law?"
ChatGPT: “Punitive damages are generally not insurable under Oklahoma law. However, limited exceptions may apply in certain cases. It is important to review the specific language of any insurance policy in order to determine if punitive damages are covered."
[This is correct. See Dayton Hudson Corp. v. Am. Mut. Liab. Ins. Co., 621 P.2d 1155 (Okla. 1980). However, Dayton Hudson allows coverage for punitive damages awarded on a vicarious liability basis. Perhaps this is what was meant by a limited exception that may apply in certain cases.]
***
Maniloff: Is it bad faith for an insurer to make a coverage determination using a Magic 8 ball?
ChatGPT: “Yes, it is bad faith for an insurer to make a coverage determination using a magic 8 ball. This is because a magic 8 ball does not provide reliable or accurate information, and therefore cannot be used to make any kind of informed decision. Furthermore, relying on a magic 8 ball for coverage determinations would be seen as an act of bad faith, as the insurer would not be taking the necessary steps to ensure that the policyholder receives an accurate and fair coverage decision."
I should have asked a follow-up – OK, but what about a Ouija board?
I figured that ChatGPT would struggle with the test. But with all the hubbub about it, I never imagined that it would flunk this badly – especially with the easiest questions possible.
This article appears courtesy of Randy Maniloff at CoverageOpinions, White and Williams, LLP.
Last Updated: March 31, 2023
_______________________________________________________________________________________________
Copyright © 2023, Big “I" Virtual University. All rights reserved. No part of this material may be used or reproduced in any manner without the prior written permission from Big “I" Virtual University. For further information, contact jamie.behymer@iiaba.net.