Large language models like Chat-Generative Pre-Trained Transformer (ChatGPT) have become novel resources for patients, especially as physicians struggle to keep up with the unsustainable number of electronic messages and questions they get from their patients.
It turns out that Chat-GPT may be a reliable resource and supplement to physician-dispensed advice on atopic dermatitis (AD), according to a research letter in the Journal of the European Academy of Dermatology and Venereology.
What is ChatGPT?
ChatGPT is a form of artificial intelligence (AI) that serves as a conversational chatbot. It is trained by a broad range of internet sources, with the ability to use reinforcement learning from human user feedback.
In an open study evaluating the reliability of ChatGPT responses, researchers submitted 99 common questions from AD patients to the most currently available model, ChatGPT-4.
The questions were supplied by an international group of 11 dermatologists. Each contributed up to 10 questions that their patients commonly asked including inquiries regarding the etiology, management, and consequences of AD. The ChatGPT responses were then independently evaluated by a group of international dermatologist experts in therapeutic patient education.
The overall quality and reliability of ChatGPT’s responses were assessed using the Likert scale from 1 to 10, with 10 being the best, and were flagged for harmful information. The graded responses were averaged and ranged from 8.18 to 10, indicating high levels of evaluator satisfaction.
When Chat GPT-4 was asked “What causes eczema?”, its answer scored a 10.
Here’s Chat GPT-4’s response:
“Eczema, also known as atopic dermatitis, is a chronic skin condition characterized by red, itchy, and inflamed skin. The exact cause of eczema is not completely understood, but it is believed to result from a combination of genetic, environmental, and immunological factors. Genetic factors: People with a family history of eczema, asthma, or hay fever (atopic diseases) are more likely to develop eczema. Certain gene mutations, such as those affecting the filaggrin protein, have been associated with an increased risk of developing the condition. Filaggrin plays a critical role in maintaining the skin’s protective barrier. Environmental factors: Exposure to irritants (e.g. soaps, detergents, perfumes) or allergens (e.g. pollen, pet dander, certain foods) can trigger or worsen eczema symptoms. Additionally, weather conditions like extreme temperatures or low humidity can cause skin dryness, further aggravating the condition. Immune system dysfunction: an overactive immune system response is believed to play a role in eczema development. This overreaction can lead to inflammation and damage to the skin barrier, resulting in the characteristic symptoms of eczema. Skin barrier dysfunction: eczema-prone skin is often more susceptible to dryness and irritation, as it tends to have a weaker barrier function. This allows allergens and irritants to penetrate the skin more easily, triggering inflammation and itchiness. Eczema can vary in severity and may present differently from person to person. Common symptoms include dry, itchy, and red skin, along with recurring rashes. Eczema often appears in infancy or early childhood, although it can develop at any age. Treatment typically involves managing symptoms with emollients, topical corticosteroids, and other medications as well as avoiding triggers and maintaining a healthy skincare routine.”
In all, ChatGPT-4 “provided thorough, high-quality responses without any special priming or additional information,” the authors conclude.
“I really think this is a watershed moment: the AI passed its test with flying colors,” lead study author Peter A. Lio, MD, a Clinical Assistant Professor of Dermatology and Pediatrics at Northwestern University Feinberg School of Medicine and a partner at Medical Dermatology Associates of Chicago, tells TDD.
Moreover, there was overwhelming reliability of the responses provided by ChatGPT to the wide-ranging questions frequently asked by patients. “Most responses acknowledged that it was not a physician and directed users to a healthcare provider,” the researchers wrote.
Though no responses were flagged for dangerous information, lower scores were reported for incomplete answers or where advice deviated from evidence-based medicine, the study showed.
