Who Did It Better? When AI Meets MSL Expertise

Are you worried that ChatGPT will replace you as an MSL? A recent publication (Am J Health-Syst Pharm. 2025;82:448-460) from an academic center indicates we shouldn’t be worried if accurate answers are important! 

The study blindly compared ten previously answered drug information center (DIC) responses to ChatGPT answers in two phases. The first phase assessed whether the answer was accurate or inaccurate, with ChatGPT having an overall accuracy of 50%. 

In the second phase, ten of the ChatGPT responses were blindly compared by 2 pharmacists and a 4th year pharmacy student to the 10 DIC responses using a rubric. In this phase, accuracy was not evaluated. ChatGPT scored higher in the categories of background, clarity and conciseness, no misleading information, answered the question, and readability. The DIC scored higher in the categories related to references as ChatGPT does not provide references. The DIC and ChatGPT scored the same on the recommendations provided. On average across the redefined eight component rubric, ChatGPT scored higher than the responses written by DIC (67.9% vs 55%, respectively). Limitations of the tool and study are outlined in the paper, including that the of the tool utilized only had data through 2021. The authors concluded that although ChatGPT could be used to address drug information questions, the answers would need to be reviewed for accuracy and completeness. ChatGPT can be used to enhance drug information response clarity and readability. 

Are you using ChatGPT to answer medical questions?

What steps are you taking to ensure accuracy?