Abstract
This study investigates the quality of the over-the-counter medicines recommended by commercial general purpose artificial intelligence (AI) platforms by analyzing the relationship between two categorical data. 139 tests were performed using publicly available over the counter medication database from US FDA and two widely available commercially available general purpose AI platforms. A chi-square analysis was conducted to examine the relationships. Results indicated that there is a statistically significant difference between the quality of over-the-counter medication recommendations from the FDA Database-based system and those recommended by commercially available generative AI systems with chi-square statistic 0.1208, the p-value of .728145, not significant at p < .01 for one AI system and the chi-square statistic = 0.9988, the p-value of .317613, not significant at p < .01 for second AI system. These results indicate that recommendations made by widely available generative AI platforms are not suitable for medical advice. Additionally, the study found that there is no statistically significant difference between the quality of over-the-counter medication recommendations from two commercially available generative AI systems with the chi-square statistic = 61.4366, the p-value of < 0.00001, significant at p < .01. This result suggests that the recommendations made by these generative AI systems are consistently inaccurate. These findings highlight the need for the development of specialized generative AI systems with the involvement of domain experts, to ensure accuracy and reliability for providing the OTC medication recommendations.
Presenters
Rahul PatelAdjunct Industry Associate Professor, Information Technology and Management, College of Computing, Information Technology and Management, Illinois Institute of Technology, Illinois, United States
Details
Presentation Type
Paper Presentation in a Themed Session
Theme
2026 Special Focus—Human-Centered AI Transformations
KEYWORDS
Generative artificial intelligence, AI recommendation quality, Commercial AI platforms
