Abstract:Objective To study and compare the differences between ChatGPT o1 and Claude 3.5 Sonnet in answering common questions related to unicentric Castleman disease in the neck. Methods A total of 36 common questions related to unicentric Castleman disease of the neck were designed and input into the ChatGPT o1 and Claude 3.5 Sonnet search engines. The answers generated by ChatGPT o1 and Claude 3.5 Sonnet were independently evaluated by a professor of otolaryngology-head and neck surgery. The evaluation contents included the readability, accuracy, quality, understandability, and practical operability. Results Regarding readability, Claude 3.5 Sonnet generated significantly more concise responses across all categories (189.36±69.09 vs. 381.56±153.28, P<0.05), with lower reading scores (1.68±5.64 vs. 11.20±11.16, P<0.05) and higher grade-level scores (54.93±35.81 vs. 16.70±2.03, P<0.05). In terms of understandability and actionability measured by the patient education materials assessment tool-patient (PEMAT-P) score, Claude 3.5 Sonnet demonstrated higher overall understandability (0.38±0.17 vs. 0.06±0.05, P<0.05) and actionability (0.25±0.22 vs. 0.08±0.09, P=0.015). Nevertheless, ChatGPT o1 had a higher overall accuracy score (4.88±0.28 vs. 4.58±0.37, P=0.002 2) and attained a superior quality rating under the revised evidence-based quality indicator for patient education (EQIP) criteria (7.47±1.28 vs. 5.75±1.20, P<0.05). Conclusion Claude 3.5 Sonnet is superior in conciseness, understandability, and actionability, whereas ChatGPT o1 is stronger in terms of accuracy, overall quality and readability.