Large language models (LLMs) are increasingly recognized as potential decision-support tools across various medical fields. Recent research has specifically explored the role of LLMs in anesthesia, focusing on their ability to assign American Society of Anaesthesiologists (ASA) physical status classifications and generate appropriate anesthetic protocols. A retrospective study conducted at Atatürk University Veterinary Hospital analyzed 225 feline and canine cases to benchmark the performance of ChatGPT-4o, ChatGPT-5, and Gemini 2.5 Pro.

Accuracy of ASA Physical Status Classification

Correctly identifying a patient's ASA status is critical for risk stratification and perioperative planning. In this study, ChatGPT-5 demonstrated the highest accuracy in ASA classification at 53.3%. This was followed by ChatGPT-4o at 46.7% and Gemini 2.5 Pro at 30.7%. Interestingly, the models performed significantly better in higher-risk categories (ASA 3‒5). Conversely, healthy patients in the ASA 1 category were frequently misclassified. This discrepancy often stemmed from an overestimation of risk, where the AI models flagged minor clinical findings as more severe than an experienced anesthesiologist would.

Clinical Sufficiency of Anesthetic Protocols

Beyond risk scoring, the researchers evaluated the clinical adequacy of the generated anesthetic protocols using a four-point scale. ChatGPT-5 emerged as the superior model, consistently producing the most clinically sufficient protocols compared to its counterparts. However, while the models could synthesize complex data into a structured format, the study emphasized that model-to-model variability remains a concern. Furthermore, clinicians observed that the models occasionally suggested drug combinations that required careful expert adjustment.

The Future of LLMs in Anesthesia

While the study utilized veterinary cases, the findings offer valuable insights for human medicine. The ability of LLMs to process unstructured clinical data and provide rapid decision support is promising. Nevertheless, the high rate of misclassification in healthy patients underscores a current limitation: the tendency for AI to prioritize caution over clinical nuance. Experts suggest that while these tools can streamline documentation and protocol drafting, they cannot yet function autonomously.

Frequently Asked Questions

How do LLMs in anesthesia assist with risk assessment?

LLMs can analyze patient history and clinical data to suggest an ASA physical status score, helping clinicians categorize preoperative risks more consistently, though they currently require expert validation.

Which model performed best in determining ASA status?

In the Atatürk University study, ChatGPT-5 achieved the highest accuracy (53.3%) and generated the most clinically relevant anesthetic protocols compared to ChatGPT-4o and Gemini 2.5 Pro.

Can AI models currently replace human anesthesiologists?

No. The study concludes that performance varies significantly across different models and cases, making expert oversight essential for ensuring patient safety and protocol accuracy.

Disclaimer: This content is for informational and educational purposes only. It is not intended to provide medical advice or to be a substitute for professional medical judgment, diagnosis, or treatment. Refer to the latest local and national guidelines for clinical practice.

References

1. Okur S et al. Assessing the performance of large language models when used to determine ASA status of cats and dogs and generate anaesthetic protocols. Vet Rec. 2026 May 12. doi: 10.1002/vetr.70741. PMID: 42117364.


2. Ramaswamy et al. Independent evaluation of LLM triage performance in emergency presentations. Nature Medicine. 2026 February.


3. Kaytancı Özçelik D. Agreement Between ChatGPT-5 and Anesthesiologists in Preoperative Risk Assessment. ClinicalTrials.gov ID NCT07459491. 2026.