Research Article: Benchmarking the performance of large language models on medical genetics questions: a comparative evaluation of GPT-5.1, Gemini Pro 3, and Claude Opus 4.5
Abstract:
This study benchmarks the performance of three state of the art large reasoning models comprising GPT 5.1, Gemini Pro 3, and Claude Opus 4.5 by evaluating their accuracy, response consistency, processing speed, and output length on medical genetics questions from the Turkish Medical Specialty Examination (TUS).
A total of 233 medical genetics questions were selected based on predefined inclusion criteria covering molecular mechanisms, inheritance patterns, genetic disorders, and diagnostic techniques. Each question was independently submitted to GPT 5.1, Gemini Pro 3, and Claude Opus 4.5 using a standardized prompt. Model accuracy was compared using McNemar's test. Response consistency was assessed using Cohen's Kappa. Response time and word count were analyzed using the Friedman test with Dunn Bonferroni post hoc comparisons.
No statistically significant differences were observed in overall accuracy among the models ( p >?0.05). GPT 5.1 achieved the highest accuracy at 97.9 percent followed by Claude Opus 4.5 at 96.6 percent and Gemini Pro 3 at 96.1 percent. Cohen's Kappa indicated moderate qualitative divergence between Gemini Pro 3 and Claude Opus 4.5 with kappa equal to 0.451. Significant differences were found in response time and output length ( p <?0.001).
Although all models demonstrate similarly high accuracy, they differ in operational behavior, suggesting distinct internal knowledge representations and response strategies.
Introduction:
In recent years, large language models (LLMs) have rapidly evolved into large reasoning models, enabling multi-step planning, autonomous agent-like behaviors, and multimodal information synthesis. As the leading representatives of this technological leap, OpenAI's GPT-5.1, Google DeepMind's Gemini Pro 3, and Anthropic's Claude Opus 4.5 offer solutions to complex data-intensive problems through distinct architectural philosophies and operational capabilities ( 1 – 3 ). Evaluation and assessment are…
Read more