why choose us

300×250 Ad Slot

Research Article: Multi-environment evaluation and decision-level analysis of clinical prediction models under dataset shift

Date Published: 2026-10-15

Abstract:
Clinical machine-learning models are often evaluated within a single dataset, although deployment occurs across heterogeneous institutions and measurement processes. We evaluated Logistic Regression, Random Forest, and Gradient Boosting for ICU hospital-mortality prediction across MIMIC-III Demo and eICU using a 24-h landmark design, explicit feature mappings, and predictors retained only when sufficiently observed in both domains. A decision-level diagnostic analysis examined how predicted-probability distributions interact with a fixed 0.5 threshold. The primary cohort contained 110 MIMIC and 1,324 eICU ICU episodes. In eICU-to-MIMIC transfer, AUC-ROC was 0.598 for Logistic Regression, 0.642 for Random Forest, and 0.613 for Gradient Boosting. Random Forest generated no positive predictions, a threshold failure already visible during internal eICU validation. Logistic Regression retained partial sensitivity with substantial false-positive burden, whereas Gradient Boosting detected only two deaths. These findings show that clinical model evaluation should examine calibration, threshold behavior, feature harmonization, and source-to-target operating-point stability, not only aggregate discrimination.

Introduction:
The use of machine learning models in clinical prediction has grown steadily due to the availability of electronic health records (EHRs) and the ability of these records to capture complex relationships among physiological and biochemical variables. However, much of this development has been evaluated under the assumption of distributional homogeneity, in which training and test data come from the same clinical setting. This condition, while convenient from an experimental perspective, does not reflect the…

Read more

300×250 Ad Slot