Clinical risk prediction

A published score can be a good starting point, but applying it without validating it in your own population means trusting a prediction whose performance and calibration have not been checked in that setting. This applies when a department wants to use a risk model, whether for deterioration, mortality or readmission, and needs to know how far it can rely on it with its own patients. A score developed in another population, another health system or another period may discriminate less well or, above all, be miscalibrated once transferred: it may predict risks that are systematically too high or too low. Before it is built into care decisions, the score should be validated locally, or recalibrated if its calibration is not adequate. The service either recalibrates the existing model on the centre's real data or develops a new one, and in both cases quantifies its performance with discrimination and calibration metrics. The difference is knowing whether the prediction works here, rather than assuming it will because it worked somewhere else.

You need clinical, laboratory and vital-sign data from a cohort of patients in which the outcome you want to predict is recorded: deterioration, readmission, mortality or a complication. The client collects and provides these data; the service prepares, models and interprets them. A key check at this stage is the timing of the variables: the model can only use information that will be available at the actual moment the prediction is made, not data recorded afterwards. Using a future variable as a predictor is a common source of information leakage: it produces a model that looks excellent in development and falls down in practice. That is why the availability and recording time of each candidate variable are assessed before modelling. The database does not need to arrive clean, but the outcome does need to be well defined. If the outcome is poorly defined, varies between centres or has no clear time window, the first task is not modelling but reconstructing a defensible operational definition.

No. The model provides an individualised, objective risk estimate; the clinical decision is always made by the clinician. Clinical risk and readmission prediction supports complex decisions, such as intensifying monitoring, considering ICU admission or weighing up surgery, by quantifying a risk that used to be implicit. It does not replace clinical judgement or knowledge of the individual patient. Its role is to inform clinical decisions, or shared decisions between clinician and patient where appropriate, with data, not to automate them. This boundary is deliberate and does not blur: the service delivers an estimation tool that has been evaluated, calibrated and made interpretable, ready to be integrated into the workflow, so that the team can decide with better information. Responsibility for the care decision stays with the clinician; the model supports it, it does not replace it.

The AUC is not enough. The reliability of a predictive model is assessed by combining discrimination, calibration and clinical utility, not with a single number. Discrimination (measured with the AUC) shows whether the model separates those who will have the event from those who will not, but a high AUC does not mean the probabilities are correct. Calibration checks exactly that: among patients with an estimated risk of around 20%, the observed frequency of the event should be roughly 20%, which matters whenever you act on the predicted value. The model's net benefit is also assessed, that is, whether using it does more good than not using it at the relevant decision thresholds. These metrics should be estimated on data not used to train the model, or through appropriate internal validation, to avoid an optimistic estimate of performance. A model that discriminates well but is poorly calibrated can lead to wrong decisions, which is why all three dimensions are reported.

Machine learning may be the better choice when the complexity of the data (many variables, non-linear interactions, relationships that are hard to specify) justifies it, and a classical statistical score when a well-built regression captures the signal more simply and transparently. The decision is not ideological: several approaches are developed and compared, from penalised regression to machine learning algorithms, and the one chosen is whichever best combines performance and applicability in the real setting. With few events or very limited data, a simpler, well-calibrated model may be preferable to a complex algorithm, which in that scenario tends to overfit. A more complex model is only justified if it brings a real gain in performance that makes up for its lower transparency. When machine learning is used, it comes with explainability techniques such as SHAP, which show which variables drive each prediction, so that the model does not act as a black box for the clinician or the reviewer.

The risk threshold should not be set only to maximise detection, but to balance sensitivity against the resources available and the real cost of false positives in the care setting. This applies whenever a model is going to be used to trigger alerts or referrals, because more alerts are not better if the team cannot respond to all of them: alert fatigue means that an overly sensitive system ends up being ignored. That is why the cut-off is set with clinical and operational judgement, taking into account the capacity to respond and the cost of each false positive and each false negative, which rarely weigh the same. Net benefit can also be assessed with decision curves when there are clinically relevant thresholds. The aim is an alert system the team can respond to sustainably, not one that fires so many signals that it becomes useless.

Clinical risk and readmission prediction is used to estimate each patient's probability of a specific future event, such as deterioration, 30-day readmission or death, usually to trigger an intervention in time. Patient stratification models fit when the aim is to group a population into risk levels or profiles to organise management and care, rather than to predict a single event. The two can be combined, since a risk prediction can feed a stratification, but they do not answer the same question. The predictive model answers "how likely is this patient to experience this event within a defined time window?"; stratification answers "which management groups does this population fall into?". One guides a timely individual intervention; the other, the organisation of resources by risk level.

The service does not end with delivering the model: it can design the evaluation of the model's impact on clinical outcomes. This applies when the centre needs to show with data whether the model, once in use, is reducing deterioration, readmissions or prolonged stays, and not only whether it works on paper. Good statistical performance does not in itself bring a clinical improvement: that improvement depends on how the model is integrated into the workflow and on what is done with the prediction. That is why we design an evaluation of its real impact on the outcomes of interest. Depending on how the model is rolled out, that evaluation may require a before-and-after design, a phased evaluation or a dedicated implementation study. The service designs and analyses the evaluation; carrying out the interventions and responding to the alerts are the responsibility of the care team. What you receive is evidence on whether the model adds value in that care setting, not just the model.