Predicting Early Death in Patients With Traumatic Bleeding
Predicting Early Death in Patients With Traumatic Bleeding
For the development of the prognostic model, we involved potential users from three settings: pre-hospital, battlefield, and emergency departments. We held meetings with paramedics, military doctors, and consultants in emergency medicine to identify variables and interactions that they considered important and convenient for their settings and to obtain information on how to present the prognostic model in a user friendly format.
We included patients from the Clinical Randomisation of an Antifibrinolytic in Significant Haemorrhage (CRASH-2) trial. The trial included 20,127 trauma patients with, or at risk of, significant bleeding, within eight hours of injury, and took place in 274 hospitals in 40 countries. The primary outcome was all cause mortality. Patients’ outcomes were recorded at discharge, at death in hospital, or 28 days after injury, whichever occurred first.
We took variables to be analysed as potential predictors from the patients’ entry forms completed before randomisation. Variables included in the entry form for the CRASH-2 trial can be divided into patients’ demographic characteristics (age and sex), characteristics of the injury (type of injury and time from injury to randomisation), and physiological variables (Glasgow coma score, systolic blood pressure, heart rate, respiratory rate, and central capillary refill time).
Age was recorded as a continuous variable measured in years. Type of injury had three categories—penetrating, blunt, or blunt and penetrating—but we analysed it as “penetrating” or “blunt and penetrating.” Time from injury was recorded as a continuous variable measured in hours. The five physiological variables were recorded according to usual clinical practice. For each of these variables, the value given on the entry form was the first measurement taken at hospital admission.
We did complete case analysis, as the amount of missing data was very low in CRASH-2. We initially included all candidate predictors in the multivariable logistic regression. We adjusted analyses for treatment by including treatment allocation as a covariate in the models. We also included a variable for economic region (that is, low, middle, or high income country, as defined by the World Bank). We used logistic regression models with random intercepts by country. We initially analysed continuous variables as linear terms. We assessed departure from linearity graphically and by adding quadratic and cubic terms into the model. We specifically explored interactions by age and by type of injury. We dichotomised time since injury into less than or more than three hours, as the effect of this variable was reasonably well captured by treating it as binary.
We used a backward stepwise approach. Firstly, we included all potential prognostic factors and interaction terms that users considered plausible. These interactions included all potential predictors with type of injury, time since injury, and age. We then removed, one at a time, terms for which we found no strong evidence of an association, judged according to the P values (<0.05) from the Wald test. Each time, we calculated a log likelihood ratio test to check that the term removed did not have a big effect in the model. Eventually, we reached a model in which all terms were statistically significant. We used the R software environment (version 2.13.1; R Foundation for Statistical Computing, Vienna, Austria).
We assessed the predictive ability of the prognostic model in terms of calibration and discrimination. Calibration indicates whether observed risks agree with predicted risks; we assessed this graphically by plotting the observed outcomes versus the predicted probabilities of the outcomes. Discrimination indicates whether patients at low risk can be separated from those at high risk; we assessed this by using a concordance (C) statistic. We assessed optimism in the performance by bootstrap re-sampling. We drew 200 samples with replacement from the original data, with the same size as the original derivation data. In each bootstrap sample, we repeated the entire modelling process, including variable selection. We averaged the C statistics of those 200 models in the bootstrap samples. We then estimated the average C statistic when each of the 200 models was applied in the original sample. The difference between the two average C statistics indicated the “optimism” of the C statistic in our prognostic model.
For the external validation, we used the data from the Trauma Audit and Research Network (TARN). Membership is voluntary and includes 60% of hospitals receiving trauma patients in England and Wales and some hospitals in Europe. Data are collected on patients who arrive at hospital alive and meet any of the following criteria: death from injury at any point during admission, stay in hospital of longer than three days, need for intensive or high dependency care, need for inter-hospital transfer for specialist care.
We excluded patients with isolated closed limb injuries and those over 65 years old with isolated fractured neck of femur or pubic ramus fracture. The physiological data available in TARN are identical to those in CRASH-2, in that for every patient the heart rate, systolic blood pressure, Glasgow coma score, respiratory rate, and capillary refill time on arrival are entered by the hospital data coordinators. For each patient, the volume of blood loss is estimated. This is done by allocating an estimated percentage of total volume of blood lost to each injury code in the abbreviated injury scale dictionary by blinded, then consensus, agreement from two emergency physicians. This estimation is based on previous work on blood loss in specific injuries.
We selected adult (age over 15 years at the time of injury) patients presenting between 2000 and 2008 to hospitals participating in TARN. The definition of significant haemorrhage used in the CRASH-2 trial was not available, so we selected only patients with an estimated blood loss of at least 20%, whom we considered would be clinically comparable to the CRASH-2 patients.
For the validation in the TARN dataset, we did multiple imputations to substitute the missing values of the predictors included in the prognostic model by using the procedure of imputation by chained equations in Stata Release 11. We applied the coefficients of the model developed in CRASH-2 with the estimated UK intercept to the five imputed datasets of TARN, obtaining five predictions of mortality for each patient in TARN. We then averaged over these five predictions to calculate calibration and discrimination.
For ease of use at the point of care, we developed a simple prognostic model. For this model, we included the strongest predictors with the same quadratic and cubic terms as used in the full model, adjusting for tranexamic acid.
We presented the prognostic model as a chart that cross tabulates these predictors with each of them recoded in several categories. We made the categories by considering clinical and statistical criteria. In each cell of the chart, we estimated the risk for a person with values of each predictor at the mid-point of the predictor’s range for that cell. We then coloured the cells of the chart in four groups according to ranges of the probability of death: <6%, 6-20%, 21-50%, and >50%. We decided these cut-offs by considering feedback from the potential users of the simple prognostic model and by looking at previous publications.
Methods
Model Development
For the development of the prognostic model, we involved potential users from three settings: pre-hospital, battlefield, and emergency departments. We held meetings with paramedics, military doctors, and consultants in emergency medicine to identify variables and interactions that they considered important and convenient for their settings and to obtain information on how to present the prognostic model in a user friendly format.
We included patients from the Clinical Randomisation of an Antifibrinolytic in Significant Haemorrhage (CRASH-2) trial. The trial included 20,127 trauma patients with, or at risk of, significant bleeding, within eight hours of injury, and took place in 274 hospitals in 40 countries. The primary outcome was all cause mortality. Patients’ outcomes were recorded at discharge, at death in hospital, or 28 days after injury, whichever occurred first.
Predictors
We took variables to be analysed as potential predictors from the patients’ entry forms completed before randomisation. Variables included in the entry form for the CRASH-2 trial can be divided into patients’ demographic characteristics (age and sex), characteristics of the injury (type of injury and time from injury to randomisation), and physiological variables (Glasgow coma score, systolic blood pressure, heart rate, respiratory rate, and central capillary refill time).
Age was recorded as a continuous variable measured in years. Type of injury had three categories—penetrating, blunt, or blunt and penetrating—but we analysed it as “penetrating” or “blunt and penetrating.” Time from injury was recorded as a continuous variable measured in hours. The five physiological variables were recorded according to usual clinical practice. For each of these variables, the value given on the entry form was the first measurement taken at hospital admission.
Multivariable Analysis
We did complete case analysis, as the amount of missing data was very low in CRASH-2. We initially included all candidate predictors in the multivariable logistic regression. We adjusted analyses for treatment by including treatment allocation as a covariate in the models. We also included a variable for economic region (that is, low, middle, or high income country, as defined by the World Bank). We used logistic regression models with random intercepts by country. We initially analysed continuous variables as linear terms. We assessed departure from linearity graphically and by adding quadratic and cubic terms into the model. We specifically explored interactions by age and by type of injury. We dichotomised time since injury into less than or more than three hours, as the effect of this variable was reasonably well captured by treating it as binary.
We used a backward stepwise approach. Firstly, we included all potential prognostic factors and interaction terms that users considered plausible. These interactions included all potential predictors with type of injury, time since injury, and age. We then removed, one at a time, terms for which we found no strong evidence of an association, judged according to the P values (<0.05) from the Wald test. Each time, we calculated a log likelihood ratio test to check that the term removed did not have a big effect in the model. Eventually, we reached a model in which all terms were statistically significant. We used the R software environment (version 2.13.1; R Foundation for Statistical Computing, Vienna, Austria).
Performance
We assessed the predictive ability of the prognostic model in terms of calibration and discrimination. Calibration indicates whether observed risks agree with predicted risks; we assessed this graphically by plotting the observed outcomes versus the predicted probabilities of the outcomes. Discrimination indicates whether patients at low risk can be separated from those at high risk; we assessed this by using a concordance (C) statistic. We assessed optimism in the performance by bootstrap re-sampling. We drew 200 samples with replacement from the original data, with the same size as the original derivation data. In each bootstrap sample, we repeated the entire modelling process, including variable selection. We averaged the C statistics of those 200 models in the bootstrap samples. We then estimated the average C statistic when each of the 200 models was applied in the original sample. The difference between the two average C statistics indicated the “optimism” of the C statistic in our prognostic model.
External Validation
For the external validation, we used the data from the Trauma Audit and Research Network (TARN). Membership is voluntary and includes 60% of hospitals receiving trauma patients in England and Wales and some hospitals in Europe. Data are collected on patients who arrive at hospital alive and meet any of the following criteria: death from injury at any point during admission, stay in hospital of longer than three days, need for intensive or high dependency care, need for inter-hospital transfer for specialist care.
We excluded patients with isolated closed limb injuries and those over 65 years old with isolated fractured neck of femur or pubic ramus fracture. The physiological data available in TARN are identical to those in CRASH-2, in that for every patient the heart rate, systolic blood pressure, Glasgow coma score, respiratory rate, and capillary refill time on arrival are entered by the hospital data coordinators. For each patient, the volume of blood loss is estimated. This is done by allocating an estimated percentage of total volume of blood lost to each injury code in the abbreviated injury scale dictionary by blinded, then consensus, agreement from two emergency physicians. This estimation is based on previous work on blood loss in specific injuries.
We selected adult (age over 15 years at the time of injury) patients presenting between 2000 and 2008 to hospitals participating in TARN. The definition of significant haemorrhage used in the CRASH-2 trial was not available, so we selected only patients with an estimated blood loss of at least 20%, whom we considered would be clinically comparable to the CRASH-2 patients.
For the validation in the TARN dataset, we did multiple imputations to substitute the missing values of the predictors included in the prognostic model by using the procedure of imputation by chained equations in Stata Release 11. We applied the coefficients of the model developed in CRASH-2 with the estimated UK intercept to the five imputed datasets of TARN, obtaining five predictions of mortality for each patient in TARN. We then averaged over these five predictions to calculate calibration and discrimination.
Simple Prognostic Model
For ease of use at the point of care, we developed a simple prognostic model. For this model, we included the strongest predictors with the same quadratic and cubic terms as used in the full model, adjusting for tranexamic acid.
We presented the prognostic model as a chart that cross tabulates these predictors with each of them recoded in several categories. We made the categories by considering clinical and statistical criteria. In each cell of the chart, we estimated the risk for a person with values of each predictor at the mid-point of the predictor’s range for that cell. We then coloured the cells of the chart in four groups according to ranges of the probability of death: <6%, 6-20%, 21-50%, and >50%. We decided these cut-offs by considering feedback from the potential users of the simple prognostic model and by looking at previous publications.
