It goes without saying that a clinical prediction rule, and for that matter an associated online tool (one of which in particular we’ll get to in a minute) is only as good as your initial clinical judgment and as practical as the environment in which you work. These types of rules also need to be grounded in solid methodology with prospective validation.

The aforementioned rule that I alluded to was published in BMJ and discussed at the Cincinnati Children’s Hospital Medical Center Pediatric Emergency Medicine journal club recently.

You can take a look at the article here: Article Link

And the online decision tool for SBI here: Online Tool

Here are a few thoughts shared from the engaging discussion:

  • The precise number of predictor variables were not clearly defined in the methods. In addition there were so many predictors that it ran the risk of overfitting – this limits applicability to broader populations.  This fits the study it was anchored to well, but not necessarily others. There was also some heterogeneity between the study sites, and in terms of what the individual providers did. The predictors were also not clearly specified a priori.
  • You need to have a sufficient number of outcomes for each predictor, generally 10 for each. In a study with 35 predictor variables you’d need 350 outcomes for a potentially rare disease/outcome. This is hard to accrue. The predictors with poor interrater reliability, those without high numbers of positive results, and if 2 predictors are similar (℅ linear) you can start to winnow down the list.
  • Unfortunately this study had some missing data – in this study they used computer magic (multiple imputation) to run the models multiple times to try to predict what those missing variables would be. In general, if >30% of the samples are missing this technique is limited.
  • The external validation population in this study (Coventry) had a higher prevalence of SBI. Was this due to luck? And how did it effect the results?
  • There were multiple cases of SBI – but it is important to remember that the conditions they defined as SBI (pneumonia, UTI, septicemia and meningitis) are heterogeneous clinically. There were very few cases of septicemia and meningitis – the worst of the bunch.
  • In summary, this turned out to be a decent pneumonia rule but not great for others. It turns out that if you have a patient with clinical pneumonia/suspicion for pneumonia CRP will increase sensitivity to see a pneumonia on the Xray. How many of us actually get a CRP in these scenarios though?
  • And finally, procalcitonin was not included – which is interesting given that this was Europe. See the following article from Luaces-Cubells et al. for more. I’ll be posting on procalcitonin in the near future.