Hospital SMRs in Scotland

A lot has been made of the publication of Hospital Standardised Mortality Ratios (HSMRs) by the NHS in England and Wales.  These were collated and published by Dr Foster in 2009.

Dr Foster have produced a useful guide to HSMRs here. To quote from the guide:

"The HSMR is a calculation used to monitor death rates in a trust. The HSMR is based on a subset of diagnoses which give rise to 80% of in-hospital deaths. HSMRs are based on the routinely collected administrative data often known as Hospital Episode Statistics (HES), Secondary Uses Service Data (SUS) or Commissioning Datasets (CDS). The HSMR was conceived by Professor Sir Brian Jarman, director of the Dr Foster Unit at Imperial College, London.

Measuring hospital performance is complex. Dr Foster understands that complexity and is clear that HSMRs should not be used in isolation, but rather considered with a basket of other indicators that give a well rounded view of hospital quality and activity."

HSMRs were first used in 2001 and Scotland have just produced their first published set of HSMRs, proudly announcing that "Unexpected deaths fall by six per cent." This is true, to some extent, but let's look at the figures which are available from the Information and Statistics Division (ISD) of NHS Scotland.

The most recent average HSMR is 0.936, which represents a six-percent reduction from the predicted hospital mortality. This figure comes from Apr-Jun 2009, and the comparison is made to the previous period when the rate was 0.992 - this is a reduction of 0.057, or 5.7%.

However, the change from one period to another is not as informative as trends over time. These are shown below:

As you can see, there is a very strange periodicity to the HSMR over time, with peaks during winter, and troughs during summer. Comparing the most recent HSMR with the same period one year ago, the reduction is 1.1% - likely to be insignificant. Quick tip: always choose the greatest differences between time points when you are wanting to look good. I'm pretty sure that the Scottish Government wouldn't have announced in December 2008 that hospital mortality had gone up by 7%!

The reason for this variation is not clear, although there are some possible contributing factors:

1. The increased morbidity in the population during the winter is reflected in increased mortality in those admitted to hospital.
2. The contribution of junior doctor changes is not obvious, but traditionally the changeover times (when new doctors start) were February and August. With changes in training, many doctors are now on 4-month, rather than 6-month rotations, and it isn't clear why doctors starting in February would increase mortality but doctors starting in August wouldn't.

What's also interesting is that this cyclical pattern is not seen in all hospitals, and is marked in others. For example, Arbroath Hospital shows a 100% variation depending on season, suggesting that mortality above the predicted rate doubles at certain times of the year.

Of course, these are new figures for NHS Scotland, and it is possible that there are errors in the data. One would hope that NHS Scotland wouldn't publish the figures unless they were reasonably confident in their reliability.

Finally, if you're interested in the hospitals with the highest HSMRs in the most recent reporting period, look no further. All of the following hospitals have a HSMR higher than the predicted rate. In order to understand that some hospitals may have made significant improvements, the change from baseline (Oct-Dec 2006) is also given. It's important to take the cyclical variations described above into account when interpreting this latter figure.