Do studies of the weekend effect really allow for differences in illness severity?

For nearly 15 years from 1997 until 2011, David Barer and his stroke team colleagues kept a prospective register of all patients admitted to hospital in Gateshead with suspected acute stroke. This was used mainly for research but also allowed independent checks to be made on the official figures from the coding department, providing useful insights into diagnostic uncertainties, the reasons for coding errors and day-to-day and year-on-year changes in the numbers and clinical characteristics of stroke admissions.  In this study he analyses whether the apparent excess mortality among patients admitted at weekends might be due to differences in stroke severity or other factors which cannot be measured in studies relying on routine administrative data.

The long-rumoured but now notorious “weekend effect” recently received the seal of scientific respectability from two huge studies, analysing routine data on 20 million hospital admissions (and 1/2 million deaths) in England and Wales. They found a 10-15% increase in the risk of dying in the first month after weekend, compared with weekday admissions, even after adjusting for differences in overall “sickness levels” by sophisticated modelling of diagnostic and administrative data.  The authors of the larger study even included non-emergency admissions, despite the obvious imbalance between weekdays and weekends, arguing that their risk model could “explain” most of the mortality variation. 

The overall 30-day mortality rate was just 1.8%.  Thus a difference in mortality rates of just under 0.2% in absolute terms, indirectly estimated by comparing completely different patient populations, became a key political issue and a central bone of contention in the junior doctors’ dispute. Although less than 5% of hospital deaths are regarded as “avoidable”, public anxiety was stoked up enough for some patients to delay seeking help, simply to avoid going to hospital at weekends.

But can these statistical models really adjust for the huge differences – 60% in overall numbers – between weekend and weekday admissions? Much of their “predictive power” simply comes from distinguishing planned from emergency admissions (although the accuracy of coding may drop at weekends), but they do have clever ways of adjusting for diagnosis and comorbidities.  They cannot allow for differences in staff and patient behaviour, however, the most obvious of which is the tendency of people with milder symptoms to wait until Monday before seeking help.

Only clinical databases are able to grade illness severity, and acute stroke, a condition with high mortality and well validated prognostic indicators, provides a good model.  It also highlights some of the diagnostic and coding difficulties with which geriatricians will be familiar.  We had collected data on nearly 15 years of acute stroke admissions in our hospital stroke register, and when we compared stroke severity and outcome between weekend and weekday admissions, a clear pattern emerged.  The in-hospital mortality rate was 13% higher for those admitted at weekends, and this excess persisted after adjusting for differences in age and comorbidities  – the sort of information available in Hospital Episode Statistics – but it was completely  eliminated after allowing for the slightly lower proportion of “mild” strokes among the weekend admissions.

Other recent stroke studies – published after the present study was submitted – have also raised serious doubts about the validity of the weekend effect, and these are discussed in an on-line appendix.

Thus the evidence for excess mortality is shaky at best – at least for stroke disease, which accounts for a high proportion of the apparent weekend effect. But assuming the effect is genuine, there is less than one excess death for every 500 patients admitted, and even making a worst case estimate that up to a quarter of these are avoidable, the best we can expect from any intervention would be to prevent one death for every 2,000 patients treated.  Obviously any such intervention would have to be rigorously evaluated in randomised trials, before being widely implemented, especially if it involved a massive upheaval in working patterns. But comparing the likely costs with the relatively modest potential benefits, what chance would such trials have of getting funding, if politics were not involved?

So how did such a small and unproven effect become a political priority, requiring big changes to be forced through without any evaluation? Could it be because the experts in charge of the huge NHS databases chose to promote their findings simply by multiplying small differences in estimated mortality rates by the enormous numbers of cases analysed, producing headlines about “thousands of avoidable deaths”?

It is unrealistic to expect politicians not to react to such headlines, so it is vital for researchers to resist the temptation to downplay the uncertainties and “sex up” their findings.  It is time to end the silly megaphone wars over death rates. Despite the traumas of mid-Staffordshire, geriatricians will understand that they are a poor way to judge quality of care.

Read the full Age & Ageing paper ‘Do studies of the weekend effect really allow for differences in illness severity? An analysis of 14 years of stroke admissions’