This site is intended for health professionals only

At the heart of general practice since 1960

Read the latest issue online

The docbot will see you now

Risk models for cardiovascular disease could be misleading, finds study

Current risk prediction models for cardiovascular disease may be misleading at the individual level because of large variations in practice data, a study has found.

Analysis of records from 3.6 million patients across 392 GP practices found considerable variability in the quality of data available.

The study, by a team at University of Manchester, showed that for a QRISK3 predicted score of 10%, an individual patient could have a risk of between 7.2% and 13.7%, depending which practice they came from.

Variation between practices were so great that a ‘substantive number of patients’ would fall into a different treatment category after this was taken into account, the researchers said.

Writing in Scientific Reports, the University of Manchester team concluded that risk prediction models based on routinely collected health data perform well for populations but ‘with great uncertainty for individuals’.

Study leader Professor Tjeerd van Staa said: ‘We looked at a large number of general practices and we say a large variation in the incidence of cardiovascular disease and when we looked further that wasn’t taken into account by the QRISK model.’

The variation could be to do with how practices record data as well as other lifestyle or generic factors that are not taken into account by the model, he explained.

‘The model gives you an estimate but our findings suggest you need to take that with a pinch of salt because there could be wide variation around that number.’

QRISK alone should not form the basis of a decision around treatment because that could lead to both under or overtreatment, he added, and it is questionable whether they should be used ‘without clinical interpretation’.

There has been concern that use of risk scores may overmedicalise large proportions of the population.

Dr Samuel Finnikin, a GP in Sutton Coldfield and research fellow at the University of Birmingham, said the paper raised some valid points about the application of risk scores.

He said: ‘Risk estimates are the starting point for a conversation, not the be all and end all. But it is important that we don’t discount them due to their imperfections.

‘If we didn’t have risk calculators we would be relying on a much less accurate and reliable way of estimating risk - just using clinical judgement alone.’

He added: ‘It’s not the information that is a problem - but how we use it. We must combine the risk estimate with our clinical skills.’

Professor Azeem Majeed, head of primary care and public health at Imperial College London, agreed that risk scores need to be used alongside additional clinical interpretation.

‘Ideally, a risk score should be presented with its measure of uncertainty (using methods such as 95% confidence intervals). However, I am not aware of any clinical system that is able to do this.

‘Guidelines bodies such as NICE treat risk scores as "fixed". However, statistically, there is not much difference between a QRISK score of 9.9% (below threshold for statin treatment) and 10.1% (above threshold for statin treatment),’ he added.

Readers' comments (7)

  • Vinci Ho

    This epitomises the classical dispute between academics sitting in their computer rooms looking at statistics and grass-root GPs bleeding at frontline fighting to clear the block of patients ten minutes each everyday . You can assess the CVD risk of a patient so meticulously that 20-30 minutes would have gone on the clock . Pressing a button and going for a threshold of 10% takes two seconds . But aren’t we falling into the same trap of prescribing antidepressants too liberally because we ran out of time .
    Yes , without a calculation tool like QRISK( whatever version) , relevant cases could be lost . Primary prevention of CVD with statin remains a contentious issue , whether protagonists like it or not. More innovative ideology is welcome.

    Unsuitable or offensive? Report this comment

  • Ivan Benett

    This is no different to any risk assessment based on population studies. In other words there are always confidence intervals that always need to be applied to an individual. If you’re using risk estimates this should be part of the explanation. The key sentence is that the risk model performs well for populations. This is not surprising since they are based on population studies. An individual's risk is X +\- Y. More importantly is the discussion about what they can do to reduce their risk

    Unsuitable or offensive? Report this comment

  • Obvs no pharma funding of this study or the Farr Institute!!

    Unsuitable or offensive? Report this comment

  • I was taught in Medical school that I was not becoming a Doctor to treat numbers... NICE insists in treating numbers... where do I stand?

    Unsuitable or offensive? Report this comment

  • http://chd.bestsciencemedicine.com/calc2.html

    http://www.thennt.com/

    Population health is so different from advising the patient in front of you. Most GPs vastly overestimate the risk-lowering impact of statins etc.

    Unsuitable or offensive? Report this comment

  • Anyone who has used these risk calculators in Practice for very long will have seen how useless they are. Most events occur in people with not particularly unusual risk values; the small number of people at very high risk often don't have problems, and are reluctant to take drugs or significantly alter their lifestyle. Another example of ivory tower irrelevance.

    Unsuitable or offensive? Report this comment

  • Ivan Benett

    Tired person | Salaried GP21 Aug 2019 3:32pm "I was taught in Medical school that I was not becoming a Doctor to treat numbers... NICE insists in treating numbers... where do I stand?"

    Well I would hope that when you became a post-graduate your GP trainer would introduce you to the concept of risk. It is crucial to managing an individual, when deciding with them what needs to be done to reduce the risk of any particular disease.
    In finding out their risk, NICE and others look at populations to determine which factors and how heavily those factors influence risk. Some people and institutions have quantified those risks so that you can discuss them with your patient. As these values are taken from statistical analyses of population studies they have with them confidence intervals.
    For example, a man with a PSA of 4ug/l have about 30% risk of prostate cancer. This risk is probably higher if you have a family history or symptoms. Its probably less if you have a normal prostate on DRE. Then you and the individual can make a decision about what to do next.
    It is rare that for individuals any intervention or test is 100%, there are always levels of uncertainty and tolerances. We need to understand this and then need to be taught how to present all this in plain language to our patients.
    No wonder you're tired if you are trying to explain to people that things are black and white.

    Unsuitable or offensive? Report this comment

Have your say