This site is intended for health professionals only

At the heart of general practice since 1960

CQC reviewing practices' 'risk ratings' data as inaccuracies emerge

Exclusive The CQC is in talks with NHS England and the GPC around reviewing the data used by the regulator for its ‘risk ratings’ of GP practices, as GPs claim that the data used is inaccurate, Pulse can reveal.

Pulse has uncovered claims that practices were being marked down as being ‘risky’ because of problems with the interpretation of data, including a failure to acknowledge practices had stopped doing QOF work on NHS England advice, calculations being based on practice lists of zero patients and problems with the GP Patient Survey.

The CQC has said it is reviewing the indicators used alongside NHS England to see how they can be ‘refined’, while the GPC has said the ratings are ‘not fit for purpose’, and said it is working with the regulator to see whether they should continue in their current state.

The review by the three bodies comes after the inspection body has been under fire for its decision to publish GP risk ratings, which are used to prioritise practices for inspection, despite warnings from GP leaders that the data should be kept as an internal tool.

The regulator uses data from practices’ QOF performance, the GP Patient Survey, and other sources to highlight potential areas where there is a risk of poor care. Based on their scores in these 38 indicators, they are giving an overall rating of 1 - meaning very risky - to 6, the lowest risk rating.

But a spokesperson for the CQC has told Pulse that it had ‘concerns’ that they were being seen as ‘judgements’, and it was looking into how the indicators can be refined.

It comes as Pulse has heard multiple complaints about the validity of the CQC data, including:

  • One LMC writing to the chief inspector of general practice Professor Steve Field stating that saying the ratings are punishing their practices for ditching QOF indicators on NHS England advice;
  • The practice of a former LMC leader being labelled as a high priority for inspection because of their red ‘elevated risk’ rating for unplanned admissions, which was based on ‘statistically impossible’ data;
  • A practice having their patient survey scores were reduced by 34% because they had a high proportion of respondents who said the question ‘doesn’t apply’ to them.

It is the latest blow to the ratings system, launched by the CQC last week, which was immediately hit with a backlash from GPs astonished at how they could be given a risk rating without even being visited by the CQC, and the situation became ever more incendiary when the Government said it would use the risk scores on its new TripAdvisor-style NHS comparison website aimed at patients.

A CQC spokesperson said: ‘We have had many responses to the first publication of our intelligent monitoring data for general practice. A concern raised with us is that the bandings are being reported as judgements and ratings. We share this concern – the bandings help prioritise practices for inspection and are not a judgement about care quality. Judgement and ratings only come after inspection, drawing upon what we see and hear as well as the available data.

‘Intelligent monitoring clearly shows that the vast majority of practices are of low concern based on the available data. We are also pleased to announce we will be publishing a further batch of inspector reports early next week that show GP services delivering excellent care.

‘We are currently reviewing all feedback we have received and together with NHS England looking at how we can further refine indicators. Any changes we make will be reflected on our website and any practice whose banding changes such that they become a priority for inspection will be notified in advance.’

GPC deputy chair Dr Richard Vautrey told Pulse: ‘We [the GPC and CQC] have also agreed to set up a group together with the RCGP to review both the data being used and how it is used.

‘The intention is to have a fundamental look at what could be used by CQC rather than the current indicators and ranking which is not fit for purpose.’

The concerns about the inaccuracies were raised by practices after they realised they were being marked down for inaccurate data.

Dr Mark Sanford-Wood, medical secretary of Devon LMC – who wrote to Professor Field – said the CQC was ‘unfairly flagging many of those practices as being high risk, when they are nothing of the sort’.

He claims practices in Devon and Cornwall have found themselves marked down for their low QOF scores towards the end of the 2013-14 financial year, despite NHS England signing off a CCG-led initiative for them to stop reporting on QOF indicators that were scrapped under this year’s GP contract.

This is in in contrast to neighbouring Somerset, where practices were not given risk ratings having signed up to a local incentive scheme from January.

Dr Sanford-Wood wrote: ‘Sadly, your methodology is now unfairly flagging many of those practices as being high risk, when they are nothing of the sort. You are clearly not aware of the agreement reached in Devon otherwise I am sure you would have compensated for this, but it remains of great concern that a national programme such as this could be launched when so badly flawed by something so obvious.’

Former Sefton LMC chair Dr Andrew Mimnagh said that data for his practice showed a ‘statistically impossible’ level of accident and emergency admissions.

He said: ‘The partners are looking at how our 6,800 patients managed to rack up 150,000 AED attendances, given we have only 11 frequent flyers with more than 10 AED attendances per annum.

‘We’d have to have someone who’s basically living there, being readmitted every day… we’ve been monitoring the frequent fliers and attenders and it’s just statistically impossible to be racking up that level of attendances.’

Matthew Epton, a GP practice manager in Oxford claims his practice was marked down because of supposed poor scores from the GP Patient Survey, which marked ‘not applicable’ responses as negative responses.

For example, the CQC said that only 60% of 234 patients said that the last nurse they saw or spoke to was good at giving them enough time, he said.

However, only one of these patients had marked the practice down as poor, eight as ‘neither good nor poor’, while 84 had said the question was not applicable.

Mr Epton explained: ‘The 84 patients that have clicked doesn’t apply probably did so because they didn’t see a nurse on that visit or have never seen a nurse. The correct result should have been 94%. This is a massive difference and makes our practice look relatively poor and damages our reputation.’

The GPC, who met with the CQC after seeking legal advice on the damage caused to practices’ reputations, have also insisted the regulator must work with the media, and particularly local press, to counter erroneous reports that practices are putting patients at risk.

Readers' comments (26)

  • Vinci Ho

    So much one can defend SF , the fact is the final risk banding was published by CQC. It has to be responsible for the consequence(s). This is the same like when we signed prescriptions even though they were printed by our receptionists .

    Unsuitable or offensive? Report this comment

  • Siraj Shah

    I hope Prof Field and his frantic squad of inspectors will realize that assigning practices to risk bands based on QOF / survey data and publishing these results was the most irresponsible and foolish exercise for the following reasons:
    1. Any data that is open to manipulation like QOF should be interpreted with great caution.
    2. GPs have no control over so many variables like – poor patient compliance, adverse drug effects, ethnic, cultural and linguistic barriers that affect reaching most of the QOF targets.
    3. Among black and minority ethnic groups cervical smear uptake is declining reaching a ten-year low to 78.6% despite wide publicity on the need for cervical screening after death of celebrity Jade Goody, age 27, from cervical cancer in 2009. Any practice with a significant proportion of women from such groups as in our practice in addition to its persistent defaulters will not be able to achieve cervical smear target. Assigning such practices to a high risk category based on flawed “Intelligent monitoring” data not supported by intelligent reporting is absurd. To add insult to the injury those practices who may not be aware that exception reporting Read codes 9NiT (DNA) and 9085 (defaulters) were removed from cytology business rules in 2011, risk being categorized into high risk by this “unintelligent monitoring” tool.
    4. Target levels of HbA1c in Type 1 diabetes are almost impossible to achieve - (list size 2700: total number of type 1 & type II: 152) - 99 % of our Type 1 diabetic patients are under consultant care and have HbA1c levels well above the target range of 64 mmols.
    5. Risk profiling practices based on a RETIRED controversial QOF indicator like ACR flies on the face of “intelligent monitoring” tool adopted by the CQC.
    The explanation given by the CQC that 1-6 scale is merely to prioritize practices for inspection does not hold water: CQC published their inspection report showing we had met all essential standards and our practice was fully compliant with all regulations on 8 November 2014 yet just 11 days later CQC published their intelligent monitoring data placing our practice in band 1!
    How on the earth can CQC justify placing our practice in band 1 for inspection based on this so called intelligence monitoring reporting tool that shows:
    Effectiveness domain (28 indicators) our practice is better than average in reducing emergency admissions, prescribing NSAID’s, palliative care support, multidisciplinary case reviews, and learning disabilities and there was no evidence of risk in 11 other indicators.
    Caring domain: (7 indicators) – no evidence of risk in all 7 indicators
    Responsive domain: (3 indicators) – no evidence of in all 3 indicators
    I feel the CQC has no option but to remove this highly damaging inaccurate information and apologize publically for bringing hard working GP practices into disrepute and undermining patient confidence in their GP’s.

    Unsuitable or offensive? Report this comment

  • This comment has been moderated.

  • the stasi of cqc should apologise ,retract with a national publicity campaign refuting the misuse of the highly UNintelligent data they were so stupid to release publically .they have done a lot of damage.they are incompetent bullies.in a phrase unfit for purpose.
    if field knew about this he should hang his head in shame.

    Unsuitable or offensive? Report this comment

  • Steve Field practices at Bellevue Medical Centre http://www.bellevuemedicalcentre.co.uk/staff1.aspx?t=1
    14 doctors with 7688 patients, see the PPG report http://www.bellevuemedicalcentre.co.uk/ppg.aspx
    It appears that in the 2012/13 PPG report the following important issues were raised:
    1) The PPG wanted to understand the appointment system further, and why there was recently a struggle to book appointments in advance, particularly with a doctor of choice.

    2) The PPG also mentioned the attitude and work ethic of the Reception team - ‘taking their time’ and not always providing a friendly service.

    3) Struggling to get through on the phone and can wait up to 20minutes before a receptionist picks up. Equally find the pre-recorded messages frustrating.

    Then again in the survey report 2013/14: http://www.bellevuemedicalcentre.co.uk/surveyreport.aspx?p=M85124 there are a few problems with making things add up to 100%
    Key Discussion points raised by the PPG at 27 February 2014 meeting:

    Can you book an appointment in advance?
    Every time 19%
    More often than not 16%
    Not very often 12%
    Never 1%
    Not collated 2%

    Are you able to book an urgent appointment to be seen on the same day when clinically necessary?
    Every time 16%
    More often than not 18%
    Not very often 12%
    Never 3%
    Not collated 1%
    And at the end of this report we get this little gem:
    7 day working week will be enforced (nationally) in the near future - whereby practices will open over the weekend to allow more appointment availability. PPG discussed closing the practice list and turning away new patients in an attempt to have more bookable appointments for current patients – currently not an option as we are contractually required to keep list open to all.

    Well, with 14 doctors to chip in...

    Unsuitable or offensive? Report this comment

  • and the prescription fee Steve is £8.05 these days, not £7.65 , do keep up to date!
    http://www.bellevuemedicalcentre.co.uk/prescriptions1.aspx?t=1
    and I note that you dont allow patients to phone in requests for repeat prescriptions?
    "Please note we do not accept requests for medication by fax or telephone unless you are housebound and only after prior agreement with one of the partners."
    now that's what I call quality!

    Unsuitable or offensive? Report this comment

  • This has just been flagged up on the midday news but how many people will know what an incompetent bunch is monitoring their practice ...the damage is done once the mis-information is out there - revisions won't necessarily be trusted .....

    Unsuitable or offensive? Report this comment

View results 10 results per page20 results per page50 results per page

Have your say