DH forced back to drawing board over patient survey
By Nigel Praities
Exclusive: The Department of Health is to rework its controversial patient survey after a storm of criticism over statistical flaws in this year's version, Pulse can reveal.
The move will be seen as a major victory for the GPC, which was vocal in its criticism of the survey methodology, and insisted practices should not be penalised for poor response rates.
It also emerged this week that the DH had been warned in advance by its own advisers that low response rates could ‘skew' survey results, but chose to press ahead regardless.
A new report – authored by representatives from Ipsos MORI and Government advisers at the University of Cambridge – says problems were flagged up during the testing phase, with only a ‘modest response' seen and ‘evidence of skew'.
The evaluation, published online by BMC Family Practice but submitted to the DH in advance of the patient survey, also predicted non-responder bias was likely to be worse in deprived areas, as turned out to be the case.
‘Non-response is likely to be an issue when considering use of surveys and in relation to financial reimbursement,' the paper concluded.
The DH told Pulse next year's survey would be changed to specifically address the flaws. A spokesperson said it had asked Ipsos MORI to look at the effect of non-response, and that the 2009/10 results would be weighted for age and gender – the main factors for non-response.
‘This will assist commissioners in making accurate comparisons. However, payments to practices will continue to be based on actual responses, as this is the basis of the agreement with the GPC,' the spokesperson said.
The GPC said it felt ‘vindicated' after repeatedly calling for the survey to be redesigned.
GPC negotiator Dr Chaand Nagpaul said: ‘The paper quotes many of the issues we raised at the outset, but the Government was intransigent. The system of paying GPs on the basis of two arbitrary indicators chosen by the Government is not a sensible way of funding GPs. This survey is not fit for purpose.'
There are now hopes the criticisms revealed in the analysis will strengthen the case of the estimated 3,000 GPs who are appealing to primary care organisations for reimbursement.
A Pulse analysis in July showed general practice lost about £35m in funding from the 2008/9 patient survey, and practices in deprived areas lost an average of 25% more than those in affluent areas.
But Professor Martin Roland, author of the new study and professor of health services research at the University of Cambridge, said changes to payment thresholds agreed with the GPC remained the most significant reason for GP losses, as Pulse's analysis also suggested.
‘The response rate was about what was predicted – that is not to say very high but comparable to other surveys. The controversy arose because the way survey results related to GP pay changed when it was moved from a DES to part of the QOF.'Warnings over the patient survey
'Concerns have been raised about the ability of patients to recall their experience of access up to 6 months previously'
'Respondents are more likely to be literate and used to completing forms'
'Non-response bias can still influence the survey performance measures described in the present paper'
Length of survey
'There might be potential for reducing the length of the survey'
'Any global focus on access needs to take into account the fact that the value of access is contingent on a number of factors'
Source: BMC Family Practice, 22 August early online publicationGPC negotiator Dr Chaand Nagpaul GPC negotiator Dr Chaand Nagpaul