The Friends and Family Test is simple enough to understand. But, like the thumbs up/thumbs down test used to decide the fate of gladiators in ancient Rome, it is not subtle.
Patients have the ability to make or break a provider through answering a simple questionnaire and, like the spectators in the Colosseum, they don’t have to explain their answers.
It had been called ‘useless and meaningless’ by commissioning bodies before its launch.
But - now we have seen how it has worked in its infancy - how can commissioners use the results, especially considering concerns over the data? And what effect will it have on CCGs’ funding?
Political will
The test has strong political backing. It was announced by the Prime Minister in May 2012, and both David Cameron and health secretary Jeremy Hunt threw their weight behind rolling it out to the whole NHS in the wake of Robert Francis’s damning report into the problems at Mid Staffordshire NHS Foundation Trust - in particular, the key criticism that patient´s voices were not being heard.
Piloted in NHS Midlands and East, the test became mandatory for acute in-patients and patients discharged from A&E on 1 April this year and its rollout across all NHS services was an integral part of Putting Patients First, NHS England’s business plan for 2013/14 - 2015/16.
As part of the test, every patient is asked ’How likely are you to recommend our A&E department/ward/GP practice to friends and family if they needed similar care or treatment?’ Patients choose one of six different options, ranging from ‘extremely likely’ to ‘extremely unlikely’ and they can offer further comments if they wish to do so.
These scores are calculated to produce a ‘net promoter score’, based on the proportion of positive comments to negative comments – though those who say they are ‘likely’ to recommend are completely left out of the equation.
Early results for hospital A&E and inpatients published last month suggested patients on 36 of 4,500 hospital wards in England would not recommend them to relatives.
However, the concept of an ‘NHS list of shame’, as the national press branded it, was somewhat undermined by what even NHS England acknowledged were relatively low response rates.
The repercussions of the first results
Some CCGs, such as NHS Calderdale, have been able to bask in the reflected glory of these results.
In the first quarter of 2013/14, the two local hospitals in Calderdale achieved a net promoter score of 50 out of 100, putting the trust in what it termed the ‘excellent category’ of scores of 50 and above.
Dr Matt Walsh, chief officer of Calderdale CCG, said: ‘We are pleased with our local hospitals’ overall positive first results and look forward to seeing further improvements in care as a result of our ongoing use of the Friends and Family Test. We urge all patients to give their feedback: it really does count.’
Elsewhere, the reviews have been less glowing. East Surrey Hospital, for example, scored averagely for A&E and 90% of patients were ‘likely’ or ‘extremely likely’ to recommend the hospital. However, it still came in the bottom 20% of hospitals in England as judged by the test.
Karen Devanny, director of quality and nursing at East Surrey CCG, said: ‘The results for East Surrey Hospital show there is room for improvement and we are closely monitoring the trust’s ongoing performance.’
Most CCGs are now publishing details of the test - and the results for hospitals on their patch - on their websites and offering links for patients to continue to rate recent episodes of care.
Dr Steve Kell, chair of Bassetlaw CCG and co-chair of the NHS Clinical Commissioners leadership group, says response rates in his area so far are poor - 2% for A&E and 14% for inpatients in June. He says: ‘Bassetlaw has, with all CCGs, worked with providers to plan implementation of the Friends and Family Test. Initial uptake has been relatively low and we are discussing this with providers.’
However he also sees positive aspects of the test: ‘Commissioners welcome the test as another source of information about patient experience. We recognise its limitations and it’s important that it’s not used in isolation, but providing patients with publicly available information about local services creates a valuable opportunity for local discussions and improvements.’
The extent of variation in the way the test was administered… certainly suggests caution in interpreting the findings’
Professor Peter Lynn, professor of survey methodology at the University of Essex
Whether the benefit obtained from a ‘valuable opportunity for local discussions and improvements’ is worth the cost to the NHS of carrying out the test is another matter, but its methodological flaws effectively prevent it from being used for anything more.
Flaws
Dr Kell’s view is slightly more positive than the NHS Alliance’s position earlier in the year, when the organisation deemed the test ‘useless and meaningless’. It said: ‘The Friends and Family Test is not the mechanism we need. It is vital to ensure that patients have the opportunity to explain why their experience was good or bad.’ The NHS Alliance is working up an alternative.
National Voices, the body representing a coalition of health and social care charities, calls the test ‘a crude measure on which the NHS would be unwise to place too much reliance’.
Chief among its limitations, according to critics, is that the test does not take into account differences between organisations in types of patients, the treatments offered or the proportions of patients who respond.
Related stories
The Friends and Family Test is unfit for purpose
CCG leaders doubtful over extending Friends and Family Test to GPs
‘Friends and family test’ to be rolled out to GP practices next year
Professor Peter Lynn, professor of survey methodology at the University of Essex, believes variations in response rates and methods used to collect the data call into question whether differences in scores between trusts or between wards ‘are in any way meaningful’.
A difference in scores may reflect ‘a real difference in care standards or it may just as well reflect differences in the type of people who responded’ or in the way they interpreted the question, he adds. ‘We just don’t know. But the extent of variation in the way the test was administered, and the fact that the scores take none of the variation between trusts into account, certainly suggests caution in interpreting the findings.’
Uses
The Friends and Family Test is not without its advocates, however. Dr Shane Gordon, chief officer of NHS North East Essex CCG, believes the test has its place in the process: ‘It just provides one way of looking at services, so I have no problem with it as a tool. The crucial bit is it must not be used in isolation.’
‘The art of finding out what people think of the care they receive is still evolving,’ he says, but he cites the three strands to quality defined by Lord Darzi in his 2008 NHS Next Stage Clinical Review - patient safety, clinical effectiveness and the experience of patients.
‘We need to look at all three and patient experience measures are probably the least developed of the three, so the Friends and Family Test really helps.’
‘Whatever the drawbacks, it’s absolutely essential we start asking patients what they think,’ he adds.
The test is due to be extended to GP practices next year in a move trailed by Mr Cameron in January, but CCG leaders are sceptical.
Dr Huw Charles-Jones, chair of West Cheshire CCG, told Pulse earlier this month: ‘I do share the concerns that are being raised about the sampling methodology and whether it is an accurate way of gauging patient experience. Having said that, the fact it is being carried at all may improve “customer” service.’
He added: ‘‘The test might be simple but it is not the complete answer. I worry that politicians will get hold of it and use it to beat up NHS staff when what they actually need is more support, praise and resources.’
Dr Kell says he also recognises a risk of the test burdening practices with paperwork: ‘GP practices have a long history of obtaining patient surveys, many of which ask a similar question. We will ensure we work together as a CCG to support practices in ensuring this is meaningful and transparent but not too much of a bureaucratic burden for practices.’
Quality premium
But what makes the test - and its failings - potentially crucial for CCGs is that they will have to ‘improve or achieve high standards of quality’ as measured by the Friends and Family Test to receive their full payments as part of the controversial quality premium.
NHS England has yet to issue specific guidance on whether this includes general practice data. If it does, then CCGs’ quality premium payments could depend on the performance of GP practices over which they have no formal control as they are not responsible for commissioning primary care.
And, to compound the confusion, it is not yet clear whether CCGs will have direct access to the GP data when it starts to flow - or whether it will bypass them going straight to NHS England via local area teams.
‘CCGs’ role is not to performance-manage the GP contract but they need to be aware of practices´ performance as a member organisation’
Dr Nigel Watson, chair of the GPC commissioning subcommittee
Toby Knightley-Day, managing director of Fr3dom Health, the company behind the Friends and Family Test questions, believes CCGs must have access to the primary care data.
‘CCGs need to see the raw data on primary care as well as the services they commission and have direct responsibility for – in fact, they should demand it,’ he says.
‘If you look at the arguments around integrated services, for example, it’s important CCGs have information on all providers involved in the process – to do so, they’ll need to act as hubs for local data. Only then can CCGs identify the issues and smooth pathways.’
Dr Nigel Watson, chair of the GPC commissioning subcommittee and chief executive of Wessex LMCs, believes it would be ‘ludicrous’ if GP friends and family data were to decide CCG quality premiums. However, he believes the data will inform how CCGs manage themselves as member organisations as it is likely to form part of the developing assurance framework for CCGs, which will allow patients and the public to identify how well CCGs are performing,
He adds: ‘CCGs’ role is not to performance-manage the GP contract but they need to be aware of practices’ performance as a member organisation. The assurance framework will allow CCGs to find practices that are outliers in one way or another and to work with practices to improve things - and the general practice the Friends and Family Test will form part of that.”
Dr Michael Dixon, chair of the NHS Alliance, remains sceptical about the value of the Friends and Family Test on its own but believes it is ‘entirely reasonable’ that CCGs should manage the data from GP tests. ‘As a GP you can´t please all the people all the time. But CCGs will be best placed to decide whether the results show that a practice is not technically proficient and take the requisite steps. There´s no better person to judge a poacher than another poacher - rather than the gamekeeper.’
