Measuring quality with a blunderbuss

The thinking behind it was reasonable, but the QOF is fraught with confounding variables and open to abuse from practices themselves, writes Jobbing Doctor

It is a little like a blunderbuss.

A blunderbuss was an old-fashioned gun whose only virtue was that the lead shot was spread so far and so wide that some of it actually hit the target. A good tool for a poor marksman.

Over the last six years we have had a system called QOF (Quality and Outcomes Framework) whose role, ostensibly, was to drive up quality by focussing attention on detail of managing patients. It has always been an inexact tool, although the thinking behind it was reasonable. In the best hands it can be a useful guide to some of the quality delivered in primary care. However, in the hands of the Department of Health, it has become an increasingly significant way to measure what they see as ‘quality' in primary care.

It is a fairly poor marker for this, for two principal reasons - firstly, it is a bad way to measure quality because there are too many confounding variables, and secondly it is subject to abuse from the practices themselves.

As a practice in a deprived area, we always suffer from the fact that we have an unhealthy population. Poverty and health are inextricably linked. If you are poor you get ill; if you get ill you become poor. I have lost count of the number of people who have registered with my practice because they have lost their jobs due to illness. No job, so can't afford to live in a ‘leafy' area, and so they move to my patch.

This has always been a problem, that was highlighted in one of the earliest ‘performance-related' measures in primary care - cervical cytology rates. Poor people are less likely to have smears; it is always a struggle to get them to undergo what is, after all, a rather undignified procedure. In wealthier areas, there has never been this problem. Now this continues to be a problem with many of the measures used in the QOF.

However, this pales into insignificance when compared to the other issue, and that is ‘gaming' with the QOF. I have no evidence for this assertion except that which is anecdotal. So I will consider an anecdote.

Around four years ago, a PCT called me in to consider a governance issue about the death of a patient of a particular GP. I had little concrete evidence about this doctor, except that his clinical care had been regularly called into question at the PCT and they wanted to know if there was a prima facie case to investigate the case for negligence. So, as an educator, I asked to see the case concerned, and other information about the practice (complaints, demography, other data including QOF).

Without going into too much detail, the clinical case was a barn-door mess-up, and was subsequently investigated. There were, interestingly, very few complaints, but those that were seemed pretty serious; and the doctor had achieved maximum QOF points.

Yes, the maximum.

I pointed out this to the relevant PCT, and they said that it was the best measure of quality that they had. I told them that I found the attainment of maximum points to be scarcely believable in the light of other evidence. They shrugged their shoulders.

Take blood pressure. I'm willing to put a wager on the fact that some practices will have a glut of BPs of 138/78 in March, and the rest of the year the BP increases.

PCTs should be wary of using QoF as a quality marker.

What matters, you can't always count. What you can count doesn't always matter.

The Jobbing Doctor is a general practitioner in a deprived urban area of England.