PHM Glossary: F

False Positives

False positives are members identified as having a condition who do not, in fact, have the condition that they are identified with.

DETAILED EXPLANATION

Developing a claims-based methodology that identifies appropriate patients for inclusion in a population health management program can be challenging because of the inherent complexity and diversity of claims data. The first step is to decide which codes will be used to select the conditions of interest. As indicated in the chronic condition definition, claims-identification codes might be ICD-9-CM diagnostic codes, CPT-4 procedure codes, HCPCS Level II codes, and NCD drug codes. Once the appropriate codes are determined, the search for these codes in the claims is initiated.

One critical issue is that of false positives (i.e., patients who do not have the condition of interest). For example, a patient visits his/her primary care physician with complaints of chest pain and the physician refers the patient to a cardiologist. The primary care physician’s professional claim may reasonably state “rule-out ischemic heart disease (IHD),” but correct diagnostic coding convention makes no distinction between diagnoses and rule-out diagnoses. Moreover, if subsequent visits reveal that the patient had an esophageal problem rather than angina pectoris (a manifestation of IHD), the IHD diagnosis will still be present on the primary care physician’s original claim. Unless corrective measures are taken, a claims analyst, given the task of identifying all patients with IHD, will select this patient based on the false, provisional diagnosis.

There is a related but different issue of “statistical false positives.” A statistical false positive is an individual who is identified with the disease in one period but who does not meet the criteria for identification in the following period. This may occur because the criterion is set narrowly (to ensure specificity) or it may occur because the individual has obtained medical treatment outside the system whose claims are being investigated (for example, by obtaining drugs from the Veterans Affairs system) or because the initial identification was a clinical false positive. The following table illustrates the problem of false positive identification using different claims-based rule sets:

[INSERT FALSE POSITIVES, IMAGE 1]

  • Narrow: A set of criteria requiring primary diagnosis on a hospital claim
  • Broad: A set of criteria using all professional claims
  • +Rx: Narrow definition plus prescription drug claim identification

In the example above, in some instances, there is a reasonably high likelihood of a member not meeting the same set of identifying conditions in the second year, despite having met those conditions in the first year. For example, only 75.9% of members who meet the narrow definition in Year 1 also meet the definition in Year 2. Conversely, 92.6% of members who meet the broadest definition in Year 1 also meet the same definition in Year 2. Note that the table addresses statistical false positives only. Some statistical false negatives may be clinical true positives. Using broad identification criteria of clinical at the cost increases sensitivity (more statistical true positives are identified). Although intuitively one might expect that a narrower set of criteria would reduce the incidence of statistical false positives, this is not the case in this example because a high percentage of members do not have an inpatient admission in the following year.

To reduce the number of false positives, two methods may be employed. Both methods target the problem of false positive diagnoses. Note that when procedures/services are used to identify patients, there is reasonable assurance that a procedure mentioned on a claim is a procedure performed. When drug prescription fills are used to identify patients, the claim is general confirmation that the prescription was filled. Note, however, that a prescription fill may have been a one-time trial and does not ensure continued use, so it is problematic to define the existence of a particular diagnosis or condition (or even use of the drug itself) on the basis of a single drug fill.

The first method is to ignore any diagnosis that appears as a test-only claim. A test-only claim is a claim that has one or more laboratory tests but no visits and no procedures/services other than tests. Diagnoses on a test-only claim are inherently questionable because they may well reflect what a physician thinks could be diagnosed by performing the tests, rather than what is really present.

The second method is to require more than one appearance of a diagnosis belonging to a condition of interest and to make sure that the two (or more) occurrences are separated by a reasonable number of days. The occurrences should be scattered over time to allow ample time for diagnostic work-ups involving multiple tests and referrals. For example, a rule-out lung cancer diagnosis could easily appear several times within a 1- or 2-week period, when lung cancer was being ruled out and the diagnosis appeared on claims submitted by the primary care physician, the radiologist (for imaging), the surgeon (to perform a biopsy), and the pathologist (to interpret the biopsied material). Note that different conditions may best be handled by different occurrences and by different time gaps. Typical numbers of mentions are two or three, and typical time gaps are 7, 15, or 30 days. This requirement of multiple mentions applies to diagnoses appearing on professional and non-inpatient facility claims. For inpatient claims, a single mention of a diagnosis may be sufficient. If the inpatient facility claims distinguish discharge diagnoses from other diagnoses, discharge diagnoses should be used. Population health management organizations will need to determine their tolerance for false positives based on the specific nature of their program and implement methods such as those listed above as appropriate.