Evaluating Screening Tests
Is this test any good? - Evaluating diagnostic and screening tests
J Irlam
Primary Health Care Directorate
UCT Faculty of Health Sciences
jirlam@cormack.uct.ac.za
Clinical scenario:
Is rapid testing for childhood influenza necessary?
This past winter you saw a lot of children in your primary care practice with influenza like illnesses (ILI) characterised by fever, dry cough and sore throat.
Laboratory tests are not generally useful in the diagnosis of influenza, but you are asked to investigate whether the practice should perform rapid diagnostic testing during the consultation to reassure concerned parents.
What is diagnosis?
The interpretation of a patient’s history, physical examination, and clinical tests to yield a working hypothesis about the most likely condition or disease in the patient.
When is a diagnostic test useful?
- Diagnostic tests may help refine your hypothesis about a disease /condition.
Only useful if they:
- Modify the (pre-test) probability sufficiently to affect clinical management in a way that will improve outcomes.
- Help you to differentiate between competing diagnoses.
- Are applicable to a broad spectrum of patients.
- Are easily reproducible.
1. Ask the right question:
2. Access the evidence:
MEDLINE (PubMed Clinical Queries: Diagnosis): influenza children
primary care - yields 17 results, the most relevant of which is
3. Appraise the evidence - 1
Is the study design appropriate for evaluating diagnostic tests?
- Do we have: cross-sectional (validation / validity) studies with a well-defined representative sample and an independent, blind comparison of the test against a reference test in all subjects within the same time period?
3. Appraise the evidence – 2
Is the study valid?
Check for the following:
- Is there diagnostic uncertainty about this condition?
- Has the test been evaluated in populations with different demographics (such as.. age, social class, race) and disease profiles (for example, symptoms, severity of disease)?
- Was there a blind comparison of the test with an independent reference test (‘gold standard’)?
- Were both tests performed on all participants?
How good is the test?
- What is the sensitivity and specificity of the test?
- What are the predictive values (PPV and NPV) of the test?
- What is the overall accuracy of the test?
- What are the likelihood ratios (LR+ and LR-) of the test?
Sensitivity and specificity
Sensitivity (Sn) = a/(a+c)
Proportion of people with the target disorder/ disease who have a positive test result.
Specificity (Sp) = d/(b+d)
Proportion of people without the target disorder / disease who have a negative test result.
- Give proportions of those with or without disease who will be correctly identified by test.
- Useful for groups of people e.g. in public health screening,
- BUT little intuitive clinical usefulness for individuals.
- No test has perfect sensitivity and specificity.
- More sensitivity usually means less specificity.
- Appropriateness depends on purpose of test and consequences of results.
- In general, a good screening test should be highly sensitive (Sn) to pick up all possible cases.
- can then “rule out” diagnosis if result is Negative (SnNout).
- In general, a good confirmatory diagnostic test should be highly specific (Sp) to exclude false positive cases
- can then “rule in” diagnosis if result is Positive (SpPin)
Predictive values
Positive predictive value (PPV) = a/(a+b)
Proportion of people with a positive test who have the target disorder.
Negative predictive value (NPV) = d/(c+d)
Proportion of people with a negative test who are free of the target
disorder.
- Gives probability of disease/condition in individuals with a positive or negative test result
- BUT predictive values vary with prevalence of disease in the population being tested:
- As prevalence increases, PPV increases, NPV decreases.
- As prevalence decreases, PPV decreases, NPV increases
How good is the test?
Example: Rapid testing for childhood influenza:
A total of 157 children with ILI were tested for influenza by means of the rapid diagnostic test (NPT) and a laboratory assay test (RT-PCR).
30 children were positive on the rapid test, a finding that was confirmed by the lab test in 27 children.
A total of 61 children were positive on the lab test. (Harnden et al 2003)
Compare the performance of the rapid diagnostic test to the laboratory test:
| |
Lab test + |
Lab test - |
TOTAL |
| Rapid test + |
|
|
|
| Rapid test - |
|
|
|
| TOTAL |
|
|
|
- Prevalence of influenza (according to RT-PCR) =
- Sensitivity (true positive rate) =
- Specificity (true negative rate) =
- Positive predictive value =
- Negative predictive value =
- Accuracy (true positives + true negatives)/ total subjects =
Likelihood ratios (LR):
This is a more clinically useful measure of the usefulness of a test. It is defined as
LR+: The likelihood (probability) that a given test result (positive or negative) would be expected in a patient with the target disorder
compared withLR-: the likelihood (probability) that this same result would be expected in a patient without the target disorder."
LRs allow one to calculate post-test probabilities of disease for a positive or negative test result in the following way:
Positive result: Post-test probability (odds) of disease = Pre-test probability (odds) x LR+
Negative result: Post-test probability (odds) of disease = Pre-test probability (odds) x LR-
LRs are independent of the prevalence of the disease in the population, unlike predictive values.
4. Apply the evidence:
- Is the test applicable to my patients?
- Is the test accurate enough to be useful?
- Is the test reproducible?
- Is the test acceptable to my patients?
- Does the test have any harmful side-effects?
- Is the test affordable?
|
 |
Using the Nomogram to evaluate screening tests:
Exercise:
(Centre for Evidence-Based Medicine (www.cebm.net/likelihood_ratios.asp)
