Determination of the Optimal Cutoff Value for a Serological Assay: an Example Using the Johne's Absorbed EIA
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
JOURNAL OF CLINICAL MICROBIOLOGY, May 1993, p. 1256-1261 Vol. 31, No. 5
0095-1137/93/051256-06$02.00/0
Copyright ©) 1993, American Society for Microbiology
Determination of the Optimal Cutoff Value for a Serological
Assay: an Example Using the Johne's Absorbed EIA
SALLY E. RIDGE`* AND ANDREW L. VIZARD2
Victorian Institute of Animal Science, Department ofAgriculture, Mickleham Road, Attwood, Victoria 3049,1
and Veterinary Clinical Centre, University ofMelbourne, Werribee, Victoria 3030,2 Australia
Received 24 September 1992/Accepted 10 February 1993
Traditionally, in order to improve diagnostic accuracy, existing tests have been replaced with newly
developed diagnostic tests with superior sensitivity and specificity. However, it is possible to improve existing
Downloaded from http://jcm.asm.org/ on February 9, 2021 by guest
tests by altering the cutoff value chosen to distinguish infected individuals from uninfected individuals. This
paper uses data obtained from an investigation of the operating characteristics of the Johne's Absorbed EIA
to demonstrate a method of determining a preferred cutoff value from several potentially useful cutoff settings.
A method of determining the financial gain from using the preferred rather than the current cutoff value and
a decision analysis method to assist in determining the optimal cutoff value when critical population parameters
are not known with certainty are demonstrated. The results of this study indicate that the currently
recommended cutoff value for the Johne's Absorbed EIA is only close to optimal when the disease prevalence
is very low and false-positive test results are deemed to be very costly. In other situations, there were
considerable financial advantages to using cutoff values calculated to maximize the benefit of testing. It is
probable that the current cutoff values for other diagnostic tests may not be the most appropriate for every
testing situation. This paper offers methods for identifying the cutoff value that maximizes the benefit of
medical and veterinary diagnostic tests.
No diagnostic test exhibits perfect operating characteris- testing and in all cases there may be opportunity to improve
tics under all conditions. The traditional approach to im- the test by changing the cutoff value.
prove the accuracy of diagnosis is to develop new tests with In this study we describe a method to optimize the
superior sensitivity and specificity. An alternative, and operating characteristics of a serological test by (i) estimat-
rarely used approach, is to improve existing tests by altering ing cutoff values that maximize the economic benefit of using
the cutoff value that distinguishes infected individuals from the test for various probabilities of disease, (ii) estimating the
uninfected individuals. cost of continued use of the currently recommended cutoff
For any given serological test, sensitivity and specificity compared with the cutoffs that maximize the economic
are determined by the cutoff value. Ideally, the cutoff value benefit of testing as determined in (i), and (iii) describing a
of a test should be chosen to maximize the benefit that method to estimate the cutoff value of a test which maxi-
accrues from testing a population. This is equally true in mizes the expected monetary value of testing when the
human and veterinary medicine. To determine the benefit of probability of disease is not known with certainty. The
testing, the economic and social consequences of misdiag- serological test that is used as an example is an ELISA for
noses and the prevalence of the disease in the population the diagnosis of paratuberculosis in cattle, but the method-
must be considered. Invariably, cutoff values for diagnostic ology is suggested as being applicable to many medical and
tests have been determined by arbitrary methods that fail to veterinary diagnostic tests.
consider these issues. For example, a survey (by title) of
papers published during 1991 in the Journal of Clinical
Microbiology revealed 21 articles describing the develop- MATERIALS AND METHODS
ment, evaluation, or improvement of enzyme-linked immu- Paratuberculosis is a chronic granulomatous enteritis of
noassays (ELISA). In 12 (6, 8, 10, 12-14, 19, 20, 23, 24, 29, ruminants, caused by Mycobacterium paratuberculosis. The
32) of the 21 articles, the cutoff value was based on an disease has worldwide distribution and causes considerable
arbitrary statistic, such as 3 standard deviations above the production losses due to both clinical and subclinical dis-
mean of the negative controls. Four papers (7, 17, 26, 30) ease. Control and eradication of the disease from infected
used an optical density (OD) value as the cutoff value farms and regions are severely hindered by the inability of all
without any reference to how it was derived. One article (21) current diagnostic tests to accurately identify subclinically
chose a cutoff value to optimize both the positive and infected animals (2).
negative predictive value of the test result without consider- An ELISA (the Johne's Absorbed EIA [Commonwealth
ing the effect of disease prevalence on predictive values, and Serum Laboratories, Parkville, Victoria]) has been recently
four (3, 9, 11, 28) used a receiver operating characteristic developed for the diagnosis of M. paratuberculosis infection
(ROC) curve or some other method to optimize sensitivity in cattle (5). The cutoff value for the test that is recom-
and/or specificity. mended is 0.1 absorbance unit above the mean OD value for
In short, none of the 21 papers demonstrated that the a negative control serum, tested in duplicate, on the same
chosen cutoff value was likely to maximize the benefit of plate as the test samples. No justification or supporting
evidence was advanced for the selection of this value.
Animals. Serum samples from three groups of cattle were
*
Corresponding author. used in this study.
1256VOL. 31, 1993 DETERMINATION OF OPTIMAL CUTOFF FOR SEROLOGICAL ASSAY 1257
The first group of 136 samples were from animals slaugh- of 16 arbitrarily selected cutoff values (0.05, 0.06, 0.07, 0.08,
tered under the Victorian Cattle Compensation Scheme 0.09, 0.10, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.19, 0.22,
between 1986 and 1990 because they exhibited clinical signs and 0.25 absorbance units). The benefit of testing was
consistent with paratuberculosis and for which the diagnosis calculated for each of these cutoff values for a given preva-
was confirmed by histopathological examination of ileocecal lence, proportion of infected animals displaying clinical signs
tissues collected at slaughter. The samples were collected of disease, and relative cost of misdiagnosis (r). The cutoff
during the clinical phase of disease in these animals. value that minimized the above equation was identified and
The second group of sera were collected at slaughter from defined as the preferred cutoff value for that prevalence,
998 Western Australian cattle. While there were no fecal proportion of infected animals displaying clinical signs, and
culture or histopathological details available for these ani- relative cost of misdiagnosis. Calculations were repeated for
mals, the local animal health authorities in Western Australia 1,064 combinations of prevalence and r.
believe the State to be free of paratuberculosis and have Estimating the additional benefit of using the preferred
maintained surveillance and an active program to prevent cutoff value instead of the currently recommended cutoff
the introduction or spread of the disease. On this basis, these value. The additional benefit from using the preferred cutoff
Downloaded from http://jcm.asm.org/ on February 9, 2021 by guest
animals were regarded as paratuberculosis free. value rather than the currently recommended cutoff value
A further 164 serum samples were obtained from subclin- was estimated by subtracting the absolute benefit of testing
ically infected cattle: 48 from an Australian collection and 100 cows by using the currently recommended cutoff value
116 from samples collected as part of the U.S. National from the absolute benefit of testing the same herd by using
Repository for Paratuberculosis Specimens. A detailed de- the preferred cutoff value.
scription of these cattle, the determination of their disease Absolute benefits were determined by assigning -$80.00
status, and the methods of sample collection have been as the value of a false-positive result. This represents the
described in earlier papers (22, 25). Briefly, the Australian cost of replacement of the animal minus the average price
samples were collected from animals in three endemically paid for animals with subclinical M. paratuberculosis infec-
paratuberculosis-infected dairy herds. The results of fecal tion under the Victorian Cattle Compensation Scheme in
culture, complement fixation test, and in some cases post- 1990 to 1991. The value of a false-negative result was
mortem histopathology were available for individual ani- determined by multiplying -$80.00 by r. These figure were
mals. Samples from animals that had remained in the herd incorporated in the algorithm described above. Calculations
that showed no clinical signs of paratuberculosis but had a were repeated for 1,064 combinations of prevalence and
positive fecal culture and/or histopathological or bacterio- relative cost of misdiagnosis.
logical evidence of infection on necropsy were used in this Estimation of the preferred cutoff value when the probabil-
study. Subsamples of those held in the U.S. National Re- ity of disease is not known with certainty. The calculations
pository were included in this study if the cattle were given above assume that the probability of infection (preva-
clinically normal at the time of sampling but M. paratuber- lence) is well defined. The "best-option" cutoff values for
culosis had been cultured from their feces by either conven- the test were estimated by using decision analysis (18) for
tional or radiometric culture methods. Vaccinated cattle two circumstances in which the probability of infection was
were excluded. not known with certainty. The first situation was a herd of
Johne's Absorbed ELA. Assays were performed by using cattle for which the probability of infection was expected to
the commercial kits according to the manufacturer's direc- be low but was unknown. The second situation was a herd
tions. Negative, low-positive, and high-positive control sera for which the prevalence of infection was expected to be
supplied with each kit were included on every plate. Samples high but was undefined. A decision tree was mapped for each
were tested in duplicate. Those with greater than 30% example using five cutoff values and a false-negative/false-
discrepancy between wells or that had an OD value greater positive ratio of 1.0. Disease prevalence and probability
than 0.1 were repeated in duplicate. The OD at 450 nm values were selected to reflect the situation described.
(OD450) values for each sample were obtained, and the mean Monetary values were calculated for each cutoff value by
value of the replicated samples was calculated. using the method described above.
Estimation of cutoff values that maximize the benefit of
testing. Preferred cutoff values were estimated by the RESULTS
method described by Anderson (1) in which the preferred
cutoff value of a test is the value of c that minimizes Estimation of cutoff values that maximize the benefit of
testing. The preferred cutoff values for 1,064 combinations of
(1-P) fc' d, (x) d-x + rp f_ -.c d2(x &9c prevalence and relative cost of misdiagnosis are displayed as
where r is the cost of a false-negative result/the cost of a a decision surface in Fig. 1. Only 5 of the 16 candidate cutoff
false-positive result,p is disease prevalence, d1 is the density values (OD450 values of 0.05, 0.06, 0.07, 0.13, and 0.25) were
distribution of test values from uninfected individuals, and identified as preferred cutoff values for the range of relative
d2 is the density distribution of test values from infected costs and prevalences that were tested. None of these values
individuals. correspond to the usual placement of cutoff values currently
The equation above may also be expressed in terms of recommended (about 0.145 to 0.15). The preferred cutoff
sensitivity (Se) and specificity (Sp) such that the preferred value was extremely sensitive to changes in both the disease
cutoff value is the value that minimizes prevalence and the relative cost of misdiagnosis.
Estimating the additional benefit of using the preferred
(1 p)(l Sp) + rp(l Se).
- - -
cutoff value instead of the currently recommended cutoff
In this analysis, it was assumed that the population value. Table 1 summarizes the additional benefit of using the
distributions were identical to the sample distributions. preferred cutoff value for 50 of the 1,064 combinations of
An algorithm was developed on a commercial spreadsheet prevalence and relative cost of misdiagnosis. The greatest
program (Lotus 123, version 2.0) to perform the necessary benefit of using the preferred cutoff value rather than the
calculations. The algorithm was limited to the examination recommended cutoff value occurred when there was high1258 RIDGE AND VIZARD J. CLIN. MICROBIOL.
10
3.2
:i1.0
0~~~~~~~~~00
0~~~~~01
Downloaded from http://jcm.asm.org/ on February 9, 2021 by guest
. 0.32
0.25
I , II III
0.1 ... \ . X. ,,,, I, IZ
.,I , ,
0 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55
0.05 0.1
True Prevalence of Paratuberculosis
FIG. 1. Decision surface showing the preferred cutoff value for the Johne's Absorbed EIA (1% clinical cases) for prevalence ranging from
0 to 55% and relative cost of misdiagnosis ranging from 0.1 to 10. FN, false negative; FP, false positive.
prevalence of disease and when false-negative test results
were considered much more costly than false-positive test Cut-off Prevalence Probability Benefit
results.
Estimation of the best-option cutoff value when the proba- 0.0 ----(0.8) -1378.7
bility of disease is not known with certainty. The best-option 0. 1
0.025 )0.1) -1310.9
cutoff value for the situation of unknown but probably low
prevalence was calculated to be 0.13. Figure 2 displays this $1314.06 0.15 (0.05) -1016.4
decision tree. The best-option cutoff value when there was
an unknown but probably high disease prevalence was an 0.38 0.05) -583.8
OD450 value of 0.07. 0.0 0.8) -208.4
Changes in the relative cost of misdiagnosis significantly
altered the outcome of the analysis for each situation exam- 0.06 0 0.025 0.1) -180.8
ined. -$140.34 0.15 (0.05) -87.0
DISCUSSION 0.38 (0.05) -23.8
Several methods have been employed to determine the 0.0 0.8) -80.1
cutoff value for serological assays with continuous or ordinal
test result ranges. They can be grouped into five categories: 0.07
00.1) 0 025 -59.4
(i) arbitrary methods, (ii) methods to optimize sensitivity or -70.02 0.1 0.05) 0.0
~VOL. 31, 1993 DETERMINATION OF OPTIMAL CUTOFF FOR SEROLOGICAL ASSAY 1259
specificity, (iii) methods to optimize test accuracy, (iv) ROC cutoffvalues for diagnostic tests should be reviewed as more
curves, and (v) methods to optimize the predictive value. data become available.
Each has limitations. An estimation must also be made about the cost of a
Arbitrary methods include the recommendation of a sub- false-negative diagnosis relative to the cost of a false-
jective cutoff value or the use of a statistical parameter (such positive diagnosis. Often this estimation can be made in
as 3 standard deviations above the mean for negative con- veterinary medicine on a purely economic basis, but this is
trol) to set the cutoff value. These procedures fail to consider not so in medical diagnosis. Weighing the social, physical,
test sensitivity and specificity and the number or costs of and psychological costs of misdiagnosis is not easy. The
misdiagnoses produced by the test settings. In addition, difficulty in determining the relative cost of misdiagnosis
while the statistics employed may have been applicable to may lead some clinicians to believe that this method is
the original population used to evaluate the test, they may inappropriate for determining the cutoff value for medical
not have any relevance in the populations to be tested. tests. However, we contend that consciously or uncon-
Determining cutoff values by optimizing either test sensi- sciously, these assessments are being made whenever cutoff
tivity alone or test specificity alone will do so at the expense points are being set. An advantage of the method outlined is
Downloaded from http://jcm.asm.org/ on February 9, 2021 by guest
of the other parameter (15). Such methods also fail to that it brings into the open the hidden assumption of the
examine the cost incurred from misdiagnoses or the effect of relative cost of misdiagnosis implicit in any calculation of a
disease prevalence on the frequency of false-positive or preferred cutoff value.
false-negative test results. The algorithm used to determine the preferred cutoff value
Some researchers have defined the optimal cutoff as the was limited to the examination of only 16 possible cutoff
value which maximizes the total number of correct diag- values. The preferred cutoff value selected for each combi-
noses in the experimental population. This method fails to nation of prevalence and r was the best of these 16 but may
not have been the optimum cutoff value if all possible cutoff
recognize that test accuracy is a function of disease preva- values were examined. However, the difference between
lence (27) and it will produce a cutoff value that maximizes adjacent possible cutoff values (generally 0.01 OD unit) was
test accuracy only for populations with the same prevalence close to the precision limits of the ELISA testing system.
as the experimental population. Even if the prevalence of Since the sample distributions of the uninfected and in-
disease is accounted for, this method fails to consider the fected individuals were not uniformly smooth, only five
economic and social costs associated with misdiagnoses. preferred cutoff values were identified. The two extreme
The costs incurred by incorrectly classifying an uninfected preferred cutoff values, 0.05 and 0.25, represent the points
individual as infected (false positive) may be different from that provide nearly perfect sensitivity and perfect specificity,
the costs incurred by incorrectly classifying an infected respectively. They have limited value diagnostically. The
individual as uninfected (false negative). Therefore, mini- intermediate points represent cutoff values for which small
mizing the total number of misdiagnoses may not minimize changes in the number of animals classified as positive
the total cost associated with these misdiagnoses. resulted in large changes in the test sensitivity or specificity.
ROC curves graphically illustrate the effect of changing In the example shown, the preferred cutoff value was
the cutoff value on test sensitivity and specificity (16). If the similar to the manufacturer's recommended cutoff value
curve lies above the major diagonal of the graph (bottom left when disease prevalence was low and/or when the cost of a
to top right), the test provides useful diagnostic information. false-positive misdiagnosis was far greater than the cost of a
ROC curves have been used to select cutoff values for false-negative misdiagnosis. This situation could arise, for
numerous tests by identifying the point which is maximally example, in preexport testing of cattle by exporters. The
distant from the diagonal. However, this cutoff value maxi- preferred cutoff value was very different from the manufac-
mizes the benefit of testing only when disease prevalence is turer's recommended cutoff value when disease prevalence
50% and each type of misdiagnosis is equally costly (r = 1) was high and the cost of a false-negative result was far
(4) or for a single and unknown r value (of the infinite greater than the cost of a false-positive diagnosis. This
possible r values) for any other given prevalence. situation would arise if the test were used to control paratu-
Using predictive values (positive or negative) to set a berculosis in a heavily infected herd. Using the preferred
cutoff value for diagnostic tests maximizes the number of cutoff value instead of the manufacturer's recommendation
true-positive or true-negative test results. Predictive values in such situations resulted in large financial gains. For
are sensitive to changes in disease prevalence (15), so that example, it was estimated (Table 1) that in a herd of 100
the cutoff value selected for one population may be inappro- cows with 30% prevalence of paratuberculosis there was a
priate for another. Once again, no consideration is given to $2,680.00 advantage in using the preferred cutoff value when
the costs associated with each type of misdiagnosis. false negatives were valued at -$800.00 and false positives
The process outlined in this report provides a method for were valued at -$80.00. The monetary value assigned to the
determining the optimal cutoff value for a test that takes into cost of a false-positive test result may vary considerably and
consideration disease prevalence and the costs associated can be calculated simplistically, as in this example, or in
with misdiagnosis. To determine the optimum cutoff value of more detail, to allow for lost genetic material, restricted
a test by this method, assumptions must be made about the access to markets, and other sources of loss.
distribution of test values in the uninfected and infected There are considerable benefits associated with standard-
populations to which the test is to be applied. In this study it ized diagnostic tests, particularly in relationship to export
was assumed that the population distributions of ELISA OD testing, animal health regulation, and disease control. How-
values were identical to the sample distributions. Optimum ever, as this study demonstrates there are considerable
cutoff values are highly dependent on the assumptions made advantages to tailoring cutoff values to suit the specific
regarding the distribution of test values in the infected and requirements of the testing situation. For example, the cutoff
uninfected populations (31). There is often insufficient infor- value for the Johne's Absorbed EIA could be changed to a
mation to define the parent density distributions adequately, value in the range 0.07 to 0.13 for almost all routine diag-
especially in the tails of the distribution. Decisions regarding nostic testing and for herds for which a planned eradication1260 RIDGE AND VIZARD J. CLIN. MICROBIOL.
program is undertaken. The current cutoff value (0.15) or tion of specific antibodies to measles virus. J. Clin. Microbiol.
another somewhat higher value is appropriate for preexport 29:1466-1471.
testing by exporters to ensure that very few false-positive 9. Gilbert, M., J. Kirihara, and J. Mills. 1991. Enzyme-linked
diagnoses are made. immunoassay for human immunodeficiency virus type 1 enve-
lope glycoprotein 120. J. Clin. Microbiol. 29:142-147.
If a clinician believed that the prevalence of disease in the 10. Gonik, B., M. Seibel, A. Berkowitz, M. B. Woodin, and K. Mills.
population being tested was about 25% and that a false- 1991. Comparison of two enzyme-linked immunoassays for
negative result was as costly as a false-positive result (r = 1), detection of herpes simplex virus antigen. J. Clin. Microbiol.
then an examination of the decision surface shown in Fig. 1 29:436-438.
indicates that the preferred cutoff value is 0.07. However, in 11. Hansen, K., K. Pii, and A.-M. Lebech. 1991. Improved immu-
situations in which the probability of disease is not known noglobin M serodiagnosis in Lyme borreliosis by using a ,u-cap-
with any certainty, it is difficult to use the decision surface ture enzyme-linked immunosorbent assay with biotinylated Bor-
shown in Fig. 1 or other normal cost-benefit analysis tech- relia burgdorferi flagella. J. Clin. Microbiol. 29:166-173.
niques to determine the preferred cutoff value. Other ana- 12. Hartley, T. M., G. E. Malone, R. F. Khabbaz, R. B. Lal, and
J. E. Kaplan. 1991. Evaluation of a recombinant human T-cell
lytical methods such as decision analysis are available to lymphotrophic virus type 1 (HTLV-1) p21E antibody detection
Downloaded from http://jcm.asm.org/ on February 9, 2021 by guest
assist researchers making decisions about test cutoff values enzyme immunoassay as a supplementary test in HTLV-I/II
under conditions of uncertainty about disease prevalence or antibody testing algorithms. J. Clin. Microbiol. 29:1125-1127.
cost factors. Even though some of the data used for decision 13. Knowles, D. P., Jr., L. E. Perryman, L. S. Kappmeyer, and S. G.
analysis may be quite imprecise, the resultant cutoff values Hennager. 1991. Detection of equine antibody to Babesia equi
should be more reliable than purely arbitrarily derived merozoite proteins by a monoclonal antibody-based competitive
figures. The decision analysis results in this study indicate inhibition enzyme-linked immunosorbent assay. J. Clin. Micro-
that the best-option cutoff values for two commonly encoun- biol. 29:2056-2058.
tered situations (prevalence unknown but probably low and 14. Lal, R. B., W. Heneine, D. L. Rudolph, W. B. Present, D.
Hotbienz, T. M. Hartley, R. F. Khabbaz, and J. E. Kaplan. 1991.
prevalence unknown but probably high) were lower than the Synthetic peptide-based immunoassays for distinguishing be-
manufacturer's recommended cutoff value as well as being tween human T-cell lymphotropic virus type 1 and type 2
quite different from each other. infections in seropositive individuals. J. Clin. Microbiol. 29:
In summary, the results of this study indicate that the 2253-2258.
currently recommended cutoff value for the Johne's ab- 15. Martin, S. W. 1977. The evaluation of tests. Can. J. Comp.
sorbed EIA is close to optimum only when the disease Med. 41:19-25.
prevalence is very low and false-positive test results are 16. Metz, C. W. 1978. Basic principles of ROC analysis. Semin.
deemed to be very costly. In other situations, there were Nucl. Med. 8:283-298.
considerable financial advantages in using cutoff values 17. Montagna, R. A., L. Papsidero, and B. J. Poiesz. 1991. Evalua-
calculated to maximize the benefit of testing. It is probable tion of a solid-phase immunoassay for the simultaneous detec-
tion of antibodies to human immunodeficiency virus type 1 and
that the current cutoff values for other diagnostic tests may human lymphotropic virus type 1. J. Clin. Microbiol. 29:897-
not be the most appropriate for every testing situation. This 900.
paper offers a method for identifying the cutoff value that 18. Ngategize, P. K., J. B. Kaneene, S. B. Harsh, P. C. Bartlett, and
maximizes the benefit of testing for any testing situation. E. L. Mather. 1986. Decision analysis in animal health pro-
grams: merits and limitations. Prev. Vet. Med. 4:187-197.
19. Nielsen, R., T. Plambeck, and N. T. Foged. 1991. Blocking
REFERENCES enzyme-linked immunosorbent assay for detection of antibodies
1. Anderson, T. W. 1958. An introduction to multivariate statistical to Actinobacillus pleuropneumoniae serotype 2. J. Clin. Micro-
analysis, p. 126-131. John Wiley & Sons, Inc., New York. biol. 29:794-797.
2. Blood, D. C., 0. M. Radostits, and J. A. Henderson. 1983. 20. Olson, J. G., T. W. Scott, L. H. Lorenz, and J. H. Hubbard.
Veterinary medicine: a textbook of the diseases of cattle, sheep, 1991. Enzyme immunoassay for detection of antibodies against
pigs, goats, and horses, 6th ed., p. 643-649. Bailliere Tindall, eastern equine encephalomyelitis virus in sentinel chickens. J.
London. Clin. Microbiol. 29:1457-1461.
3. Castillo, R. M., P. Grados, C. Carcamo, E. Miranda, T. Mon- 21. Paerragaard, A., G. H. Shand, K. Gaarslev, and F. Epersen.
tenegro, A. Guevara, and R. H. Gilman. 1991. Effect of treat- 1991. Comparison of crossed immunoelectrophoresis, enzyme-
ment on serum antibody to Hymenolepis nana detected by linked immunosorbent assays, and tube agglutination for sero-
enzyme-linked immunosorbent assay. J. Clin. Microbiol. 29: diagnosis of Yersinia enterocolitica serotype 0:3 infection. J.
413-414. Clin. Microbiol. 29:302-309.
4. Connell, F. A., and T. D. Koepsell. 1985. Measures of gain in 22. Ridge, S. E., I. R. Morgan, D. C. Sockett, M. T. Collins, R. J.
certainty from a diagnostic test. Am. J. Epidemiol. 121:744-753. Condron, N. W. Skilbeck, and J. J. Webber. 1991. Comparison
5. Cox, J. C., D. P. Drane, S. L. Jones, S. E. Ridge, and A. R. of the Johne's absorbed EIA and the complement-fixation test
Milner. 1991. Development and evaluation of a rapid enzyme for the diagnosis of Johne's disease in cattle. Aust. Vet. J.
immunoassay test for the diagnosis of Johne's disease in cattle. 68:253-257.
Aust. Vet. J. 68:157-160. 23. Sandin, R. L., C. C. Knapp, G. S. Hall, J. A. Washington, and
6. Decoster, A., B. Slizewicz, J. Simon, C. Bazin, F. Darcy, G. I. Rutherford. 1991. Comparison of the Vitek immunodiagnostic
Vittu, C. Boulanger, Y. Champeau, J. L. Demory, M. Duhamel, assay system with an indirect immunoassay (Toxostat test kit)
and A. Capron. 1991. Platelia-Toxo IgA, a new kit for early for detection of immunoglobin G antibodies to Toxoplasma
diagnosis of congenital toxoplasmosis by detection of anti-p30 gondii in clinical specimens. J. Clin. Microbiol. 29:2763-2767.
immunoglobin A antibodies. J. Clin. Microbiol. 29:2291-2295. 24. Schwarz, T. F., S. Modrow, B. Hottentrager, B. Hoflacher, G.
7. DiPersio, J. R., F. J. Varga, D. L. Conwell, J. A. Kraft, K. J. Jager, W. Schartl, R. Sumazki, H. Wolf, J. Middeldorp, M.
Kozak, and D. H. Willis. 1991. Development of a rapid enzyme Roggendorf, and F. Deinhardt. 1991. New oligopeptide immu-
immunoassay for Clostndium difficile toxin A and its use in the noglobin G test for human parvovirus B19 antibodies. J. Clin.
diagnosis of C. difficile-associated disease. J. Clin. Microbiol. Microbiol. 29:431-435.
29:2724-2730. 25. Sockett, D. C., D. J. Carr, W. D. Richards, and M. T. Collins.
8. Erdman, D. D., L. J. Anderson, D. R. Adams, J. A. Stewart, 1992. A repository of specimens for comparison of diagnostic
L. E. Markowitz, and W. J. Bellini. 1991. Evaluation of mono- testing procedures for bovine paratuberculosis. J. Vet. Diagn.
clonal antibody-based capture enzyme immunoassays for detec- Invest. 4:188-191.VOL. 31, 1993 DETERMINATION OF OPTIMAL CUTOFF FOR SEROLOGICAL ASSAY 1261
26. Toth, I., T. J. Barrett, M. L. Cohen, H. S. Rumschlag, J. H. Microbiol. 29:5-9.
Green, and I. K. Wachsmuth. 1991. Enzyme-linked immunosor- 30. Verstinen, C. P. H. J., H. M. Ly, K. Polman, C. Richter, S. P.
bent assay for products of the 60-megadalton plasmid of Esch- Smits, S. Y. Maselle, P. Peerbooms, D. Rienthong, N. Montree-
erichia coli serotype 0157:H7. J. Clin. Microbiol. 29:1016-1019. wasuwat, S. Koanjanart, D. D. Trach, S. Ku"per, and A. H. J.
27. Trajstman, A. C. 1979. Diagnostic tests, sensitivity, specificity, Kollk 1991. Enzyme-linked immunosorbent assay using mono-
efficiency and prevalence. Aust. Vet. J. 55:501. clonal antibodies for identification of mycobacteria from early
28. van den Oever, H. L. A., R. J. L. F. Loffeld, and E. E. cultures. J. Clin. Microbiol. 29:1372-1375.
Stobberingh. 1991. Usefulness of a new serological test (Bio- 31. Vizard, A. L., G. A. Anderson, and R. B. Gasser. 1990.
Rad) to diagnose Helicobacter pylon-associated gastritis. J. Determination of the optimum cut-off value of a diagnostic test.
Clin. Microbiol. 29:283-286. Prev. Vet. Med. 10:137-143.
29. van Oirschot, J. T. 1991. Comparative evaluation of an en- 32. Zoller, A., I. Kramer, R. Kappe, and H. G. Sonntag. 1991.
zyme-linked immunosorbent assay (ELISA) to detect anti- Enzyme immunoassays for invasive Candida infection: reactiv-
bodies directed against glycoprotein 1 of pseudorabies virus ity of somatic antigens of Candida albicans. J. Clin. Microbiol.
and a conventional ELISA and neutralization tests. J. Clin. 29:1860-1867.
Downloaded from http://jcm.asm.org/ on February 9, 2021 by guestYou can also read
