The CareStart™ Malaria HRP-2/pLDH (Pf/pan) Combo Test was retrospectively evaluated in a reference laboratory on a panel of stored blood samples obtained in international travellers suspected of malaria. The reference method was microscopy corrected by Polymerase Chain Reaction (PCR). The study design was in compliance with the STARD guidelines for presentation of diagnostic studies [5].

A panel was selected from a collection of EDTA-blood samples stored at -70°C at the Institute of Tropical Medicine (ITM), Antwerp, Belgium. Between January 1996 and May 2009 these samples were obtained in patients suspected of malaria, including international travellers and, to a lesser extent, natives of endemic countries returning from visiting friends and relatives. Most of these samples were obtained in patients attending the outpatient clinic of the ITM, another part was sent by other Belgian laboratories to ITM for confirmation in the scope of the national reference laboratory. The samples collected at ITM were aliquoted and frozen at -70°C the day of collection. Between collection and storage, the samples remained a maximum of 8 hours at laboratory temperature (below 25°C). The samples submitted by Belgian laboratories for confirmation, were sent by mail and had been exposed to ambient temperature for the period of shipment (generally 24 hours, with a maximum of 48 hours). The delay and processing before storage at -70°C had been validated and were compliant with routine laboratory procedures. The selected panel included samples with the four malaria species at varying parasite densities as well as malaria-negative samples. The samples were classified in regions according to the United Nations classification of geographical region and composition [6].

All samples were blindly analyzed by microscopy and real-time polymerase chain reaction for diagnosis of malaria, species identification and determination of parasite density, as described previously [7]. A species-specific PCR was adapted from Rougemont et al as described previously [7,8]. In case of discordant results between microscopy and PCR, the result of PCR was used as the reference method.

The CareStart™ Malaria HRP-2/pLDH (Pf/pan) Combo Test is a lateral flow antigen detection test in a cassette format. It is a three-band RDT targeting HRP-2 and pan-pLDH. The presence of a unique HRP-2 line indicates an infection with P. falciparum, whereas a unique pan-pLDH line is found in infection with one or more of the non-falciparum species. The presence of both HRP-2 and pan-pLDH lines indicates an infection with P. falciparum or a mixed infection with P. falciparum and one or more of the non-falciparum species.

For the evaluation of CareStart™ Malaria HRP-2/pLDH (Pf/pan) Combo Test, test kits from four different lot numbers were used for evaluation, supplied in two kit presentations : a self-test kit containing 60 individually wrapped packages including cassette, lancet, assay buffer, sample pipette and instructions, and a kit designed for laboratory use with 25 individually packed cassettes. The test kits had been stored between 18°C and 24°C before use. All malaria tests carried out at ITM are accredited in accordance with the requirements of the standard NBN EN ISO 15189:2007.

Tests were performed according to the instructions of the manufacturer, except that the plastic transfer straws supplied in the kit were replaced by a transfer pipette (Finnpipette, Helsinki, Finland). Readings were carried out at daylight assisted by a standard electric bulb by three subsequent observers, of whom the first always was the one performing the tests. The first observer performed readings at 20 minutes (which is the reading time recommended by the manufacturer), followed by observers 2 and 3 within an additional 10 minutes. The observers were blinded to each other's readings and to the results of microscopy and PCR.

In case no control line appeared the test was considered invalid and was repeated. To score line intensities we used a scoring system of five categories as defined previously [7]: none (no line visible), faint (barely visible line), weak (paler than the control line), medium (equal to the control line) or strong (stronger than the control line). The test results were based on consensus agreement, which means that an identical result read by at least two out of three observers was withheld. In case of no consensus, the results of the first reader were considered.

To assess inter-observer agreement, results of positive and negative readings as well as line intensities were considered. Test reproducibility was evaluated by testing 15 samples representing all species at variable parasite densities on five consecutive occasions.

Sensitivity and specificity were calculated separately for P. falciparum and the non-falciparum species with 95% confidence intervals (C.I.). The interpretation of test results for P. falciparum and the non-falciparum species is shown in Table 1. Samples with pure gametocytaemia were included among the P. falciparum species. The Pearson Chi-square test was used to determine significance of results, or in case of small sample size, a two-tailed Fisher's exact test. A p-value < 0.05 was considered as significant.

Inter-observer agreements for line intensities and positive and negative test results were expressed by kappa values for each pair of observers and by the percentage of overall agreement between the three observers. To assess strength of associations between line intensity readings and parasite densities Cramer's V for categorical variables was used [7]. To assess the potential interference of sample storage on test sensitivity, multivariate analysis (logistic regression) was used for each Plasmodium species separately. It was hypothesized that sensitivity declined in relation to (i) decreasing parasite densities (ii) longer duration of storage at -70°C and (iii) that there was an interaction between the parasite densities and the duration of storage (antigen degradation related to the amount of antigen present at the beginning). For the multivariate analysis, parasite densities were included after log10 transformation and declines of sensitivity were expressed as Odds ratios. Analyses were conducted using Stata11 (Stata Corporation, Texas, USA).

In case of species mismatch (P. falciparum diagnosed as non-falciparum, or non-falciparum diagnosed as P. falciparum), the sample was retested with the CareStart™ Malaria HRP-2/pLDH (Pf/pan) Combo Test and with two other RDTs (based on HRP-2 and HRP-2/pan-pLDH).

The technicians who performed the tests were asked to evaluate the kit's content and instructions for clarity, and problems and incidents during test performance were consequently recorded.

The study was approved by the Institutional Review Board of ITM and by the Ethical Committee of Antwerp University, Belgium.

The samples were selected from the ITM collection, which consists of 1,200 samples. Mixed infections were excluded and only the first sample of each patient was considered. For P. falciparum, which is the most frequently retrieved species at ITM, a panel representing the different parasite densities, stages (including only gametocytes) and geographical origin, was selected. The final panel consisted of 590 samples including infections with P. falciparum (n = 320), Plasmodium vivax (n = 76), Plasmodium ovale (n = 76), Plasmodium malariae (n = 23) and Plasmodium negative samples (n = 95) (Table 2). These samples were obtained in 590 patients with a male:female ratio of 2.16:1. The median age was 35 years (range 1 - 84 years), only 15 patients (2.5%) were children under the age of five years. Table 3 shows the results of the microscopic identification corrected by PCR: of the 76 P. ovale samples, seven (9.2%) had originally been diagnosed as P. vivax by microscopy, and four out of 76 (5.3%) P. vivax samples had originally been diagnosed as P. ovale. The geographical distribution of the samples used is shown in Table 2. Samples were obtained in Africa (n = 444), Asia (n = 54), Latin America and the Caribbean (n = 12) and Oceania (n = 4). Of 76 samples no data on the geographic origin were known and could not be retrieved. Most (268/320, 83.8%) P. falciparum infections had been acquired in sub-Saharan Africa.

No invalid test results were observed. One cassette was broken, and one blister contained no cassette. Table 4 shows the number of positive HRP-2 and pan-pLDH lines for all species. Tables 5 and 6 show the test characteristics according to parasite density. For P. falciparum, the overall sensitivity was 88.8%. Sensitivity was related to parasite density: it was 69.6% at parasite densities below 100/μl and increased to 94.3% and 99.3% at parasite densities above 100 and 1,000/μl respectively. Nineteen out of 28 false-negative samples had a parasite density below 100/μl (including five samples with pure gametocytaemia), eight had parasite densities ranging from 120 - 467/μl and the remaining sample had a parasite density of 703/μl. All of these infections had been acquired in sub-Saharan Africa. Eight (2.5%) P. falciparum samples only reacted with the pan-pLDH line and consequently represented species mismatch, as they were incorrectly diagnosed as non-falciparum species. Parasite densities of these samples ranged from 32 - 371/μl, except for one sample with a parasite density of 1,123/μl; all had been acquired in sub-Saharan Africa (Tables 3, 4 and 6). The species mismatch results were reproducible upon retesting with CareStart™ Malaria HRP-2/pLDH (Pf/pan) Combo Test. Additional testing of these samples with two other RDT brands (HRP-2 and HRP-2/pan-pLDH) scored five of them as P. falciparum; the three other samples, obtained in travellers returning from Ghana and DRC, were negative (parasite densities were 32, 70 and 210/μl).

For the non-falciparum species, there were 80/175 (45.7%) samples correctly identified, two samples had both a visible HRP-2 and pan-pLDH line, a single sample showed a unique HRP-2 line and 91 samples showed no test line (Table 4). Species mismatch results were reproducible upon retesting with the CareStart™ Malaria HRP-2/pLDH (Pf/pan) Combo Test: upon testing with two other RDT brands they were diagnosed as non-falciparum species. Overall sensitivities for P. vivax, P. ovale and P. malariae were 77.6%, 18.4% and 30.4% respectively (Table 6). None of the Plasmodium negative samples was scored as positive.

Tables 7 and 8 list the line intensity readings related to parasite density. For P. falciparum, the majority (205/284, 72.2%) of positive HRP-2 test lines was read as strong or medium, and 27.8% (79/284) were read as weak or faint. Pan-pLDH test lines were read as strong and medium in 129 (51.4%) and as weak or faint in 122 (48.6%) of 251 positive test lines. For the pan-pLDH line intensity in non-falciparum species, there was a clear difference between the three species: the P. vivax samples generated strong and medium line intensities in half (30/61, 49.2%) of positive samples, whereas P. ovale uniquely showed weak or faint lines. For P. malariae, 3 out of 7 positive pan-pLDH lines were read as medium line intensity.

For both the HRP-2 and the pan-pLDH test lines, there was a significant relation between line intensity and parasite density with a substantial correlation (HRP-2: V = 0.353, p < 0.001; pan-pLDH: V = 0.590, p < 0.001), but there was considerable overlap between categories. For P. falciparum, faint HRP-2 lines occurred exclusively at parasite densities below 1,000/μl, whereas a strong HRP-2 line intensity indicated a parasite density higher than 100/μl in a vast majority of samples (128/139, 92.1%). In addition a pan-pLDH line of strong intensity indicated in all but four (114/117, 96.6%) P. falciparum samples a parasite density exceeding 1,000/μl. Of the latter four samples one had a parasite density of 930/μl and the three other samples had pure gametocytaemia. In case of species mismatch for both P. falciparum and non-falciparum species, line intensity readings were faint or weak for the discordant test lines.

For both HRP-2 and pan-pLDH lines, overall agreement and kappa values between pairs of observers for positive and negative readings were excellent, for line intensities they were good (Table 9). Most differences in line intensity readings between two observers occurred within one category of difference (218/231 (94.4%) and 288/294 (98.0%) for HRP-2 and pan-pLDH respectively).

The first observer (who read the test exactly within the recommended reading time of 20 minutes) tended to score weaker pan-pLDH test lines and less positive results compared to observer 2 and 3 (Figure 1), although the differences did not reach statistical significance. As a result, sensitivities for non-falciparum species were slightly higher for observers 2 and 3 (49.1% and 48.6%) compared to observer 1 (44.6%). On the other hand, observer 2 and 3 scored, on different occasions, two times a faint positive pan-pLDH test line in case of a negative sample. For HRP-2 no such tendencies in differences were observed.

Tables 10 and 11 list the results of reproducibility for HRP-2 and pan-pLDH. The results for HRP-2 and pan-pLDH lines were reproducible. For the HRP-2 lines, all differences occurred within a single category of intensity. In two samples this resulted in subsequent faint and negative readings. For the pan-pLDH lines, differences occurred within one (n = 5) and two (n = 2) line intensity categories, and two samples showed subsequent faint and negative readings.

As suspected by the sensitivities shown in Tables 5 and 6, the relation between parasite densities and test sensitivities was statistically significant in a multivariate model for all four Plasmodium species (Table 12). In the same model, an effect of storage, i.e. lower sensitivities for samples stored for long versus short periods, was noted only for P. ovale. If only samples stored from 2001 onwards were considered, sensitivity for P. ovale (n = 38) was 29.0% (95% C.I. 15.4% - 45.9%).

The CareStart™ Malaria HRP-2/pLDH (Pf/pan) Combo Test was scored as easy to perform. The clearing of the test strip was good, and the test lines were easy visualized. The HRP-2 and pan-pLDH test lines showed slightly different color tones (more blue) as compared to the control line. The test instructions in the package insert were scored as clear and the use of pictures was well appreciated. There were slight discordances in interpretation of the test results between the instructions of the self-test kits and the laboratory kits. For instance, the former stated that only the absence of both the control line and the test lines should be read as invalid, whereas the instructions of the laboratory test mentioned the absence of the control line by itself as an invalid test. In addition, the self-test mentioned a positive HRP-2 and pan-pLDH line as positive for P. falciparum, whereas the laboratory test added the option of mixed infection with P. falciparum as well. Finally, the unique presence of a pan-pLDH test line should be considered as a P. vivax infection according to the instructions of the self-test kit, whereas the laboratory test more correctly mentioned the possibility of (mixed) infections by the three non-falciparum species.