1. For more information, please contact Edith Leclercq: e.leclercq@amc.uva.nl
Electronic searching versus handsearching of conference abstracts
Edith Leclercq, Leontien CM Kremer, Elvira C van Dalen
Cochrane Childhood Cancer Group, Department of Paediatric Oncology, Emma Children's Hospital (EKZ) / Academic Medical Center, Amsterdam, the Netherlands
Handsearching of conference abstracts is time consuming, but necessary for both identifying
randomized controlled trials (RCTs) and controlled clinical trials (CCTs) not available in medical
databases like PubMed/Medline and Embase, and for establishing and updating the Trials
Registers of Cochrane Review Groups. Nowadays conference abstracts are often not only
available in print, but also as a pdf file, which makes it possible to perform an electronic search of
these abstracts. The aim of this study was to compare the results of an electronic search of
conference abstracts with the results of handsearching.
The 2008 conference abstracts of the International Society of Paediatric Oncology (SIOP) meeting
were searched electronically using the following keywords: evidence, intervention, protocol,
clinical trial, random, trial, controlled, RCT, CCT, phase, treatment arm, assign and stratif.
Searches were performed for each keyword separately. The results of the electronic search were
compared with the results obtained by handsearching the printed version of the abstract book.
The abstracts describing RCTs and CCTs were identified by electronic searching with one or more
of the following keywords: random, trial, protocol, intervention, and stratif. The other keywords did
not add any additional abstract. In Table 1 the frequency of all used keywords identifying relevant
abstracts for SIOP 2008 abstracts are summarized.
By electronic searching 438 abstracts were identified, of which 40 were assessed as RCT or CCT
(9.1%). The printed version contained 893 abstracts, of which 24 abstracts were assessed as being
a RCT or CCT (2.7%). Handsearching identified no RCTs or CCTs that were not identified in the
electronic search.
Assuming that 40 of the 893 SIOP 2008 abstracts are describing a RCT or CCT, then the sensitivity
of the electronic search and handsearching is 100% and 60% respectively. The Number Needed to
Read (NNR) is 10.95 and 37.2 for electronic and handsearching respectively.
Sixteen abstracts were missed by handsearching. When an abstract is identified by electronic
searching, it seems that the reviewer is unintentionally convinced that it is a relevant abstract,
whereas with handsearching the decision is mainly based on information provided in title and
methods section. When these sections are not providing relevant information, the abstract is likely
to be judged as not relevant.
Although less abstracts had to be read by electronic searching, time to identify relevant studies
seems to be similar to handsearching, due to more administrative paperwork relating to performing
the different searches.
Table 1. Identification of SIOP abstracts
Background
Conclusions
Methods
Results
All abstracts Relevant abstracts
Keyword Number of
abstracts
Number of
abstracts (%)
Number of
abstracts
Number of
abstracts (%)
evidence 62 14.2 3 7.7
intervention 48 11.0 5 12.8
protocol 202 46.2 14 35.9
clinical trial 38 8.7 3 7.7
random 53 1.1 25 64.1
trial 119 27.2 23 59.0
controlled 14 3.2 4 10.3
RCT 1 0.2 0 0.0
CCT 4 0.9 0 0,0
phase 69 15.8 3 7.7
treatment arm 0 0.0 0 0.0
assign 12 2.8 2 5.1
stratif 9 2.1 1 2.6
Electronic searching of SIOP conference abstracts is a good option to identify abstracts of trials in
paediatric oncology. Less abstracts had to be evaluated by searching the electronic version, but
this did not seem to be time saving. No abstracts were missed by electronic searching compared
to handsearching, but the other way around relevant abstracts were missed. The sensitivity and
NNR of the electronic search were outstanding compared to handsearching. However, the results
of this study need to be confirmed in a larger study including more conference abstracts.