Glomerular disease search filters for Pubmed, Ovid Medline, and Embase: a development and validation study

Retrieving health literature is the cornerstone to evidence-based practice. However, the sheer volume of available information presents a challenge to even the most skilled physicians and researchers. Many users lack knowledge of information sources, have difficulty formulating an optimal search strategy, and are short on time [1‐3]. These obstacles may be even greater when dealing with an area of nephrology such as glomerular disease, which is particularly broad, multidisciplinary, and difficult to define. Indexing of articles is often inconsistent, with variable terminology used for similar clinical entities or histologic diagnoses [4]. In this case, searches need to be highly sensitive to ensure important evidence is not overlooked, while minimizing the retrieval of non-relevant articles to ensure efficiency.

Search filters are a logical way to deal with these barriers. Filters are pre-tested searches created by strategically combining individual and combinations of search terms to achieve optimal article retrieval for a given purpose. Many filters already exist including those optimized to retrieve studies and systematic reviews of diagnosis, etiology, treatment, outcomes, adverse events, prognosis, and clinical prediction guides [5‐14]. More recently, topic-based search filters have started to emerge [15‐17]. Within the area of nephrology, search filters already exist to retrieve renal information and articles relevant to kidney transplantation [15, 18]. However, none of these filters were designed to enhance retrieval of articles relevant only to glomerular disease.

A search filter for glomerular disease would allow physicians to perform searches within a subset of articles in an online database that were preselected as relevant to this content area. For example, if a user wanted to determine the most effective immunosuppressive therapy for a case of membranous nephropathy, they could combine the terms ‘treatment membranous’ with the glomerular disease search filter to improve the precision of article retrieval. The search filter acts as an optimized substitute for topic-specific terms required to increase the sensitivity and specificity of the search and eliminates the need to enter these glomerular disease terms and synonyms in the search query (e.g., nephropathy, glomerulopathy, glomerulonephritis). This strategy, in theory, should maximize the retrieval of articles relevant to glomerular disease and minimize non-relevant articles, increasing the overall precision of each search.

We conducted this study to develop and test glomerular disease search filters for PubMed, Ovid Medline, and Embase. Afterwards, we did some proof of concept searches to illustrate the potential effectiveness of these new filters with real physician searches in the PubMed database at large.

We used a diagnostic test assessment framework to develop and validate search filters for glomerular disease. For the purpose of this study, glomerular disease was defined as any disease in which the glomerulus of the kidney is affected, resulting in hyperplasia, atrophy, necrosis, scarring, or deposits in the glomeruli.

Building on the same concepts our group has used to create novel high performance search filters for general nephrology and renal transplantation [15, 18], we have succeeded in developing and validating search filters for glomerular disease that are highly sensitive and specific. All filters achieved a balance of at least 93.8% sensitivity and specificity. Our best performing high-sensitivity filter was in Embase, achieving 99.0% sensitivity and 95.3% specificity. The best performing high-specificity filter was also in Embase, which reached 95.7% sensitivity and 98.6% specificity. Without changing their PubMed search terms, in an illustrative example physicians were able to retrieve articles with a higher degree of precision (less non-relevant articles) with use of these filters.

These filters are complex, often combining in excess of 50 terms with Boolean operators. Coding these filters into the PubMed and Ovid search engine interfaces will permit their easy use by anyone doing a search. In the meantime, we provide these filters at the following link: http://​hiru.​mcmaster.​ca/​hiru/​hiru_​hedges_​nephrology_​filters.​aspx. As of September 2011, use of the high-sensitivity glomerular disease filter reduced the PubMed database from 21 million to 195,374 articles, and the high specificity filter reduced this to 107,658 articles.

Depending on the search terms entered by the user, these filters may serve many purposes, which are best understood in the context of our illustrative proof of concept searches (Table 3). First, without changing the original search term(s), selecting a filter applies the search only to a subset of articles that are richer in glomerular disease content. The result is an increase in precision of the search, similar to the increase in positive predictive value of a screening test when applied to a high-risk population. This was demonstrated by the use of search terms ‘minimal change treatment’ in Table 3. Fewer non-relevant articles were retrieved with use of the filter (1236 versus 4662 articles), without impacting relevant article retrieval. Second, the filter acts as an optimized substitute for glomerular disease specific terms and synonyms allowing users to simplify the search query. This avoids unnecessarily limiting the search due to indexing inconsistencies inherent with the terminology used to define glomerular disease. For example, if a user was searching for dietary recommendations in diabetic nephropathy, the search terms may be simplified to ‘low protein diet diabetes’, instead of searching for ‘low protein diet diabetes’ with selected terms such as ‘nephropathy’, ‘kidney disease’, or ‘glomerulosclerosis’ that may negatively impact relevant article retrieval. In this case, even without use of search terms pertaining to glomerular disease, precision of the results was enhanced (Table 3). Third, users may opt to exclude disease specific terms entirely and use the filter to address questions that potentially relate to all glomerular disease equally. An example of this may include entering ‘immunization’ when addressing the impact of vaccinations in patients with glomerular disease.

Our results also highlight that even with high performance validated search filters, a single search will rarely retrieve everything of relevance on a particular topic. There is simply too much variation in the quality of accompanying search terms entered by the user, completeness of the database, and quality and consistency of indexing. This explains why in some proof of concept searches, retrieval of relevant articles was incomplete both with and without use of the filter (Table 3). Also, the extent to which the search filter is generalizable depends upon the sample of journals selected for study and the method by which articles were defined as relevant. Our selection of journals was deliberately enriched with leading clinical nephrology journals. Although it also included a random sampling of other journals, this set of journals may not adequately represent the complete set of multi-disciplinary journals that feature glomerular disease content in PubMed. This may explain the significant drop in precision when the filter was applied to the validation set, which was a smaller database by design with a lower proportion of relevant articles. Our choice to divide articles into the development and validation sets at the journal level may have also contributed to the lower proportion of articles with glomerular disease content in the validation set. However, this approach provided insight as to what would occur if the search database were expanded to include the over 5000 journals indexed in PubMed.

Proof of concept searches were used to illustrate the functionality of our best performing filters with real physician searches. In each case, the clinical questions formulated from recent systematic reviews were relevant to glomerular disease and physician searches appear typical for the average user. These examples show a gain in search strategy precision with use of the high-sensitivity and high-specificity filter through a dramatic reduction in non-relevant articles. This occurs without sacrificing retrieval of relevant articles in most cases. However, the methods for defining the reference standard based on articles used in systematic reviews of variable quality is indirect and has not been compared with one derived from hand searching [34]. For this reason the proof of concept searches should be viewed as illustrative examples, not as evidence of further filter validation.

These search filters for glomerular disease were designed to offer physicians and researchers a strategy to optimize results by sensitivity or specificity, depending on the level of article retrieval they deem manageable on a practical level. Filters that maximize sensitivity involve a compromise on the level of precision achieved, though this still may appeal to a researcher conducting a systematic review. For busy physicians at the point of care, we recommend starting with the high-specificity filter. To narrow results even further, physicians may prefer use these search filters in conjunction with previously developed methods filters, such as the therapy filter for randomized controlled trials available via PubMed’s Clinical Queries section [5‐14]. This approach has not been formally tested with the glomerular disease filters, but in a recent study has been shown to increase the efficiency of retrieval of articles relevant to renal care [35].

In conclusion, we have succeeded in developing and validating high performance search filters for glomerular disease that can be easily applied by the busy physician. We expect this will contribute to more efficient and effective evidence-based decision-making, education, and patient care. Future research is required to measure this impact, and to better understand the usefulness of these filters when used in combination with previously developed methods filters for physician searches.