Automatic extraction of protein-protein interactions using grammatical relationship graph

Relationships among different biological terms such as genes, proteins, diseases, small molecules, pathways, and gene ontology (TO) terms (called bio-entities in this paper) form the backbone of our knowledge. Bio-entity relationships such as protein-protein interactions (PPIs) are indispensable for understanding of complex diseases, biological processes, and guiding drug discoveries [1]. Human annotation has been used in the past to extract this information from scientific literature, which is then deposited into various databases [2‐21].

However, human annotation can be very time and resource consuming, and keeping pace with the ever increasing amount of biomedical publications has become more and more difficult. As a result, computational methods have been designed to extract bio-entity relationships automatically from the literature, and used to assist scientists in their efforts to build databases using manual annotation approach [22‐48]. Most computational studies attempted to extract PPIs from PubMed abstracts due to the easy accessibility of deposited articles [49, 50]. Most of the PPI extraction methods are based on one of the two ways: (1) specify some rules (or patterns, templates etc.) manually [34, 50‐66]; or (2) infer/learn the rules computationally from manually labeled sentences [67‐69].

Simple rules, such as co-occurrence, were used in the early efforts of PPI extraction. Co-occurrence assumes that two proteins likely interact with each other if they co-occurred in the same sentence/abstract [70, 71]. The drawback of these approaches is that the false positive rate of the methods tends to be quite high. Later studies used manually-specified rules, which can sometimes achieve much lower false positive rate, but often suffered from low recall rate [34, 50‐66].

Recently, machine learning solutions have been proposed to extract PPI information automatically. By learning the language rules from annotated texts, machine learning techniques can perform better than other methods in terms of both decreasing the false-positive rate and increasing the coverage [67‐69]. Huang et al.... [67] used a dynamic programming algorithm, similar to that used for sequence alignment, to extract patterns from sentences tagged by part-of-speech taggers. Kim et al.. [69] and Murugesan et al [72] used a kernel-based approach for learning genetic and protein-protein interaction patterns.

Although extensive studies have by far been carried out, existing methods only achieved partial success in small datasets [55, 58‐60, 67, 73] [54]. Kim et al [74] developed a web server: PIE, and tested their method on BioCreative dataset [38, 39, 75], achieving a reasonably good performance for a PPI article filtering task.

A machine learning based PPI extraction method was developed by Chowdhary et al. [73]. In this study, a novel methodology was developed based on Bayesian networks (BNs) for extracting PPI triplets (a PPI triplet consists of two protein names and the corresponding interaction word) from unstructured text. Various of features were extracted from sentences with potential PPIs, including preposition close to the protein names, the preposition close to the interaction word, the type of interaction word, the order of the words in the triplet, the distance between the first and second triplet word, the distance betwenn the second and third triplets words, existence of comma between triplet words, the distance of the comma to one of the triplet word, existence of the negative words such as “but”, “not”, “no” etc., existence of “which”, and number of interaction words in the sentence, in addition to other features. The method achieved an overall accuracy of 87% on a cross-validation test using manually annotated dataset with 2550 triplets. It was also showed, through extracting PPI triplets from a large number of PubMed abstracts, that the method was able to complement human annotations to extract large number of new PPIs from literature. Through manual validation of some of the predictions, they concluded that the current databases likely missed at least 130,000 PPIs [45]. The method was later applied to a large scale PPI extraction task for automatic knowledge discovery using an integrated bio-entity network made using heterogeneous types of bio-entities, including proteins, genes, diseases, gene oncology terms, pathways etc. [45]. A variation of the method that allows the extraction of directionality was also developed later using a mixture logistic model and ensemble approach [76]. A new PPI corpus, called PICAD (Protein Interaction Corpus with Annotated Directions), was manually curated with more than 1500 sentences and more than 10,000 triplet cases.

Thus far, there have been few methods that extract both the protein names and the interaction words at the same time. However, only the protein names are insufficient to understand PPIs. As a result, there is an urgent need to extract the PPI triplet (two different protein names and one interact word) in order to reveal how the proteins are interacted [77].

There is a practical issue in extracting PPI triplets if we omit the structure of a sentence. Ideally the PPI triplet appears in the order of (protein1---interaction word --- protein2), and one single sentence contains only one triplet; In practice, however, a PPI triplet ordered as (interaction word --- protein1 --- protein2) may occur, and for each sentence, multiple distinguished triplets may exist as well. In most cases, there is only one triplet that describes the true PPI. For example, the sentence in Fig. 1 contains four protein names (FKBP12-like is not considered as a protein name) PAHX, FKBP52, FKBP12, and FKBP52 (the second occurrence of FKBP52 in the sentence) and one interaction word interacts. There are five PPI triplets (Fig. 1), only one of the triplets correctly describes this specific PPI (triplet 1 in Fig. 1).

Recently Natural Language Processing (NLP) techniques have been utilized in many machine learning approaches [63‐66] to parse sentences into dependency trees or constituent trees, which could further be used in pattern matching or rule-based search. However, to our best knowledge, all the methods have to adopt some given rules/patterns. The given rules are typically rather general; therefore, they fail to represent all the patterns in the training sentences.

Bui et al. has developed a hybrid approach for extracting PPIs [78]. The method consists of two phases. First, the data were automatically categorized into subsets based on its semantic properties and candidate PPI pairs were extracted from these subsets. Second, support vector machines (SVMs) were applied to classify candidate PPI pairs using features specific for each subset. They obtained promising results on five benchmark datasets: AIMed, BioInfer, HPRD50, IEPA and LLL with F-scores ranging from 60 to 84%.

A comprehensive benchmark was developed for Kernel based PPI extraction methods by Tikk et al. [43]. In the work, the authors study whether the reported performance metrics are robust across different corpora and learning settings and whether the use of deep parsing actually leads to an increase in extraction quality. They concluded that for most kernels no sensible estimation of PPI extraction performance on new text is possible, given the current heterogeneity in evaluation data [43].

In this paper, we propose a method based on NLP and automatically learn rules/patterns to extract the PPI triplets from sentences. We then classify them as true or false with probabilities based on whether the interaction words correctly describe the interaction relationship between the two participant protein names.

Our method, GRGT, utilized the grammatical relationship among each Protein-Protein-Interaction triplet extracted by natural language processing (NLP) techniques and a graph theorem algorithm (shortest path algorithm) as feature to build a classifier. A dictionary of protein names and interaction words with their morphemes were built based on our previous study [28]. All interaction words in our dictionary were a single word, and were grouped manually into 22 categories by the similarity of their grammatical properties to reflect the fact that some interaction word can be used interchangeably without altering the sematic of the sentence.

Table 1 summarizes the datasets we used for testing the performance of our method (GRGT), including three benchmark datasets: HPRD50, IEPA, LLL, and a corpus we constructed: PICAD (protein interaction corpus with annotated directions). PICAD contains not only the interactions of protein pairs, but the directionality of interactions, which is important for analyzing biological network.

Table 2 shows the experiment results based on leave-one-out classification. The performance of top-performing methods in literature [23, 47, 48, 72] was also included for comparison. Compared with other methods, GRGT largely improved precision while maintaining comparable F-score, especially on IEPA and LLL. High precision is very important when the discovered (classified) results are going to be used as prior knowledge to guide experiment design. If one model has low precision, the results could be doubtful, and the researchers would receive incorrect information, which may provide false guidance for downstream studies. On the other hand, the lower recall rate of GRGT resulted from that most misclassified cases were false negatives, where true triplets cannot be matched to any known patterns. This would be acceptable since most true interactions (PPI triplets) tend to occur more than once in literature. The interaction will be extracted as long as one of PPI triplets is classified as true. A system with high precision can thus be used to more effectively extract PPIs from biomedical literature and deposit them into databases. In such task, the value of precision would be more important than the value of recall, and the tradeoff that decreases F-score with improving precision significantly is worthy. Figure 5 shows the precision-recall curves of our system on different datasets.

Recently, several studies introduced deep learning methods for PPI extraction [80‐83]. We also compare the performance of our method with Zhao et al. [83], which uses the same benchmark datasets. Our method again had better precision for the benchmark datasets compared with [83], while they get improvement in recall.

To further improve the performance of GRGT, the patterns can be simplified further so that more true triplets can be matched if they are similar to true patterns, but not exactly the same. The hierarchical structure of the typed dependencies can be used for this purpose. For example, nsubj (nominal subject) can be reduced to subj (subject) or even further to arg (argument). We need to balance recall and precision rate while doing this, as simplification would improve the recall rate, but with a cost of lowered precision rate. Some more experiments can be performed on various ways of reducing the exact patterns, and on how to combine the new relaxed patterns with our existing patterns by designing different decision trees to achieve better performance.

We further analyzed the extracted patterns (subgraphs), and in Table 3 we can see that not a lot of patterns appeared more than once, only about 10–20% of the extracted subgraphs appeared at least twice in the entire dataset, which leaves the coverage of triplets per sample relatively low, so that there is not much information can be borrowed from other triplets in the dataset. To further improve the performance, one can annotate more interaction cases to increase the size of the training set, which should significantly improve the recall rate of our method since we will have more coverage per pattern.

This method can be used to extract other relationships as well, as long as the triplet is well defined and the library for terms and interaction words are given.

Consistent with literature, Table 2 showed that deep learning approaches cannot beat traditional kernel-based or machine-learning methods all the time in PPI extraction task. The reasons would be 1) deep neural networks would not be beneficial without effectively large amount of training data and 2) deep neural networks are relatively difficult to train because of the large number of parameters. For a well-trained deep neural networks from large amount of training data, the performance may still get improved by combining with traditional feature-based machine learning methods [84]. The features we designed in this study can be applied to other machine learning methods, as well as be incorporated into deep learning methods. The current work is an expanded version of a previous study [82].

In this work, we developed a new NLP-based method, GRGT, for extracting the protein-protein interactions from biomedical literature. The performance of GRGT was demonstrated by comparing with top performing methods using benchmark datasets. GRGT obtained better precision, indicating that researchers can use PPIs extracted by GRGT as prior knowledge to guide experiment design with high confidence. We believe that GRGT will be a very useful tool for PPI-extraction task.