Predicting Disease-related Genes Using Biomedical Literature Based on GloVe Word Embedding

Ⅰ. Introduction

In the field of biology, considerable effort has been devoted to understanding the onset and treatment of diseases, where identifying disease-related genes is particularly important for understanding disease mechanisms and treating patients. For decades, researchers have proposed approaches for identifying disease-related genes. Because wet-lab experiments are time-consuming and expensive, performing research on all combinations of gene-disease associations is difficult [1]. Recently, studies using data mining-based computational methods have attempted to infer the relationships between genes and diseases and to predict novel associations. As the amount of available public data increases, computational methods can be used to acquire information more efficiently compared to traditional approaches [2]-[4].

Liu et al. proposed a method to construct a human brain-specific gene network by combining various genomic and proteomic data and to discover candidate genes sensitive to brain diseases [5]. They collected public microarray data, co-citation and protein-protein interaction (PPI) between brain genes for the human brain, and then built an integrated network based on the Bayesian statistical model. They identified crucial genes based on seed genes for specific diseases in the network and constructed a sub-network of those diseases using high-score interactions between genes. They predicted 46 susceptibility genes for Alzheimer’s disease and identified 23 genes as known genes related to this disease.

The goal of biomedical research is to discover novel and advanced biological information and apply this information to disease diagnosis, prevention, and treatment [6]. Text mining, which is the process of discovering interesting patterns or obtaining information from vast amounts of unstructured data, is an appropriate approach to accomplish such a task [7][8]. Text mining-based approaches can consistently acquire novel information considering that more than 100,000 sets of textual data are generated per day. In particular, various studies based on text mining have been proposed to discover gene-disease associations [9].

Kim et al. proposed a method to identify disease-related genes using Google search results and literature data [10]. From the literature, they collected sentences containing references to specific diseases and constructed a network by measuring the co-occurrence frequency of two genes. A node is a gene, and an edge is a co-occurrence frequency of two connecting genes in the network. They employed the Google search engine to measure the number of documents containing references to the two genes. They used the gene symbol as referenced in the literature as well as the full name of the gene derived from a Google search engine to measure the co-occurrence. They normalized the measured frequency and Google search results to a Z-score and then integrated and used two normalized scores to predict the candidate disease-related genes.

Word embedding (or word representation) is a natural language processing approach based on a neural network in which words are mapped to vectors of real numbers. Word embedding reflects the meaning and syntax of individual words in a low dimension [11]. Dimension reduction can help to overcome the sparsity of data, which is one of the limitations of traditional text mining approaches. Moreover, this approach is used to solve a variety of problems such as text classification and semantic analysis, as words in a vector space can be identified by word groups having similar meanings [12].

Koiwa et al. proposed a method of extracting disease-related genes from the literature using word2vec [13]. From the literature, they collected titles and abstracts about "schizophrenia", which is a psychiatric disorder from literature. They performed vectorization using a skip-gram, which is the one of the algorithms that execute word2vec. They then identified the top 10 words most similar to each known gene and predicted the gene symbols that emerged from the top words as candidate disease-related genes for schizophrenia. They predicted 62 novel disease-related genes using 461 known genes. They measured the classification performance using microarray data on schizophrenia to verify the validity of candidates.

Existing word embedding-based disease-related gene inference methods do not take into account the randomness of the initial embedding position, which is one of the characteristics of word embedding, so it is very difficult to reproduce the experiment. However, because the proposed method converges to a certain result through repeated experiments, it is possible to reproduce the experiment and it is more accurate than the existing method. The proposed method can predict genes that are highly related to disease among genes that are not predicted by the existing methods. Furthermore, the proposed method can continuously update gene-disease associations and discover novel relationships as considerable amount of bibliographic data are being produced daily.

Ⅱ. Materials and Methods

In this paper, we propose a method to predict novel disease-related genes using a vast amount of biological literature based on GloVe, one of the word embedding algorithms. GloVe integrates the characteristics of global matrix factorization (e.g., latent semantic analysis) and local context window methods (e.g., skip-gram) [14]. Global matrix factorization methods measure the word frequency in documents or sentences, where a row lists words and a column lists documents or sentences. These methods use statistical information but have difficulty in word analogy. Local context window methods predict target words by considering context words. These methods are strong for analogy but are limited by the fact that global frequency cannot be considered. GloVe is an algorithm that considers both the global frequency of words and the frequency within a context window. This method thus overcomes the limitations of existing methods. We collect sentences containing references to a specific disease from the abstracts of the literature and construct word vectors using GloVe. On the basis of known disease-related genes and the rest of the genes, we separate word vectors into gene vectors and seed vectors. We construct a gene-seed matrix by measuring the similarity between gene vectors and seed vectors. We calculate the sum of the similarities with the seed genes included in each gene in the gene-seed matrix and normalize the values of the genes to ranking. Since GloVe changes the result of the word vectors each time it is trained, these processes are repeated, and the total ranking is calculated and normalized to a value between 0 and 1. We predict candidate disease-related genes based on normalized total ranking. Fig. 1 shows the system overview of this paper.

Fig. 1.

System overview

2.4 Separating word vectors into gene and seed vectors

We separated word vectors that were built on sentences for a disease into gene and seed vectors. Gene vectors were composed of the same words as HGNC gene symbols among the words constituting word vectors. Seed vectors were constructed by the same words with known disease-related genes in OMIM for a disease among the words constituting word vectors. Gene symbols constituting gene vectors and seed vectors did not overlap each other. Fig. 2 shows the process of separating word vectors into gene and seed vectors for breast cancer.

Fig. 2.

Process of separating word vectors into gene and seed vectors

In Fig. 2, V is a dimension in the word vectors. "A1BG" is the gene included in the HGNC gene symbol and "BRCA2" is the gene associated with breast cancer collected from OMIM. "BRCA2" is a gene included in HGNC, but it is not included in gene vectors because it is a gene constituting seed vectors. The word "cancer" is not considered in constructing either gene or seed vectors because it does not appear in the HGNC and OMIM.

2.8 Predicting candidate disease-related genes

We constructed a classifier to predict disease-related genes using normalized total ranking. First, we sorted the normalized total ranking in descending order. We assigned "True" class label to gene with values greater than 0.5 and "False" to genes with values less than 0.5. Next, we measured the classification performance by increasing and decreasing the value by 0.05 starting at 0.5. We named the "True" class threshold as the positive threshold and "False" as negative threshold. Of all the combinations of positive and negative thresholds, we identified the thresholds with the highest AUC and used this case as a classifier to predict disease-related genes. Because the combination of thresholds with an AUC of 1 can occur when there is a small number of data while measuring classification performance, the combination with the highest AUC with a less than 1 is used as the thresholds of the classifier. Fig. 3 shows the process of identifying the optimal thresholds for constructing the classifier.

Fig. 3.

Process of identifying the optimal positive and negative threshold

In Fig. 3, G_a,…,G_z and NTR_a,…,NTR_z are genes and their values sorted in descending order of normalized total ranking. The red dotted line is the portion where the normalized total ranking is 0.5. The blue line is positive threshold and the green line is negative threshold. AUC_p,n is an AUC value for positive threshold p and negative threshold n. AUC was measured as the threshold was adjusted, and the combination with the highest AUC value other than 1 was used as the positive and negative thresholds for classifying disease-related genes. While adjusting the positive threshold and the negative threshold by 0.05 each, AUC was measured for all combinations, and the optimal threshold was specified for the highest AUC.

Ⅲ. Results and discussion

3.1 Experimental data

To evaluate the proposed method and predict novel disease-related genes, we selected cancers that could be applied to the proposed method out of 36 cancers with high frequency in 185 countries [28].

We chose diseases under the condition that 1) disease-related genes should be provided in OMIM and 2) more than five seeds should appear in word vectors constructed from disease sentences. In other words, we only selected disease for which there is a sufficient number of seeds. We collected sentences from PubMed that include lung cancer, breast cancer, prostate cancer, colon cancer (and colorectal cancer), stomach cancer, liver cancer, and bladder cancer among 36 cancers. Table 1 shows the number of sentences containing each disease.

Table 1.

Number of sentences for each disease

While constructing the word vectors, we applied the optimal parameters in Table 2 to model training process of GloVe.

Table 2.

GloVe parameter

3.2 Optimal positive and negative threshold

We tuned the positive and negative threshold for each disease to identify optimal thresholds and construct a classifier with highest AUC score. Table 3 shows the optimal threshold of the positive and negative, and AUC for the combination of the thresholds for each disease.

Table 3.

Optimal thresholds and AUC for each disease

For each disease, we assigned "True" label to genes with a value greater than the positive threshold and "False" label to genes with a value less than the negative threshold in Table 3.

3.3 Comparison with other methods

We compared the classification performance of the proposed method with those of other disease-related gene inference methods, such as DTMiner and RENET [29][30]. DTMiner is an approach to assessing how closely a gene-disease pair is related from a sentence in which genes and diseases appear simultaneously based on NER (Named entity recognition). RENET is a gene-disease inference method that considers classification at both sentence and document-level. RENET infers true associations among candidate gene-disease associations through the NER and RE (Relation extraction) steps. We constructed a confusion matrix based on the predicted true/false associations for each disease and curated gene-disease associations collected from DisGeNet, and measured the precision, recall and F-score for each method and compared the classification performance. Fig. 4 shows the confusion matrix, and Equation 5, 6, and 7 show the precision, recall, and F-score, respectively.

$\[ \mathrm{P}\mathrm{r}ecision=\frac{a}{a+b} \]$

(5)

$\[ Recall=\frac{a}{a+c} \]$

(6)

$\[ Fscore=\frac{2\times \mathrm{P}\mathrm{r}ecision\times Recall}{\mathrm{P}\mathrm{r}ecision+Recall} \]$

(7)

Fig. 4.

Confusion matrix

We confirmed that the proposed method has higher classification performance than the existing methods for seven diseases except liver cancer through Table 4. Liver cancer has much fewer disease sentences than other diseases as mentioned in Table 1, so we confirmed that liver cancer has low performance because the number of sentences is not enough to construct word vectors.

Table 4.

Precision, recall, and F-score for each method

We compared the classification performance of the proposed method and the network-based disease association inference method. PRINCIPLE prioritizes disease-related genes based on closeness in a PPI network for disease query, and this method is executed as a plug-in to the Cytoscape [31]-[33].

We compared the classification performance for six of the eight diseases that resulted in PRINCIPLE. In PRINCIPLE, we collected the top 100 genes and their scores most relevant to each disease, and we assigned DisGeNet curated gene-disease associations as class labels. Fig. 5 shows the AUC comparison between PRINCIPLE and the proposed method.

Fig. 5.

AUC comparison between PRINCIPLE and the proposed method

We confirmed that the classification accuracy of the proposed method for five of the six disease is high from Fig. 5.

PRINCIPLE requires disease-disease similarity and known gene-disease associations to construct the network. The proposed method has the advantage the no additional information about the disease or gene is needed because only known disease-related genes are considered to infer novel disease-related genes.

3.4 The number of predicted disease-related genes

We compared the number of disease-related genes for each disease predicted by the proposed method with RENET and DTMiner. We compared the number of genes inferred in each method for seven diseases except liver cancer, which had the lowest classification performance of the proposed method. Fig. 6 shows the Venn diagram for the predicted number of genes for seven diseases.

Fig. 6.

AUC comparison between PRINCIPLE and the proposed method

In Fig. 6, the red area represents the number of predicted disease-related genes by RENET, the blue represents DTMiner, and the green represents the proposed method. Because RENET and DTMiner are sentence-level approaches, we confirmed that most of the disease-related genes predicted by the two methods overlap. The proposed method predicts fewer number of genes in total, but more genes in its own.

3.5 Literature-based evaluation of novel disease-related genes

We made a literature-based evaluation of novel disease-related genes inferred for breast cancer. We predicted 11 novel genes for breast cancer. Table 5 shows the novel disease-related genes, descriptions, and PMIDs for breast cancer discovered by the proposed method.

Table 5.

Descriptions of novel disease-related genes for breast cancer

Ⅳ. Conclusion

We proposed a novel method for predicting disease-related genes from a vast amount of biomedical literature based on GloVe. We constructed word vectors using sentences containing disease and constructed a classifier based on the similarity between gene vectors and seed vectors.

We evaluated the proposed method with various diseases and derived candidate disease-related genes for breast cancer among these diseases.

We confirmed that the proposed method outperforms existing literature-based disease-related gene inference methods. Existing word embedding-based disease-related gene inference methods do not take into account the randomness of the initial embedding position, which is one of the characteristics of word embedding, so it is very difficult to reproduce the experiment. However, because the proposed method converges to a certain result through repeated experiments, it is possible to reproduce the experiment and it is more accurate than the existing method. The proposed method can predict genes that are highly related to disease among genes that are not predicted by the existing methods. Furthermore, the proposed method can continuously update gene-disease associations and discover novel relationships as considerable amount of bibliographic data are being produced daily.

However, the proposed method has the limitation that the experiment can be performed only for diseases with a sufficient number of seeds. Eight out of 36 cancers could be used for this paper. In future research, we will apply an approach to derive novel disease-related genes, regardless of the number of seed for the disease.

Acknowledgments

This research was supported by a grant from the National Research Foundation of Korea (NRF), funded by the Government of Korea (Ministry of Science, ICT and Future Planning) (NRF-2018R1A2B6006223).

References

• J. Zhou and F. Bo-quan, "The research on gene-disease association based on text-mining of PubMed", BMC bioinformatics, Vol. 19, No. 1, pp. 37, Dec. 2018. [https://doi.org/10.1186/s12859-018-2048-y]
• S. W. Shin, Y. E. Shin, G. U. Jang, and Y. M. Yoon, "Co-occurrence Based Drug-disease Relationship Inference with Genes as Mediators", Journal of KIIT, Vol. 16, No. 11, pp. 1-9, Nov. 2018. [https://doi.org/10.14801/jkiit.2018.16.11.1]
• G. U. Jang, Y. H. Hwang, M. Oh, T. K. Lee, and Y. M. "Novel Drug Similarity Measuring Method based on Text Mining", Journal of KIIT, Vol. 14, No. 7, pp. 127-137, Jul. 2016. [https://doi.org/10.14801/jkiit.2016.14.7.127]
• Y. Keon, H. Kim, J.Y. Choi, D. Kim , S.Y. Kim, and S. Kim, "Call Center Call Count Prediction Model by Machine Learning", Journal of Advanced Information Technology and Convergence, Vol. 8, No. 1, pp. 31-42, Jul. 2018. [https://doi.org/10.14801/JAITC.2018.8.1.31]
• B. Liu, T. Jiang, S. Ma, H. Zhao, J. Li, X. Jiang, and J. Zhang, "Exploring candidate genes for human brain diseases from a brain-specific gene network", Biochemical and biophysical research communications, Vol. 349, No. 4, pp. 1308-1314, Nov. 2006. [https://doi.org/10.1016/j.bbrc.2006.08.168]
• A. M. Cohen and W. R. Hersh, "A survey of current work in biomedical text mining", Briefings in bioinformatics, Vol. 6, No. 1, pp. 57-71, Mar. 2005. [https://doi.org/10.1093/bib/6.1.57]
• A. H. Tan, "Text mining: The state of the art and the challenges", In Proceedings of the PAKDD 1999 Workshop on Knowledge Disocovery from Advanced Databases, Vol. 8, pp. 65-70, Apr. 1999.
• J.U. Heu, "Korean Language Clustering Using Word2vec.", The Journal of Institute of Internet, Broadcasting and Communication, Vol. 18, No. 5 pp. 25-30, Oct. 2018.
• T. Ono and S. Kuhara, "A novel method for gathering and prioritizing disease candidate genes based on construction of a set of disease-related MeSH® terms", BMC bioinformatics, Vol. 15, No. 1, pp. 179, Dec. 2014. [https://doi.org/10.1186/1471-2105-15-179]
• J. Kim, H. Kim, Y. Yoon, and S. Park, "LGscore: a method to identify disease-related genes using biological literature and Google data", Journal of biomedical informatics, Vol. 54, pp. 270-282, Apr. 2015. [https://doi.org/10.1016/j.jbi.2015.01.003]
• O. Melamud, J. Goldberger, and I. Dagan, "Context2vec: Learning generic context embedding with bidirectional lstm", In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, Berlin, Germany, pp. 51-61, Aug. 2016. [https://doi.org/10.18653/v1/K16-1006]
• J. Lilleberg, Y. Zhu, and Y. Zhang, "Support vector machines and word2vec for text classification with semantic features", In 2015 IEEE 14th International Conference on Cognitive Informatics & Cognitive Computing (ICCI* CC), Beijing, China, pp. 136-140, Jul. 2015. [https://doi.org/10.1109/ICCI-CC.2015.7259377]
• T. Koiwa and H. Ohwada, "Extraction of disease-related genes from PubMed paper using word2vec", In Proceedings of the 8th International Conference on Computational Systems-Biology and Bioinformatics, Nha Trang City Viet Nam, pp. 46-49, Dec. 2017. [https://doi.org/10.1145/3156346.3156355]
• J. Pennington, R. Socher, and C. D. Manning, "Glove: Global vectors for word representation", In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), Doha, Qatar, pp. 1532-1543, Oct. 2014. [https://doi.org/10.3115/v1/D14-1162]
• K. Canese and S. Weis, "PubMed: the bibliographic database", The NCBI Handbook, pp. 2-1, Mar. 2013.
• L. M. Schriml, C. Arze, S. Nadendla, Y. W. W. Chang, M. Mazaitis, V. Felix, G. Feng, and W. A. Kibbe, "Disease Ontology: a backbone for disease semantic integration", Nucleic acids research, Vol. 40, No. D1, pp. D940-D946, Jan. 2012. [https://doi.org/10.1093/nar/gkr972]
• W. A. Kibbe, C. Arze, V. Felix, E. Mitraka, E. Bolton, G. Fu, C. J. Mungall, J. X. Binder, J. Malone, D. Vasant, and H. Parkinson, "Disease Ontology 2015 update: an expanded and updated database of human diseases for linking biomedical knowledge through disease data", Nucleic acids research, Vol. 43, No. D1, pp. D1071-D1078, Jan. 2015. [https://doi.org/10.1093/nar/gku1011]
• S. Povey, R. Lovering, E. Bruford, M. Wright, M. Lush, and H. Wain, "The HUGO gene nomenclature committee (HGNC)", Human genetics, Vol. 109, No. 6, pp. 678-680, Dec. 2001. [https://doi.org/10.1007/s00439-001-0615-0]
• A. Hamosh, A. F. Scott, J. S. Amberger, C. A. Bocchini, and V. A. McKusick, "Online Mendelian Inheritance in Man (OMIM), a knowledgebase of human genes and genetic disorders", Nucleic acids research, Vol. 33, pp. D514-D517, Jan. 2005. [https://doi.org/10.1093/nar/gki033]
• J. Piñero, J. M. Ramírez-Anguita, J. Saüch-Pitarch, F. Ronzano, E. Centeno, F. Sanz, and L. I. Furlong, "The DisGeNET knowledge platform for disease genomics: 2019 update", Nucleic acids research, Vol. 48, No. D1, pp. D845-D855, Jan. 2020. [https://doi.org/10.1093/nar/gkz1021]
• J. Piñero, À. Bravo, N. Queralt-Rosinach, A. Gutiérrez-Sacristán, J. Deu-Pons, E. Centeno, J. García-García, F. Sanz, and L. I. Furlong, "DisGeNET: a comprehensive platform integrating information on human disease-associated genes and variants", Nucleic acids research, pp. gkw943, Oct. 2016. [https://doi.org/10.1093/nar/gkw943]
• D. Gillick, "Sentence boundary detection and the problem with the US", In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Boulder, Colorado, pp. 241-244, Jun. 2009. [https://doi.org/10.3115/1620853.1620920]
• Sentence Boundary Detection (SBD), https://github.com/lukeorland/splitta, . [accessed: Sep. 09, 2019]
• W. J. van Hoeve and I. Katriel, "Global constraints", In Foundations of Artificial Intelligence, Vol. 2, pp. 169-208, Jan. 2006. [https://doi.org/10.1016/S1574-6526(06)80010-6]
• D. Ciresan, A. Giusti, L. M. Gambardella, and J. Schmidhuber, "Deep neural networks segment neuronal membranes in electron microscopy images", In Advances in neural information processing systems, pp. 2843-2851, 2012.
• B. Chiu, G. Crichton, A. Korhonen, and S. Pyysalo, "How to train good word embeddings for biomedical NLP", In Proceedings of the 15th workshop on biomedical natural language processing, pp. 166-174, Aug. 2016. [https://doi.org/10.18653/v1/W16-2922]
• text2vec.org
• F. Bray, J. Ferlay, I. Soerjomataram, R. L. Siegel, L. A. Torre, and A. Jemal, "Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries", CA: a cancer journal for clinicians, Vol. 68, No. 6, pp. 394-424, Sep. 2018. [https://doi.org/10.3322/caac.21492]
• D. Xu, M. Zhang, Y. Xie, F. Wang, M. Chen, K. Q. Zhu, and J. Wei, "DTMiner: identification of potential disease targets through biomedical literature mining", Bioinformatics, Vol. 32, No. 23, pp. 3619-3626, Dec. 2016.
• Y. Wu, R. Luo, H. C. Leung, H. F. Ting, and T. W. Lam, "Renet: A deep learning approach for extracting gene-disease associations from literature", In International Conference on Research in Computational Molecular Biology, Padua, Italy, pp. 272-284, May 2019. [https://doi.org/10.1007/978-3-030-17083-7_17]
• O. Vanunu, O. Magger, E. Ruppin, T. Shlomi, and R. Sharan, "Associating genes and protein complexes with disease via network propagation", PLoS computational biology, Vol. 6, No. 1, pp. 1-9, Jan. 2010. [https://doi.org/10.1371/journal.pcbi.1000641]
• A. Gottlieb, O. Magger, I. Berman, E. Ruppin, and R. Sharan, "PRINCIPLE: a tool for associating genes with diseases via network propagation", Bioinformatics, Vol. 27, No. 23, pp. 3325-3326, Dec. 2011. [https://doi.org/10.1093/bioinformatics/btr584]
• P. Shannon, A. Markiel, O. Ozier, N. S. Baliga, J. T. Wang, D. Ramage, N. Amin, B. Schwikowski, and T. Ideker, "Cytoscape: a software environment for integrated models of biomolecular interaction networks", Genome research, Vol. 13, No. 11, pp. 2498-2504, Nov. 2003. [https://doi.org/10.1101/gr.1239303]