Data Mining the Bible

Abstract

In todays’ society, technological changes are oscillating at lightspeed all over the world. No matter the business world or the industrial world are all highly depended on software and technological means. Big data or data analysis is one of the hottest topics nowadays. Data mining is unlike the traditional text keyword searching method in the past decades, it is a process of sorting and digging deep through a large volume of data set by which hidden relationship or connections and similar pattern within a collection of documents are extracted. And ultimately it acts as a tool which allows us to predict the future trend of the outcomes. The purpose of this project is to perform data mining on Bible (mainly focus on "The Book of Psalms" and "The Book Proverbs"). The project is divided into two main sections, which are the 1) Topic Modelling using Python and 2) Clustering using R. Experiments were conducted using the LDA (Latent Dirichlet Allocation) topic modelling algorithm together with the affinity propagation clustering algorithm. The result of the experiments showed the relationship within each chapter of the two book and clustered similar chapters together.