helpfn

Big Data Opportunities and Challenges for IR, Text Mining and NLP

Beth Plale School of Informatics and Computing

Indiana University Bloomington [email protected]

ABSTRACT

Big Data poses challenges for text analysis and natural language processing due to its characteristics of volume, veracity, and velocity of the data. The sheer volume in terms of numbers of documents challenges traditional local repository and index systems for large-scale analysis and mining. Computation, storage and data representation must work together to provide rapid access, search, and mining of the deep knowledge in the large text collection. Text under copyright poses additional barriers to computational access, where analysis has to be separated from human consumption of the original text. Data preprocessing, in most cases, remains a daunting task for big textual data particularly data veracity is questionable due to age of original materials. Data velocity is rate of change of the data but can also be the rate at which changes and corrections are made.

The HathiTrust Research Center (HTRC) provides new opportunities for IR, NLP and text mining research. HTRC is the research arm of HathiTrust, a consortium that stewards the digital library of content from research libraries around the country. With close to 11 million volumes in HathiTrust collection, HTRC aims to provide large-scale computational access and analytics to these text resources.

With the goal of facilitating scholar’s work, HTRC establishes a cyberinfrastructure of software, staff, and services to assist researchers and developers more easily process and mine large scale textual data effectively and efficiently. The primary users of HTRC are digital humanities, informatics, and librarians. They are of different research backgrounds and expertise and thus a variety of tools are made available to them.

In the HTRC model of computing, computation moves to the data, and services grow up around the corpus to serve the research community. In this manner, the architecture is cloud-based. Moving algorithms to the data is important because the copyrighted content must be protected, however, a side benefit is that the paradigm frees scholars from worrying about managing a large corpus of data.

The text analytics currently supported in HTRC is the SEASR suite of analytical algorithms (www.seasr.org). SEASR algorithms, which are written as workflows, include entity

extraction, tag cloud, topic modeling, NaiveBayes, Date Entities to Similie Timeline.

In this talk, I introduce the collections, architecture, and text analytics of HTRC, with a focus on the challenges of a BigData corpus and what that means for data storage, access, and large- scale computation.

HTRC is building a user community to better understand and support researcher needs. It opens many exciting possibilities for the NLP, text mining, IR types of research: with so large an amount of textual data and many candidate algorithms, with support for researcher contributed algorithms, many interesting research questions emerge and many interesting results are to follow.

Categories and Subject Descriptors H.3.7 [Information Storage and Retrieval]: Digital Libraries.

Keywords HathiTrust, Big Data Access, Information Retrieval, NLP, Text Mining and Analysis.

Biography

The author is a Professor at the School of Informatics and Computing at Indiana University Bloomington. She is the founder and director of the Data to Insight Center (D2I), which conducts and promotes interdisciplinary research and education in the preservation of scientific data, digital humanities, large-scale data management, data analytics, and visualization in geography, sustainability science, atmospheric science, informatics, computer science, and digital libraries domain spaces.

Dr. Plale has broad research and governance interest in long term preservation and access to scientific data, and computational access and analysis of large-scale data. Her specific research interests are in metadata and provenance capture, representation, and use; scientific data repositories; cyberinfrastructure; large- scale data analysis; and workflow systems. Plale is deeply engaged in interdisciplinary research and education and has substantive experience in developing stable and useable scientific cyberinfrastructure.

Professor Plale is the co-director of The HathiTrust Research Center (HTRC), enabling computational access for nonprofit and educational users to published works in the public domain and, in the future, on limited terms to works in-copyright from the HathiTrust digital library. Plale is general chair for the premier provenance conference, IPAW 2014, and premier distributed systems conference, HPDC 2014.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage, and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). Copyright is held by the author/owner(s). UnstructureNLP’13, October 28, 2013, San Francisco, CA, USA. ACM 978-1-4503-2415-1/13/10 http://dx.doi.org/10.1145/2513549.2514739

1