Back to overview

Mining Structured Data in Natural Language Artifacts with Island Parsing

Type of publication Peer-reviewed
Publikationsform Original article (peer-reviewed)
Author Bacchelli Alberto, Cleve Anthony, Mocci Andrea, Lanza Michele,
Project PROBE - Live Actionable Software Analytics
Show all

Original article (peer-reviewed)

Journal Science of Computer Programming (SCP)
Publisher Elsevier
Volume (Issue) 150
Page(s) 31 - 55
Title of proceedings Science of Computer Programming (SCP)

Open Access

Type of Open Access Repository (Green Open Access)


Software repositories typically store data composed of structured and unstructured parts. Researchers mine this data to empirically validate research ideas and to support practitioners’ activities. Structured data (e.g., source code) has a formal syntax and is straightforward to analyze; unstructured data (e.g., documentation) is a mix of natural language, noise, and snippets of structured data, and it is harder to analyze. Especially the structured content (e.g., code snippets) in unstructured data contains valuable information. Researchers have proposed several approaches to recognize, extract, and analyze structured data embedded in natural language. We analyze these approaches and investigate their drawbacks. Subsequently, we present two novel methods, based on scannerless generalized LR (SGLR) and Parsing Expression Grammars (PEGs), to address these drawbacks and to mine structured fragments within unstructured data. We validate and compare these approaches on development emails and Stack Overflow posts with Java code fragments. Both approaches achieve high precision and recall values, but the PEG-based one achieves better computational performances and simplicity in engineering.