Back to overview

Efficient Embedding of Dynamic Languages in Big-data Analytics

Type of publication Peer-reviewed
Publikationsform Proceedings (peer-reviewed)
Author Salucci, Luca; Bonetta, Daniele; Binder, Walter
Project Fundamentals of Parallel Programming for Platform-as-a-Service Clouds
Show all

Proceedings (peer-reviewed)

Title of proceedings ICDCS Workshops 2016
Place Nara, Japan
DOI 10.1109/ICDCSW.2016.40


Over the last years several frameworks have emerged in the field of big-data analytics. Recent frameworks expose a developer-friendly API via dynamic languages such as Python. Unfortunately, the integration of dynamic languages with the parallel and distributed runtime of such frameworks is cumbersome, as it requires the integration of two or more language virtual machines via inter-process communication, introducing communication overheads and reducing the benefits of the shared memory present in modern multicore machines. In this paper we highlight the advantages of hosting multiple language runtimes in a single shared (language) virtual machine, and the possible performance gain of such an approach in the context of the Apache Spark framework.