Project

Back to overview

Fundamentals of Parallel Programming for Platform-as-a-Service Clouds

Applicant Binder Walter
Number 153560
Funding scheme Project funding (Div. I-III)
Research institution Software Institute Facoltà di scienze informa Universit`a della Svizzera italiana
Institution of higher education Università della Svizzera italiana - USI
Main discipline Information Technology
Start/End 01.11.2014 - 31.10.2018
Approved amount 451'785.00
Show all

Keywords (5)

Parallel Programming Models; Platform as a Service (PaaS); Service-Oriented Computing; Cloud Computing; Runtime Systems

Lay Summary (German)

Lead
In diesem Projekt erforschen wir neue Programmiermodelle für Platform-as-a-Service (PaaS) Clouds, die durch Task-Parallelismus und asynchrone Kommunikation gekennzeichnet sind. In dem resultierenden Programmiermodell wird der Entwickler die Möglichkeit haben, jede parallele Anwendung auf hohem Abstraktionsniveau zu spezifizieren, während das PaaS-System die Ausführung automatisch parallelisiert und optimiert, den Ressourcenverbrauch kontrolliert, sowie die Konsistenz der Daten sicherstellt.
Lay summary
Eine der wichtigsten Eigenschaften von Cloud Computing ist der Parallelismus aufgrund der virtualisierten Ausführungsumgebungen, der gleichzeitigen Client-Anfragen, sowie der asynchronen und verteilten Interaktionen innerhalb der Cloud und mit externen Systemen und Datenquellen. Bestehende PaaS Clouds bieten keine Unterstützung für allgemeine parallele Programmiermodelle und verursachen bedeutende architektonische Einschränkungen, wie Anwendungen den zugrundeliegenden Parallelismus nutzen können.

Um diese Einschränkungen zu überwinden, werden wir ein neues paralleles PaaS-Programmiermodell entwickeln, welches auf funktionaler und ereignis-basierter Programmierung aufsetzt, welche sich gut für die Entwicklung von Web-Anwendungen eignen. Wir werden relaxed Konsistenzmodelle erkunden, um Synchronisation zu reduzieren. Weiters werden wir spezielle Kommunikationsprimitiven für Streaming Berechnungen und deren Zusammensetzung erforschen. Auf der Ebene des Laufzeitsystems werden wir Mechanismen zur genauen Ressourcenkontrolle untersuchen, wobei wir plattformunabhängige Ressourcenverbrauchsmetriken verwenden werden.
Direct link to Lay Summary Last update: 19.05.2014

Lay Summary (English)

Lead
In this project we explore novel programming models for Platform-as-a-Service (PaaS) clouds that are characterized by task parallelism as well as by asynchronous communication. In the resulting programming model, the developer will have the ability to run any parallel application using high-level abstractions, while the PaaS runtime system will automatically parallelize and optimize the execution, control resource consumption, and ensure data consistency.
Lay summary
One of the main characteristics of cloud computing is the pervasiveness of parallelism, which emerges from the underlying virtualized execution environment, from concurrent client requests, and from asynchronous and distributed interactions within the cloud and with external systems and data sources. Existing PaaS clouds lack support for general parallel programming models and impose significant architectural constraints on how the services can leverage the underlying parallelism. As a consequence, developers who want to run parallel applications in cloud platforms taking advantage of their elasticity and pay-as-you-go resource provisioning features must resort to Infrastructure-as-a-Service clouds, which provide them only with lower-level abstractions that make it more difficult to build complex applications.

To overcome these limitations, we will design a new parallel PaaS cloud programing model providing new computation, storage, and communication abstractions. Our investigation will focus on functional and event-based programming, which have become popular for developing Web applications. In this context, we will relax the assumption of share-nothing message-based coordination and explore different consistency models for mutable shared state. Our programming model will support relaxed consistency models to reduce the need for synchronization. It will also offer dedicated communication primitives for streaming computations and their composition into pipelines. At the level of the runtime system, we will investigate mechanisms for fine-grained resource accounting and control, focusing on platform-independent resource consumption metrics. We will also explore fundamental concurrency management mechanisms within the PaaS runtime system to enable scalable service execution in the presence of time-varying request workloads.
Direct link to Lay Summary Last update: 19.05.2014

Responsible applicant and co-applicants

Employees

Publications

Publication
Decentralized Computation Offloading on the Edge with Liquid WebWorkers
Gallidabino Andrea, PautassoCesare (2018), Decentralized Computation Offloading on the Edge with Liquid WebWorkers, in Proceedings of the 18th International Conference On Web Engineering (ICWE 2018), Cáceres, SpainSpringer International Publishing, Cham.
Understanding Task Granularity on the JVM: Profiling, Analysis, and Optimization
Rosà Andrea, Rosales Eduardo, Schiavio Filippo, Binder Walter (2018), Understanding Task Granularity on the JVM: Profiling, Analysis, and Optimization, in Conference Companion of the 2nd International Conference, Nice, FranceACM, New York, NY, USA.
The Liquid User Experience API
Gallidabino Andrea, Pautasso Cesare (2018), The Liquid User Experience API, in Companion of the The Web Conference 2018, Lyon, FranceACM, Republic and Canton of Geneva, Switzerland.
Analyzing and Optimizing Task Granularity on the JVM
Rosà Andrea, Rosales Eduardo, Binder Walter (2018), Analyzing and Optimizing Task Granularity on the JVM, in the 2018 International Symposium, Vienna, AustriaACM, New York, NY, USA.
Liquid Web Applications: ICWE2017 Tutorial
Gallidabino Andrea, Mikkonen Tommi, Mäkitalo Niko, Pautasso Cesare, Systä Kari, Taivalsaari Antero, Voutilainen Jari (2018), Liquid Web Applications: ICWE2017 Tutorial, in Current Trends in Web Engineering, Springer International Publishing, Cham.
tgp: A Task-Granularity Profiler for the Java Virtual Machine
Rosales Eduardo, Rosà Andrea, Binder Walter (2017), tgp: A Task-Granularity Profiler for the Java Virtual Machine, in 2017 24th Asia-Pacific Software Engineering Conference (APSEC), NanjingIEEE, Nanjing, Jiangsu, China.
Accurate reification of complete supertype information for dynamic analysis on the JVM
Rosà Andrea, Rosales Eduardo, Binder Walter (2017), Accurate reification of complete supertype information for dynamic analysis on the JVM, in ACM SIGPLAN Notices, 52(12), 104-116.
Accurate Reification of Complete Supertype Information for Dynamic Analysis on the JVM
Rosà Andrea, Rosales Eduardo, Binder Walter (2017), Accurate Reification of Complete Supertype Information for Dynamic Analysis on the JVM, in the 16th ACM SIGPLAN International Conference, Vancouver, BC, CanadaACM, New York, NY, USA.
Architecting Liquid Software
Gallidabino Andrea, Pautasso Cesare, Ilvonen Ville, Mikkonen Tommi, Systä Kari, Voutilainen Jari-Pekka, Taivalsaari Antero (2017), Architecting Liquid Software, in Journal of Web Engineering, 16, 433-470.
Maturity Model for Liquid Web Architectures
Gallidabino Andrea, Pautasso Cesare (2017), Maturity Model for Liquid Web Architectures, in Web Engineering, Rome, ItalySpringer International Publishing, Cham.
GEMs: shared-memory parallel programming for Node.js
Bonetta Daniele, Salucci Luca, Marr Stefan, Binder Walter (2016), GEMs: shared-memory parallel programming for Node.js, in ACM SIGPLAN Notices, 51(10), 531-547.
Generic messages: capability-based shared memory parallelism for event-loop systems
Salucci Luca, Bonetta Daniele, Marr Stefan, Binder Walter (2016), Generic messages: capability-based shared memory parallelism for event-loop systems, in ACM SIGPLAN Notices, 51(8), 1-2.
GEMs: Shared-memory Parallel Programming for Node.js
Bonetta Daniele, Salucci Luca, Marr Stefan, Binder Walter (2016), GEMs: Shared-memory Parallel Programming for Node.js, in Proceedings of OOPSLA '16, ACM, New York, NY, USA.
Lightweight Multi-language Bindings for Apache Spark
Salucci Luca, Bonetta Daniele, Binder Walter (2016), Lightweight Multi-language Bindings for Apache Spark, in 22nd International Conference on Parallel and Distributed Computing, Grenoble, France, August 24-26,, Springer-Verlag, New York, NY, USA.
Efficient Embedding of Dynamic Languages in Big-data Analytics
Salucci Luca, Bonetta Daniele, Binder Walter (2016), Efficient Embedding of Dynamic Languages in Big-data Analytics, in ICDCS Workshops 2016, Nara, JapanIEEE, Nara, Japan.
AutoBench: Finding Workloads That You Need Using Pluggable Hybrid Analyses
Zheng Yudi, Rosà Andrea, Salucci Luca, Li Yao, Sun Haiyang, Javed Omar, Bulej Lubomir, Chen Lydia Y., Qi Zhengwei, Binder Walter (2016), AutoBench: Finding Workloads That You Need Using Pluggable Hybrid Analyses, in 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Suita, JapanIEEE, Osaka, Japan.
Generic Messages: Capability-based Shared Memory Parallelism for Event-loop Systems
Salucci Luca, Bonetta Daniele, Marr Stefan, Binder Walter (2016), Generic Messages: Capability-based Shared Memory Parallelism for Event-loop Systems, in Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, ACM, New York, NY, USA.
Deploying Stateful Web Components on Multiple Devices with Liquid.js for Polymer
Gallidabino Andrea, Pautasso Cesare (2016), Deploying Stateful Web Components on Multiple Devices with Liquid.js for Polymer, in Proceedings of the 19th International ACM Sigsoft Symposium on Component-Based Software Engineering, Venice, Italy.
Migrating and Pairing Recursive Stateful Components Between Multiple Devices with Liquid.js for Polymer
Gallidabino Andrea (2016), Migrating and Pairing Recursive Stateful Components Between Multiple Devices with Liquid.js for Polymer, in Proceedings of the 16th International Conference on Web Engineering (ICWE2016), Lugano, SwitzerlandSpringer, Charm.
On the Architecture of Liquid Software: Technology Alternatives and Design Space
Gallidabino Andrea, Pautasso Cesare, Ilvonen Ville, Mikkonen Tommi, Systä Kari, Voutilainen Jari-Pekka, Taivalsaari Antero (2016), On the Architecture of Liquid Software: Technology Alternatives and Design Space, in Proceedings of the 13th Working IEEE/IFIP Conference on Software Architecture (WICSA 2016), Venice, Italy.
The Liquid.js Framework for Migrating and Cloning Stateful Web Components across Multiple Devices
Gallidabino Andrea, Pautasso Cesare (2016), The Liquid.js Framework for Migrating and Cloning Stateful Web Components across Multiple Devices, in Proceedings of the 25th International World Wide Web conference, Montreal, Montreal, Canada.
lpt: a Tool for Tuning the Level of Parallelism of Spark Applications
RosalesEduardo, RosàAndrea, BinderWalter, lpt: a Tool for Tuning the Level of Parallelism of Spark Applications, in 25th Asia-Pacific Software Engineering Conference (APSEC), IEEE, Nara, Japan.

Collaboration

Group / person Country
Types of collaboration
IaaS, University of Stuttgart Germany (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
- Research Infrastructure
- Exchange of personnel
EMC/UC Berkeley United States of America (North America)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Liquid Software Group, Tampere University of Technology Finland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
IBM Zurich Research Lab Switzerland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
- Research Infrastructure
- Exchange of personnel
Nokia Finland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Credit Suisse Switzerland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Oracle Labs Austria (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
- Research Infrastructure
- Exchange of personnel

Scientific events

Active participation

Title Type of contribution Title of article or contribution Date Place Persons involved
Programming 2018 Talk given at a conference Understanding Task Granularity on the JVM: Profiling, Analysis, and Optimization 09.04.2018 Nice, France Binder Walter; Rosales Rosero Edgar Eduardo;
CGO 2018 Talk given at a conference Analyzing and Optimizing Task Granularity on the JVM 24.02.2018 Vienna, Austria Binder Walter; Rosales Rosero Edgar Eduardo;
APSEC 2018 Talk given at a conference lpt: a Tool for Tuning the Level of Parallelism of Spark Applications 04.12.2017 Nara, Japan Binder Walter; Rosales Rosero Edgar Eduardo;
APSEC 2017 Talk given at a conference tgp: A Task-Granularity Profiler for the Java Virtual Machine 04.12.2017 Nanjing, Jiangsu, China Binder Walter; Rosales Rosero Edgar Eduardo;
EuroPar 2016 Talk given at a conference Lightweight Multi-language Bindings for Apache Spark 22.08.2016 Grenoble, France Salucci Luca; Binder Walter;
ICDCS 2016 Talk given at a conference Efficient Embedding of Dynamic Languages in Big-Data Analytics 27.06.2016 Nara, Japan Salucci Luca; Binder Walter;
ICWE 2016 Talk given at a conference Migrating and Pairing Recursive Stateful Components between Multiple Devices with Liquid.js for Polymer 06.06.2016 Lugano, Switzerland Gallidabino Andrea;
WWW 2016 Talk given at a conference The Liquid.js Framework for Migrating and Cloning Stateful Web Components across Multiple Devices 11.04.2016 Montreal, Canada Gallidabino Andrea; Pautasso Cesare;
WICSA 2016 Talk given at a conference On the Architecture of Liquid Software: Technology Alternatives and Design Space 05.04.2016 Venice, Italy Pautasso Cesare; Gallidabino Andrea;
CBSE 2016 Talk given at a conference Deploying Stateful Web Components on Multiple Devices with Liquid.js for Polymer 05.04.2016 Venice, Italy Pautasso Cesare; Gallidabino Andrea;
PPoPP 2016 Poster Generic Messages: Capability-based Shared Memory Parallelism for Event-loop Systems 12.03.2016 Barcelona, Spain Binder Walter; Salucci Luca;
APSys 2015 Poster A big-data Analytics Framework with Efficient Support for Dynamic Languages 27.07.2015 Tokyo, Japan Salucci Luca; Binder Walter;
ICWE 2015 Poster Towards Liquid Web Applications 23.06.2015 Rotterdam, Netherlands Gallidabino Andrea; Pautasso Cesare;


Self-organised

Title Date Place
VMM 2016 01.09.2016 Lugano, Switzerland
JTRES 2016 29.08.2016 Lugano, Switzerland
PPPJ 2016 29.08.2016 Lugano, Switzerland
ICWE 2016 06.06.2016 Lugano, Switzerland
Liquid Workshop 2016 06.06.2016 Lugano, Switzerland

Awards

Title Year
Best Demo Award for the demo "Migrating and Pairing Recursive Stateful Components between Multiple Devices with Liquid.js for Polymer" 2016

Associated projects

Number Title Start Funding scheme
133835 The Datacenter Observatory 01.10.2012 R'EQUIP
127386 SOSOA: Self-Organizing Service-Oriented Architectures 01.02.2010 Sinergia
141002 LoadOpt - Workload Characterization and Optimization for Multicore Systems 01.03.2013 Project funding (Div. I-III)

Abstract

One of the main characteristics of cloud computing is the pervasiveness of parallelism, which naturally emerges 1) from the underlying virtualized execution environment, 2) from the concurrent client requests to services deployed in the cloud, and 3) from asynchronous and distributed interactions within the cloud and with external systems and data sources. Existing Platform-as-a-Service (PaaS) clouds provide developers with high-level abstractions, which can be used to build scalable and elastic Web services and Web applications, but lack support for general parallel programming models. For example, even though existing PaaS cloud programming models support the deployment of traditional, multi-tier Web applications, they impose significant architectural constraints on how the services can leverage the underlying parallelism. As a consequence, developers who want to run parallel applications in cloud platforms to take advantage of their elasticity and pay-as-you-go resource provisioning features must resort to Infrastructure-as-a-Service (IaaS) clouds, which provide them with lower-level abstractions (such as operating system threads and locks, or processes and message queues). Such low-level abstractions make it more difficult to build complex applications, which, for example, require support for large-scale data processing and may involve pipelined computations over infinite streams of data.To overcome the limitations of existing cloud programming models, in this project we address the following fundamental research questions: "How can we increase the abstraction level of the programming model for PaaS cloud development?", "Which programming models allow for running any parallel application in the cloud?", and "What are the guarantees the developer can be given in terms of performance, scalability, elasticity, reliability, and data consistency?". We consider these questions fundamental challenges for simplifying (and reducing the cost of) PaaS cloud development. Our goal is to explore novel programming models characterized by task parallelism as well as by asynchronous communication. In the resulting programming model, the developer will have the ability to run any parallel application using high-level abstractions (in contrast to OS-level abstractions), while the PaaS runtime system - which is also subject of our investigation - will automatically parallelize and optimize the execution, control resource consumption, and ensure data consistency.Concretely, in this project we will design a parallel programing model for PaaS cloud development providing new computation, storage, and communication abstractions. Our investigation will focus on functional and event-based programming, which have become popular for developing Web services and Web applications. In this context, we will relax the assumption of share-nothing message-based coordination and explore different consistency models for mutable shared state. Ensuring transactional properties (i.e., atomicity, consistency, and isolation) for all operations on shared state is expensive and would hinder scalability. Hence, our programming model will support relaxed consistency models to reduce the need for synchronization. Our programming model will also offer dedicated communication primitives for streaming computations and their composition into pipelines. At the level of the runtime system, we will investigate mechanisms for fine-grained resource accounting and control, focusing on platform-independent resource consumption metrics. We will also explore fundamental concurrency management mechanisms within the PaaS runtime system to enable scalable service execution in the presence of time-varying request workloads. Finally, we will develop a novel runtime system capable of automatically reconfiguring the application deployment for performance optimization. To this end, we will investigate auto-tuning mechanisms to achieve elastic scalability with respect to the number of concurrent clients, taking given service-level objectives (i.e., quality-of-service guarantees) into account. Our approach will be thoroughly evaluated both in terms of expressiveness of the new programming model and in terms of performance and scalability of the runtime system. For the latter evaluation, the platform developed in this project will be deployed on a heterogeneous cluster of modern multicore machines.
-