Projekt

Zurück zur Übersicht

BenchFlow - A Benchmark for Workflow Management Systems

Titel Englisch BenchFlow - A Benchmark for Workflow Management Systems
Gesuchsteller/in Pautasso Cesare
Nummer 145062
Förderungsinstrument Projektförderung (Abt. I-III)
Forschungseinrichtung Facoltà di scienze informatiche Università della Svizzera italiana
Hochschule Università della Svizzera italiana - USI
Hauptdisziplin Informatik
Beginn/Ende 01.10.2013 - 30.09.2017
Bewilligter Betrag 230'700.00
Alle Daten anzeigen

Keywords (8)

Business Process Management, Workflow Engine, Benchmarking, Performance Evaluation, Service Oriented Architectures, Middleware, WS-BPEL, Web Service Composition

Lay Summary (Deutsch)

Lead
Ziel des BenchFlow Projekts ist die Erstellung eines ersten Benchmarks mit dem Workflow Engines (Prozess Engines) untersucht und verglichen werden können.
Lay summary

Um das Projekt sinnvoll durchführen zu können, ist es wegen der Vielfalt von existierenden Sprachen Engines notwendig, uns zu beschränken. Wir konzentrieren uns zunächst auf Web Services Composition Engines, die den WS-BPEL (Business Process Execution Language) Standard unterstützen. Workflow Systeme zeichnen sich aus durch eine enorme Komplexität der Architektur und eine Vielzahl von Parametern, die die Leistung des Systems beeinflussen. Benchmarking von Workflow Systemen erfordert deshalb die Entwicklung von neuartigen Techniken zur Leistungsmessung. Diese Benchmarks reihen sich neben die existierenden Benchmarks ein, wie TPC für Datenbanksysteme und SPEC für Programmiersprachen, die gut verstanden sind und die zu signifikanten Leistungsverbesserungen bei Datenbanken, Compilern und Prozessoren führten. Workflowsysteme sind eine Platform für service-orientierte Anwendungen. Deren Leistungsfähigkeit hängt im Wesentlichen von von der Leistungsfähigkeit des  Workflowsystems und der aufgerufenen Services (die jedoch außerhalb der Kontrolle des Workflowsystems liegt) ab. Wir planen einen model-getriebenen, rekursiven Ansatz für die Benchmarks zu wählen, bei denen der Einfluss der externen Services dadurch eliminiert wird, dass sie durch Prozesse ersetzt werden. Die Prozesse des Benchmarks werden aus den wesentlichen Eigenschaften vorhandener Geschäftsprozessen aufgebaut.  Wir werden die Vielzahl von Metriken zur Leistungsmessung zu einem Satz von Leistungsindikatoren reduzieren, der sowohl den einfachen Vergleich verschiedener Workflowsysteme als auch den Vergleich verschiedener Konfigurationen eines Workflowsystems ermöglicht.

Direktlink auf Lay Summary Letzte Aktualisierung: 15.10.2013

Lay Summary (Italienisch)

Lead
Il progetto BenchFlow si occupa di definire uno dei primi benchmark per studiare le prestazioni di sistemi di esecuzione di processi e flussi di lavoro aziendali (BPM). Questi sistemi si trovano alla base di ogni moderna architettura orientata ai servizi (SOA) ed e' quindi fondamentale poterne analizzare e confrontare le prestazioni.
Lay summary
Considerando il grande numero e la grande variabilita' dei linguaggi che sono stati proposti per modellare processi aziendali, il progetto si concentra su sistemi di composizione di servizi Web che usano lo standard chiamato WS-BPEL (Business Process Execution Language). Questi sistemi sono caratterizzati da una enorme complessita' architetturale e da un grande numero di parametri di configurazione che potenzialmente influiscono sulle loro prestazioni. Definire un benchmark per questa classe di sistemi richiede dunque di sviluppare nuove tecniche per misurare le loro prestazioni e per metterli sotto pressione. Ci aspettiamo che il risultato del progetto permetta di ottenere un riferimento che possa servire a provocare un significativo passo in avanti per lo sviluppo di sistemi sempre piu' efficienti, affidabili e scalabili. La performance di queste applicazioni dipende principalmente dalla performance del middleware (che si occupa di eseguire i processi e di scambiare messaggi con i servizi che partecipano al compimento dei processi) e dalla performance dei servizi stessi (che sono normalmente fuori dal controllo del sistema di workflow). Per gestire questo aspetto abbiamo in programma un approccio ricorsivo al design del benchmark per sostituire i servizi esterni con altrettanti processi in modo da ridurre il loro impatto sulle prestazioni. Inoltre costruiremo dei processi rappresentativi e realistici basandoci su casi di studio presi dall'industria. Ci proponiamo infine di studiare quali metriche siano le piu' utili per misurare, osservare e confrontare le prestazioni in modo da poter ricondurre il risultato del benchmark ad un solo valore che riassuma in modo equo le varie caratteristiche in cui e' possibile decomporre le prestazioni di sistemi complessi come quelli per eseguire i workflow. Questo permettera sia un confronto obiettivo fra sistemi diversi che si trovano in competizione sul mercato che un confronto di configurazioni diverse dello stesso sistema.
Direktlink auf Lay Summary Letzte Aktualisierung: 15.10.2013

Verantw. Gesuchsteller/in und weitere Gesuchstellende

Mitarbeitende

Publikationen

Publikation
Estimating the Cost for Executing Business Processes in the Cloud
Vincenzo Ferme, Ana Ivanchikj, Cesare Pautasso (2016), Estimating the Cost for Executing Business Processes in the Cloud, in Business Process Management Forum, Rio de JaneiroSpringer, Cham.
Micro-Benchmarking BPMN 2.0 Workflow Management Systems with Workflow Patterns
Marigianna Skouradaki, Vincenzo Ferme, Cesare Pautasso, Frank Leymann, Andre van Hoorn (2016), Micro-Benchmarking BPMN 2.0 Workflow Management Systems with Workflow Patterns, in 28th International Conference on Advanced Information Systems Engineering (CAISE), Ljubljana, SloveniaSpringer, Cham.
A Container-centric Methodology for Benchmarking Workflow Management Systems
Vincenzo Ferme, Ana Ivanchikj, Cesare Pautasso, Marigianna Skouradaki, Frank Leymann (2016), A Container-centric Methodology for Benchmarking Workflow Management Systems, in 6th International Conference on Cloud Computing and Service Science (CLOSER 2016), RomeSciTePress, Setubal, Portugal.
Integrating Faban with Docker for Reliable, Repeatable and Reusable Performance Benchmarking
Vincenzo Ferme, Cesare Pautasso (2016), Integrating Faban with Docker for Reliable, Repeatable and Reusable Performance Benchmarking, in 7th ACM/SPEC International Conference on Performance Engineering (ICPE2016 Demo), Delft, NLACM, USA.
BPMeter: Web Service and Application for Static Analysis of BPMN 2.0 Collections
Ana Ivanchikj, Vincenzo Ferme, Cesare Pautasso (2015), BPMeter: Web Service and Application for Static Analysis of BPMN 2.0 Collections, in 13th International Conference on Business Process Management (BPM Demos 2015), InnsbruckCEUR-WS.org, Aachen.
A Framework for Benchmarking BPMN 2.0 Workflow Management Systems
Vincenzo Ferme, Ana Ivanchikj, Cesare Pautasso (2015), A Framework for Benchmarking BPMN 2.0 Workflow Management Systems, in 13th International Conference on Business Process Management (BPM 2015), InnsbruckSpringer, Heidelberg.
Application of Sub-Graph Isomorphism to Extract Reoccurring Structures from BPMN 2.0 Process Models
Marigianna Skouradaki, Katharina Göerlach, Michael Hahn, Frank Leymann (2015), Application of Sub-Graph Isomorphism to Extract Reoccurring Structures from BPMN 2.0 Process Models, in IEEE Symposium on Service-Oriented System Engineering (SOSE2015), San FranciscoIEEE, USA.
On the Road to Benchmarking BPMN 2.0 Workflow Engines
Marigianna Skouradaki, Dieter Roller, Frank Leymann, Vincenzo Ferme, Cesare Pautasso (2015), On the Road to Benchmarking BPMN 2.0 Workflow Engines, in 6th ACM/SPEC International Conference on Performance Engineering (ICPE2015), Austin, TexasACM, USA.
Towards Workflow Benchmarking: Open Research Challenges
Cesare Pautasso, Dieter Roller, Vincenzo Ferme, Frank Leymann, Marigianna Skouradaki (2015), Towards Workflow Benchmarking: Open Research Challenges, in 16. Fachtagung Datenbanksysteme für Business, Technologie und Web (BTW 2015), HamburgGesellschaft für Informatik, Bonn.
"BPELanon": Protect Business Processes on the Cloud
Marigianna Skouradaki, Dieter Roller, Frank Leymann, Vincenzo Ferme, Cesare Pautasso (2015), "BPELanon": Protect Business Processes on the Cloud, in 5th International Conference on Cloud Computing and Service Science (CLOSER 2015), LisbonSciTePress, Setubal, Portugal.

Zusammenarbeit

Gruppe / Person Land
Felder der Zusammenarbeit
Prof. Dr. Guido Wirtz, Univ. Bamberg Deutschland (Europa)
- vertiefter/weiterführender Austausch von Ansätzen, Methoden oder Resultaten
- Publikation
- Austausch von Mitarbeitern
Prof. Dr. Frank Leymann, Univ. Stuttgart Deutschland (Europa)
- vertiefter/weiterführender Austausch von Ansätzen, Methoden oder Resultaten
- Publikation
- Forschungsinfrastrukturen
- Austausch von Mitarbeitern

Wissenschaftliche Veranstaltungen



Selber organisiert

Titel Datum Ort

Veranstaltungen zum Wissenstransfer

Aktiver Beitrag

Titel Art des Beitrags Titel des Artikels oder Beitrages Datum Ort Beteiligte Personen
W-JAX’15 Vortrag 04.11.2015 Munich, Deutschland


Abstract

The goal of the BenchFlow project is to design the first benchmark for assessing and comparing the performance of workflow management systems (aka business process execution engines). Given the large number of heterogeneous systems and languages that have been proposed for modeling and executing workflows, to ensure the feasibility of this project, we will initially focus our efforts on a benchmark for standard WS-BPEL compliant Web service composition engines. Due to the inherent complexity of workflow engine architectures and the very large number of parameters affecting their performance, benchmarking workflow systems poses a number of scientific research challenges that require investigating a novel set of performance evaluation techniques. These will complement existing ones used to benchmark database management systems (e.g., TPC) or programming language compilers (e.g., SPEC), which are already well understood and have contributed to drive a very large performance improvement within the database, compiler and processor architecture industry. Workflow systems have become the platform to build composite service-oriented applications, whose performance depends on two factors: the performance of the workflow system itself and the performance of the composed services (which could lie outside of the control of the workflow). We plan to use a model-driven, self-/recursive testing approach to eliminate the impact of the external services by having them implemented as processes. The processes selected for the benchmark will be synthesized out of real-world WS-BPEL processes. Given the very large number of metrics that can be used to observe the performance of a workflow system, we aim at distilling a reduced set of performance indicators to compare different engines as well as different configurations of a given engine.