Project

Back to overview

Workload- and hardware-aware transaction processing

English title Workload- and hardware-aware transaction processing
Applicant Ailamaki Anastasia
Number 146407
Funding scheme Project funding
Research institution Laboratoire de systèmes d'exploitation EPFL - IC - IIF - LABOS
Institution of higher education EPF Lausanne - EPFL
Main discipline Information Technology
Start/End 01.06.2014 - 31.08.2018
Approved amount 531'750.00
Show all

Keywords (9)

share-nothing; transaction processing; data sharing workload-aware processing; thread migration; share-everything; data management; instruction cache; data partitioning; hardware-aware processing

Lay Summary (French)

Lead
Les applications de traitement des transactions sont mission-critiques pour des nombreuses entreprises où il y a peu de marge pour ne pas risquer de compromettre la performance et l'extensibilité. Ce projet pose la fondation pour les systèmes des bases de données qui s'adaptent et se reconfigurent en ligne, en fonction de la charge de travail et du matériel, tout en assurant une utilisation efficace des ressources.
Lay summary

Le traitement des transactions en ligne (On-Line Transaction Processing - OLTP) est un secteur d’activité représentant des milliards de dollars. En plus, il s'agit d'une des applications les plus importants de base de données. La pertinence de la charge de travail des OLTP augmente avec la croissance des volumes de transaction d'applications comme les distributeurs automatiques de billets, les achats en ligne ou les transactions financières. Ces applications exigent des larges volumes des transactions pour satisfaire le nombre de clients en constant augmentation, et les exigences de performance et d’extensibilité. Par conséquent, les innovations en OLTP attirent l'attention et sont mission-critiques pour de nombreuses entreprises en Suisse. Un autre facteur qui dicte la refonte et remise en œuvre des bases de données OLTP sont les tendances émergentes du matériel. Des plateformes modernes sont abondamment parallèles et de plus en plus hétérogènes et ne ressemblent pas aux ordinateurs pour lesquelles les systèmes OLTP ont été conçus. Dans un monde technologique en évolution rapide, les entreprises contournent des limitations dans la conception de base de données OLTP par le financement du matériel qui est de plus en plus cher et sophistiqué, en espérant que leur problème OLTP diminue. Malgré cela, rapidement les problèmes du matériel et de la charge de travail réapparaissent à une échelle différente ce qui traduit à l'achat de nouveau matériel qui ne sera pas exploité au maximum se son potentiel. Dans ce projet, nous fournissons des algorithmes rapides et évolutifs, qui peuvent être implémentés avec des modifications limitées sur des systèmes OLTP existants. Nous identifions les limitations fondamentales de l'architecture informatique qui affectent les systèmes OLTP traditionnelles, et nous préparons un plan pour s'occuper de ces goulots d’étranglement, avec une série des corrections qui nous permet d'utiliser le matériel et logiciel actuel à son plein potentiel. 

Direct link to Lay Summary Last update: 10.02.2014

Lay Summary (English)

Lead
Transaction processing applications are mission-critical for many enterprises with little margin for compromising either performance or scalability. This project lays the groundwork for database systems that adapt and reconfigure online, depending on the workload and the hardware, while ensuring efficient resource utilization.
Lay summary
On-Line Transaction Processing (OLTP) is a multi-billion dollar industry

and one of the most important database applications. The relevance of OLTP

workloads is growing with the increased transaction volumes from

applications such as ATMs, On-Line Shopping or financial trading. These

applications require large-scale transactions volumes to satisfy the

ever-increasing number of customers and requirements for performance and

scalability. Therefore, innovations in OLTP attract attention, and are

mission-critical for many enterprises in Switzerland, where there is

little margin for compromising either performance or scalability.

Another factor that dictates redesign and re-implementation of OLTP

databases is the emerging hardware trends. Modern hardware platforms are

abundantly parallel and increasingly heterogeneous, and do not resemble

the computers that OLTP systems were designed for, several decades ago.

In a fast-moving technological world, companies are circumventing

limitations in OLTP database designs by financing increasingly expensive

and sophisticated hardware hoping that their OLTP problem diminishes. Soon

enough, however, the hardware and workload problems appear again in a

different scale and more expensive hardware is being purchased and

implemented but not utilized at its full potential.

In this project, we deliver fast and scalable algorithms which can be

implemented with limited changes on existing OLTP systems. We identify the

fundamental computer architecture limitations that affect traditional OLTP

systems, and establish a plan to permanently address these bottlenecks.

Our approach does not require a fundamental redesign for OLTP databases,

but suggests a set of fixes, which enables us to use the current hardware

and software to its full potential.
Direct link to Lay Summary Last update: 10.02.2014

Responsible applicant and co-applicants

Employees

Publications

Publication
Analyzing the impact of system architecture on the scalability of OLTP engines for high-contention workloads
Appuswamy Raja, Anadiotis Angelos C., Porobic Danica, Iman Mustafa K., Ailamaki Anastasia (2017), Analyzing the impact of system architecture on the scalability of OLTP engines for high-contention workloads, in Proceedings of the VLDB Endowment, 11(2), 121-134.
A methodology for OLTP micro-architectural analysis
Sirin Utku, Yasin Ahmad, Ailamaki Anastasia (2017), A methodology for OLTP micro-architectural analysis, in the 13th International Workshop, Chicago, IllinoisACM New York, NY, USA ©2016, New York, NY, USA ©2016.
The Case For Heterogeneous HTAP
Appuswamy Raja (2017), The Case For Heterogeneous HTAP, in 8th Biennial Conference on Innovative Data Systems Research, Chaminade, California, January 8-11,201CIDR Conference, USA.
Characterization of the Impact of Hardware Islands on OLTP
Porobic Danica, Pandis Ippokratis, Branco Miguel, Tözün Pınar, Ailamaki Anastasia (2016), Characterization of the Impact of Hardware Islands on OLTP, in The VLDB Journal, 25(5), 625-650.
More than a networkdistributed OLTP on clusters of hardware islands
Porobic Danica, Tözün Pınar, Appuswamy Raja, Ailamaki Anastasia (2016), More than a networkdistributed OLTP on clusters of hardware islands, in the 12th International Workshop, San Francisco, CaliforniaACM New York, NY, USA ©2016, New York, NY, USA ©2016.
Micro-architectural Analysis of In-memory OLTP
Sirin Utku, Tözün Pinar, Porobic Danica, Ailamaki Anastasia (2016), Micro-architectural Analysis of In-memory OLTP, in the 2016 International Conference, San Francisco, California, USAACM New York, NY, USA ©2016, New York, NY, USA ©2016.
OLTP on a server-grade ARMpower, throughput and latency comparison
Sirin Utku, Appuswamy Raja, Ailamaki Anastasia (2016), OLTP on a server-grade ARMpower, throughput and latency comparison, in the 12th International Workshop, San Francisco, CaliforniaACM New York, NY, USA ©2016, New York, NY, USA ©2016.
Dynamic fine-grained scheduling for energy-efficient main-memory queries
Psaroudakis Iraklis, Kissinger Thomas, Porobic Danica, Ilsche Thomas, Liarou Erietta, Tözün Pınar, Ailamaki Anastasia, Lehner Wolfgang (2014), Dynamic fine-grained scheduling for energy-efficient main-memory queries, in the Tenth International Workshop, Snowbird, UtahACM New York, NY, USA ©2016, New York, NY, USA ©2016.
ATraPos: Adaptive Transaction Processing on Hardware Islands
Porobic Danica, Liarou Erietta, Tozun Pinar, Ailamaki Anastasia (2014), ATraPos: Adaptive Transaction Processing on Hardware Islands, in ICDE 2014, 30th ICDE Chicago, IL, USANA, NA.

Scientific events

Active participation

Title Type of contribution Title of article or contribution Date Place Persons involved
Data Management on New Hardware, DaMoN 2017 Talk given at a conference A Methodology for OLTP Micro-architectural Analysis 14.05.2017 Chicago, United States of America Ailamaki Anastasia; Sirin Utku;
Biennial Conference on Innovative Data Systems Research (CIDR) 2017 Talk given at a conference The Case For Heterogeneous HTAP 08.01.2017 Chaminade, California, United States of America Porobic Danica; Ailamaki Anastasia;
Data Management on New Hardware, DaMoN 2016 Talk given at a conference OLTP On A Server-grade ARM: Power, Throughput and Latency Comparison 27.06.2016 San Fransisco, United States of America Ailamaki Anastasia; Sirin Utku;
Data Management on New Hardware, DaMoN 2016 Talk given at a conference More Than A Network: Distributed OLTP on Clusters of Hardware Islands 27.06.2016 San Fransisco, United States of America Porobic Danica; Ailamaki Anastasia;
SIGMOD '16-2016 International Conference on Management of Data Talk given at a conference Microarchitectural Analysis of In-memory OLTP 26.06.2016 San Fransisco, United States of America Porobic Danica; Ailamaki Anastasia;
Data Management on New Hardware, DaMoN 2015 Talk given at a conference Applying HTM to an OLTP System: No Free Lunch 31.05.2015 Melbourne, Australia Ailamaki Anastasia; Porobic Danica;


Awards

Title Year
Best of DaMon 2017, for the Paper "A Methodology for OLTP Micro-architectural Analysis" Utku Sirin (EPFL), Ahmad Yasin (Intel Corporation), Anastasia Ailamaki (EPFL) 2017

Abstract

Transaction processing (TP) is a multi-billion dollar industry, as it is mission-critical for enterprises with little margin for compromising either performance or scalability. Major database vendors spend significant effort in developing highly-optimized TP software releases, often with platform-specific optimizations. Over the past few decades, TP has benefited greatly by the ever-increasing uniprocessor speed - until 2004, when the frequency-scaling wall forced that trend to a screeching halt. Nowadays, faster hardware means more processing cores in each CPU chip, forming chip multiprocessors (multicore or CMP), and servers with multiple CPU sockets of multicore processors (SMP of CMP). CMP are highly parallel and heterogeneous in communication costs: sets, or islands, of processing cores communicate with each other efficiently through common on-chip caches, but communicate less efficiently with others through bandwidth-limited and higher-latency links. Even though CMP dominate in modern data-centers, TP engine architecture is oblivious to non-uniform memory access costs across computing cores, so transactional workloads exhibit suboptimal and even worse, unpredictable performance. Therefore, every new release of hardware platform is followed by a humongous investment by database companies to make their systems scale to decent performance.Recent research efforts to improve TP scalability on CMP considers data organization, sometimes combined with clever work assignment, following either a shared-nothing or a shared-memory principle. Others propose modular engine architectures and grouped query execution for performance. We identify the root cause for inefficient execution of transactional workloads on CMP to be that typical TP engines partition data and schedule their work independently of the specifics of either the workload or the hardware. Our proposal builds on the state-of-the-art by developing a complete infrastructure of techniques inspired synergistically by data, work, and hardware design considerations. The proposed work carefully balances partitioning and sharing of data and instructions, through (a) partitioning data structures across islands of communication, (b) dynamic work assignment and thread migration, and (c) novel operators, enacted by sharing opportunities. We will develop these techniques in parallel due to synergies but also potential conflicts and tradeoffs in the techniques (e.g. instruction and data co-locality).The ambition of this proposal is to lay the groundwork for database systems that adapt and reconfigure online, depending on the workload and the hardware, while ensuring efficient resource utilization. Our research results will directly impact multiple sectors of society that rely on transaction processing for their business processes (e.g. finance, accounting, e-commerce, healthcare), many of which are a significant part of the Swiss industrial sector.
-