Project

share-nothing; transaction processing; data sharing workload-aware processing; thread migration; share-everything; data management; instruction cache; data partitioning; hardware-aware processing

Lead
Les applications de traitement des transactions sont mission-critiques pour des nombreuses entreprises où il y a peu de marge pour ne pas risquer de compromettre la performance et l'extensibilité. Ce projet pose la fondation pour les systèmes des bases de données qui s'adaptent et se reconfigurent en ligne, en fonction de la charge de travail et du matériel, tout en assurant une utilisation efficace des ressources.
Lay summary
Le traitement des transactions en ligne (On-Line Transaction Processing - OLTP) est un secteur d’activité représentant des milliards de dollars. En plus, il s'agit d'une des applications les plus importants de base de données. La pertinence de la charge de travail des OLTP augmente avec la croissance des volumes de transaction d'applications comme les distributeurs automatiques de billets, les achats en ligne ou les transactions financières. Ces applications exigent des larges volumes des transactions pour satisfaire le nombre de clients en constant augmentation, et les exigences de performance et d’extensibilité. Par conséquent, les innovations en OLTP attirent l'attention et sont mission-critiques pour de nombreuses entreprises en Suisse. Un autre facteur qui dicte la refonte et remise en œuvre des bases de données OLTP sont les tendances émergentes du matériel. Des plateformes modernes sont abondamment parallèles et de plus en plus hétérogènes et ne ressemblent pas aux ordinateurs pour lesquelles les systèmes OLTP ont été conçus. Dans un monde technologique en évolution rapide, les entreprises contournent des limitations dans la conception de base de données OLTP par le financement du matériel qui est de plus en plus cher et sophistiqué, en espérant que leur problème OLTP diminue. Malgré cela, rapidement les problèmes du matériel et de la charge de travail réapparaissent à une échelle différente ce qui traduit à l'achat de nouveau matériel qui ne sera pas exploité au maximum se son potentiel. Dans ce projet, nous fournissons des algorithmes rapides et évolutifs, qui peuvent être implémentés avec des modifications limitées sur des systèmes OLTP existants. Nous identifions les limitations fondamentales de l'architecture informatique qui affectent les systèmes OLTP traditionnelles, et nous préparons un plan pour s'occuper de ces goulots d’étranglement, avec une série des corrections qui nous permet d'utiliser le matériel et logiciel actuel à son plein potentiel.

Direct link to Lay Summary

Last update: 10.02.2014

Lead
Transaction processing applications are mission-critical for many enterprises with little margin for compromising either performance or scalability. This project lays the groundwork for database systems that adapt and reconfigure online, depending on the workload and the hardware, while ensuring efficient resource utilization.
Lay summary
On-Line Transaction Processing (OLTP) is a multi-billion dollar industry and one of the most important database applications. The relevance of OLTP workloads is growing with the increased transaction volumes from applications such as ATMs, On-Line Shopping or financial trading. These applications require large-scale transactions volumes to satisfy the ever-increasing number of customers and requirements for performance and scalability. Therefore, innovations in OLTP attract attention, and are mission-critical for many enterprises in Switzerland, where there is little margin for compromising either performance or scalability. Another factor that dictates redesign and re-implementation of OLTP databases is the emerging hardware trends. Modern hardware platforms are abundantly parallel and increasingly heterogeneous, and do not resemble the computers that OLTP systems were designed for, several decades ago. In a fast-moving technological world, companies are circumventing limitations in OLTP database designs by financing increasingly expensive and sophisticated hardware hoping that their OLTP problem diminishes. Soon enough, however, the hardware and workload problems appear again in a different scale and more expensive hardware is being purchased and implemented but not utilized at its full potential. In this project, we deliver fast and scalable algorithms which can be implemented with limited changes on existing OLTP systems. We identify the fundamental computer architecture limitations that affect traditional OLTP systems, and establish a plan to permanently address these bottlenecks. Our approach does not require a fundamental redesign for OLTP databases, but suggests a set of fixes, which enables us to use the current hardware and software to its full potential.

Direct link to Lay Summary

Last update: 10.02.2014

Active participation

Title	Year

Transaction processing (TP) is a multi-billion dollar industry, as it is mission-critical for enterprises with little margin for compromising either performance or scalability. Major database vendors spend significant effort in developing highly-optimized TP software releases, often with platform-specific optimizations. Over the past few decades, TP has benefited greatly by the ever-increasing uniprocessor speed - until 2004, when the frequency-scaling wall forced that trend to a screeching halt. Nowadays, faster hardware means more processing cores in each CPU chip, forming chip multiprocessors (multicore or CMP), and servers with multiple CPU sockets of multicore processors (SMP of CMP). CMP are highly parallel and heterogeneous in communication costs: sets, or islands, of processing cores communicate with each other efficiently through common on-chip caches, but communicate less efficiently with others through bandwidth-limited and higher-latency links. Even though CMP dominate in modern data-centers, TP engine architecture is oblivious to non-uniform memory access costs across computing cores, so transactional workloads exhibit suboptimal and even worse, unpredictable performance. Therefore, every new release of hardware platform is followed by a humongous investment by database companies to make their systems scale to decent performance.Recent research efforts to improve TP scalability on CMP considers data organization, sometimes combined with clever work assignment, following either a shared-nothing or a shared-memory principle. Others propose modular engine architectures and grouped query execution for performance. We identify the root cause for inefficient execution of transactional workloads on CMP to be that typical TP engines partition data and schedule their work independently of the specifics of either the workload or the hardware. Our proposal builds on the state-of-the-art by developing a complete infrastructure of techniques inspired synergistically by data, work, and hardware design considerations. The proposed work carefully balances partitioning and sharing of data and instructions, through (a) partitioning data structures across islands of communication, (b) dynamic work assignment and thread migration, and (c) novel operators, enacted by sharing opportunities. We will develop these techniques in parallel due to synergies but also potential conflicts and tradeoffs in the techniques (e.g. instruction and data co-locality).The ambition of this proposal is to lay the groundwork for database systems that adapt and reconfigure online, depending on the workload and the hardware, while ensuring efficient resource utilization. Our research results will directly impact multiple sectors of society that rely on transaction processing for their business processes (e.g. finance, accounting, e-commerce, healthcare), many of which are a significant part of the Swiss industrial sector.