Project

Back to overview

SEDA: Sensitivity diagnostics for Bayesian hierarchical models

English title SEDA: Sensitivity diagnostics for Bayesian hierarchical models
Applicant Roos Malgorzata
Number 175933
Funding scheme Project funding (Div. I-III)
Research institution Institut für Epidemiologie, Biostatistik und Prävention Universität Zürich
Institution of higher education University of Zurich - ZH
Main discipline Mathematics
Start/End 01.01.2018 - 31.12.2021
Approved amount 272'892.00
Show all

All Disciplines (3)

Discipline
Mathematics
Methods of Epidemiology and Preventive Medicine
Material Sciences

Keywords (10)

Bayesian hierarchical models; formal sensitivity diagnostics; model identification; resampling; spatial models; measurement error models; INLA; Stan; JAGS; CODA

Lay Summary (German)

Lead
Die Bayesianischen hierarchischen Modelle (BHM) sind eine etablierte statistische Methodologie, die bei komplexen Fragestellungen eingesetzt wird. Die BHM-Analysen werden häufig mittels bekannter Bayesianischer Software-Pakete wie Stan, JAGS oder R-INLA durchgeführt. Die besondere Fähigkeit von BHMs, externe Priori-Informationen in der Analyse zu integrieren, führt zu deren Sensitivität. Leider gibt es heutzutage kein formelles diagnostisches Werkzeug für die Sensitivitäts-Analyse von BHMs. Deshalb besteht das Hauptanliegen des Projektes darin, ein solches Werkzeug für BHMs in Stan, JAGS und R-INLA zu entwickeln und zu implementieren. Ein frei zugängliches SEDA-Paket in R wird die Anwender über die Sensitivität der BHMs informieren. Damit wird die Zuverlässigkeit von BHMs gesteigert. Ferner werden zwei medizinische Anwendungen relevante Fragestellungen zu den mikrobiologischen Resistenz-Bestimmungs-Richtlinien und zur Testung von zahnärztlichen Werkstoffen beantworten.
Lay summary
Die Bayesianischen hierarchischen Modelle (BHM) sind eine gut etablierte statistische Methodologie, die häufig bei Entscheidungsfindung in komplexen Fragestellungen genutzt wird. Meistens werden die BHM-Analysen mittels bekannter Bayesianischer Software-Pakete wie Stan, JAGS oder R-INLA durchgeführt. Die BHMs verfügen über die besondere Fähigkeit, externe Informationen in der Analyse zu integrieren. Eine der schwierigsten und noch nicht gut verstandenen Eigenschaften der BHMs ist ihre Sensitivität bezüglich der Priori-Annahmen. Bislang gibt es noch kein formelles diagnostisches Werkzeug, mit dem die Sensitivität der BHMs quantifiziert werden kann. Deshalb besteht das Hauptanliegen des Projektes darin, ein neuartiges Werkzeug für eine Zwei-Komponenten-Sensitivitätsanalyse von BHMs in Stan, JAGS und R-INLA zu entwickeln und zu implementieren. Daher muss ein einheitliches und standardisiertes Vorgehen für die Handhabung von Rand-Posteriori-Verteilungen von MCMC-Systemen (Stan/JAGS) und von einer numerischen Annäherung (R-INLA) entwickelt werden. Ein frei zugängliches SEDA-Paket in R wird die Anwender über das Ausmass der Sensitivität und der Nicht-Identifizierbarkeit der BHMs informieren. Dadurch wird eine Kontrolle der Zuverlässigkeit der BHMs gewährleistet. Zwei medizinische Anwendungen werden sich einerseits mit mikrobiologischen Resistenz-Bestimmungs-Richtlinien und anderseits mit der Testung von zahnärztlichen Werkstoffen beschäftigen. Für die Ermittlung der Sensitivität der verwendeten Messfehler-Modelle und räumlichen BHMs wird die im SEDA angebotene Zwei-Komponenten Sensitivitäts-Diagnostik genutzt.

Direct link to Lay Summary Last update: 15.12.2017

Responsible applicant and co-applicants

Employees

Project partner

Publications

Publication
How vague is vague? How informative is informative? Reference analysis for Bayesian meta‐analysis
Ott Manuela, Plummer Martyn, Roos Małgorzata (2021), How vague is vague? How informative is informative? Reference analysis for Bayesian meta‐analysis, in Statistics in Medicine, 40(20), 4505-4521.
Sensitivity and identification quantification by a relative latent model complexity perturbation in Bayesian meta‐analysis
Roos Małgorzata, Hunanyan Sona, Bakka Haakon, Rue Håvard (2021), Sensitivity and identification quantification by a relative latent model complexity perturbation in Bayesian meta‐analysis, in Biometrical Journal, bimj.20200-bimj.20200.

Scientific events

Active participation

Title Type of contribution Title of article or contribution Date Place Persons involved
IBS-ROeS Conference 2021 Talk given at a conference How vague is vague? How informative is informative? Reference analysis for Bayesian meta-analysis 09.09.2021 Salzburg, Austria Roos Malgorzata;
International Workshop on Statistical Modelling 2019 Talk given at a conference Sensitivity and identification quantification by a relative latent model complexity perturbation in the Bayesian meta-analysis 11.07.2019 Guimarães, Portugal Roos Malgorzata;
DAGStat Conference 2019 Talk given at a conference Classiffcation of tail-adjusted heterogeneity priors in the Bayesian meta-analysis estimated by bayesmeta 20.03.2019 Munich, Germany Roos Malgorzata;
DAGStat Conference 2019 Poster Complete separation in data in the context of antimicrobial susceptibility breakpoints derivation 18.03.2019 Munich, Germany Roos Malgorzata; Hunanyan Sona;
The 27th Nordic Conference in Mathematical Statistics Poster Prior sensitivity quantification: To scale, or not to scale, that is the question 26.06.2018 Tartu, Estonia Hunanyan Sona; Roos Malgorzata;


Use-inspired outputs

Abstract

Bayesian hierarchical models (BHM) are nowadays a well established statistical methodology. Their versatility guarantees that they can be applied to a wide range of practical settings where they help to understand complex processes. Moreover, they have a unique ability to incorporate external prior information in the analysis. Frequently, probabilities inferred from BHMs support decision making. BHMs can be conveniently estimated by Bayesian general-purpose software systems. In particular, Stan, JAGS and R-INLA represent well the current state-of-the-art of both MCMC sampling and approximation approaches for fitting BHMs to complex hierarchical data. Although BHMs are very useful in practice, the reliability of their inference is subject to several model assumptions. The most intriguing aspect of BHMs is their sensitivity to assumed priors. There is also a substantial risk that at least some of the complex BHMs are actually non-identified. In such a case no learning about model parameters is possible even if the amount of the data is increasing. If the posterior precision is not affected by the sample size, a model is suspected to be non-identified. Were there a way to warn scientists about sensitivity and non-identification issues in BHMs, better self-control and model criticism would be allowed for. Unfortunately, to-date such a formal and flexible diagnostic tool for BHMs is missing. The proposed research aims to close this gap by developing and implementing a novel two-component sensitivity diagnostic tool for BHMs estimated by Stan, JAGS or INLA. At the first component the epsilon-local sensitivity with respect to the prior assumptions will be quantified. To accomplish this task, a unified approach to handle marginal posterior distributions generated by MCMC samples (Stan/JAGS) and numerical approximations (INLA) must be developed. At the second component, model non-identification will be assessed by posterior precision's characteristic with changing amount of observations. For this task, a general resampling approach for complex hierarchical models must be established. Optimal factors for down- and up-sampling will be derived theoretically and applied in practice. A free accessible SEDA package in R for sensitivity diagnostics in BHMs will complement functionality of CODA and ShinyStan packages for convergence diagnostics.Two medical applications dealing with Breakpoints for bacterial resistance and Material wear in 3D will provide a strong motivation for use of BHMs in the proposed project. They will have immediate practical relevance by answering questions relevant to microbiological and dental materials research. For criticism of applied measurement error and spatial BHMs, the two-component sensitivity diagnostics will be assessed by SEDA. A dedicated simulation study will determine properties of the two-component sensitivity diagnostics with respect to prior-data disagreement and prior informativeness.Although the proposed project concentrates mainly on sensitivity to prior assumptions and on non-identification of BHM, it is intentionally designed to be the seed for a series of projects. In the long term, we aim to extend SEDA functionality in a step-by-step manner to embrace sensitivity diagnostics in BHM pertaining to other model assumptions such as sampling model, model structure and data uncertainty.
-