The R package MitISEM : efficient and robust simulation procedures for Bayesian inference

N. Baştürk, S. Grassi, L. Hoogerheide, A. Opschoor, H.K. van Dijk

Research output: Working paperProfessional

364 Downloads (Pure)

Abstract

This paper presents the R package MitISEM (mixture of t by importance sampling weighted expectation maximization) which provides an automatic and flexible two-stage method to approximate a non-elliptical target density kernel - typically a posterior density kernel - using an adaptive mixture of Student-t densities as approximating density. In the first stage a mixture of Student-t densities is fitted to the target using an expectation maximization (EM) algorithm where each step of the optimization procedure is weighted using importance sampling. In the second stage this mixture density is a candidate density for efficient and robust application of importance sampling or the Metropolis-Hastings (MH) method to estimate properties of the target distribution. The package enables Bayesian inference and prediction on model parameters and probabilities, in particular, for models where densities have multi-modal or other non-elliptical shapes like curved ridges. These shapes occur in research topics in several scientific fields. For instance, analysis of DNA data in bioinformatics, obtaining loans in the banking sector by heterogeneous groups in financial economics and analysis of education's effect on earned income in labor economics. The package MitISEM provides also an extended algorithm, 'sequential MitISEM', which substantially decreases computation time when the target density has to be approximated for increasing data samples. This occurs when the posterior or predictive density is updated with new observations and/or when one computes model probabilities using predictive likelihoods. We illustrate the MitISEM algorithm using three canonical statistical and econometric models that are characterized by several types of non-elliptical posterior shapes and that describe well-known data patterns in econometrics and finance. We show that MH using the candidate density obtained by MitISEM outperforms, in terms of numerical efficiency, MH using a simpler candidate, as well as the Gibbs sampler. The MitISEM approach is also used for Bayesian model comparison using predictive likelihoods.
Original languageEnglish
Place of PublicationMaastricht
PublisherMaastricht University, Graduate School of Business and Economics
Publication statusPublished - 1 Jan 2015

Cite this

Baştürk, N., Grassi, S., Hoogerheide, L., Opschoor, A., & van Dijk, H. K. (2015). The R package MitISEM : efficient and robust simulation procedures for Bayesian inference. Maastricht: Maastricht University, Graduate School of Business and Economics.
Baştürk, N. ; Grassi, S. ; Hoogerheide, L. ; Opschoor, A. ; van Dijk, H.K. / The R package MitISEM : efficient and robust simulation procedures for Bayesian inference. Maastricht : Maastricht University, Graduate School of Business and Economics, 2015.
@techreport{05317dce1682428c9bf974eb35dc6017,
title = "The R package MitISEM : efficient and robust simulation procedures for Bayesian inference",
abstract = "This paper presents the R package MitISEM (mixture of t by importance sampling weighted expectation maximization) which provides an automatic and flexible two-stage method to approximate a non-elliptical target density kernel - typically a posterior density kernel - using an adaptive mixture of Student-t densities as approximating density. In the first stage a mixture of Student-t densities is fitted to the target using an expectation maximization (EM) algorithm where each step of the optimization procedure is weighted using importance sampling. In the second stage this mixture density is a candidate density for efficient and robust application of importance sampling or the Metropolis-Hastings (MH) method to estimate properties of the target distribution. The package enables Bayesian inference and prediction on model parameters and probabilities, in particular, for models where densities have multi-modal or other non-elliptical shapes like curved ridges. These shapes occur in research topics in several scientific fields. For instance, analysis of DNA data in bioinformatics, obtaining loans in the banking sector by heterogeneous groups in financial economics and analysis of education's effect on earned income in labor economics. The package MitISEM provides also an extended algorithm, 'sequential MitISEM', which substantially decreases computation time when the target density has to be approximated for increasing data samples. This occurs when the posterior or predictive density is updated with new observations and/or when one computes model probabilities using predictive likelihoods. We illustrate the MitISEM algorithm using three canonical statistical and econometric models that are characterized by several types of non-elliptical posterior shapes and that describe well-known data patterns in econometrics and finance. We show that MH using the candidate density obtained by MitISEM outperforms, in terms of numerical efficiency, MH using a simpler candidate, as well as the Gibbs sampler. The MitISEM approach is also used for Bayesian model comparison using predictive likelihoods.",
author = "N. Başt{\"u}rk and S. Grassi and L. Hoogerheide and A. Opschoor and {van Dijk}, H.K.",
year = "2015",
month = "1",
day = "1",
language = "English",
publisher = "Maastricht University, Graduate School of Business and Economics",
address = "Netherlands",
type = "WorkingPaper",
institution = "Maastricht University, Graduate School of Business and Economics",

}

Baştürk, N, Grassi, S, Hoogerheide, L, Opschoor, A & van Dijk, HK 2015 'The R package MitISEM : efficient and robust simulation procedures for Bayesian inference' Maastricht University, Graduate School of Business and Economics, Maastricht.

The R package MitISEM : efficient and robust simulation procedures for Bayesian inference. / Baştürk, N.; Grassi, S.; Hoogerheide, L.; Opschoor, A.; van Dijk, H.K.

Maastricht : Maastricht University, Graduate School of Business and Economics, 2015.

Research output: Working paperProfessional

TY - UNPB

T1 - The R package MitISEM : efficient and robust simulation procedures for Bayesian inference

AU - Baştürk, N.

AU - Grassi, S.

AU - Hoogerheide, L.

AU - Opschoor, A.

AU - van Dijk, H.K.

PY - 2015/1/1

Y1 - 2015/1/1

N2 - This paper presents the R package MitISEM (mixture of t by importance sampling weighted expectation maximization) which provides an automatic and flexible two-stage method to approximate a non-elliptical target density kernel - typically a posterior density kernel - using an adaptive mixture of Student-t densities as approximating density. In the first stage a mixture of Student-t densities is fitted to the target using an expectation maximization (EM) algorithm where each step of the optimization procedure is weighted using importance sampling. In the second stage this mixture density is a candidate density for efficient and robust application of importance sampling or the Metropolis-Hastings (MH) method to estimate properties of the target distribution. The package enables Bayesian inference and prediction on model parameters and probabilities, in particular, for models where densities have multi-modal or other non-elliptical shapes like curved ridges. These shapes occur in research topics in several scientific fields. For instance, analysis of DNA data in bioinformatics, obtaining loans in the banking sector by heterogeneous groups in financial economics and analysis of education's effect on earned income in labor economics. The package MitISEM provides also an extended algorithm, 'sequential MitISEM', which substantially decreases computation time when the target density has to be approximated for increasing data samples. This occurs when the posterior or predictive density is updated with new observations and/or when one computes model probabilities using predictive likelihoods. We illustrate the MitISEM algorithm using three canonical statistical and econometric models that are characterized by several types of non-elliptical posterior shapes and that describe well-known data patterns in econometrics and finance. We show that MH using the candidate density obtained by MitISEM outperforms, in terms of numerical efficiency, MH using a simpler candidate, as well as the Gibbs sampler. The MitISEM approach is also used for Bayesian model comparison using predictive likelihoods.

AB - This paper presents the R package MitISEM (mixture of t by importance sampling weighted expectation maximization) which provides an automatic and flexible two-stage method to approximate a non-elliptical target density kernel - typically a posterior density kernel - using an adaptive mixture of Student-t densities as approximating density. In the first stage a mixture of Student-t densities is fitted to the target using an expectation maximization (EM) algorithm where each step of the optimization procedure is weighted using importance sampling. In the second stage this mixture density is a candidate density for efficient and robust application of importance sampling or the Metropolis-Hastings (MH) method to estimate properties of the target distribution. The package enables Bayesian inference and prediction on model parameters and probabilities, in particular, for models where densities have multi-modal or other non-elliptical shapes like curved ridges. These shapes occur in research topics in several scientific fields. For instance, analysis of DNA data in bioinformatics, obtaining loans in the banking sector by heterogeneous groups in financial economics and analysis of education's effect on earned income in labor economics. The package MitISEM provides also an extended algorithm, 'sequential MitISEM', which substantially decreases computation time when the target density has to be approximated for increasing data samples. This occurs when the posterior or predictive density is updated with new observations and/or when one computes model probabilities using predictive likelihoods. We illustrate the MitISEM algorithm using three canonical statistical and econometric models that are characterized by several types of non-elliptical posterior shapes and that describe well-known data patterns in econometrics and finance. We show that MH using the candidate density obtained by MitISEM outperforms, in terms of numerical efficiency, MH using a simpler candidate, as well as the Gibbs sampler. The MitISEM approach is also used for Bayesian model comparison using predictive likelihoods.

M3 - Working paper

BT - The R package MitISEM : efficient and robust simulation procedures for Bayesian inference

PB - Maastricht University, Graduate School of Business and Economics

CY - Maastricht

ER -

Baştürk N, Grassi S, Hoogerheide L, Opschoor A, van Dijk HK. The R package MitISEM : efficient and robust simulation procedures for Bayesian inference. Maastricht: Maastricht University, Graduate School of Business and Economics. 2015 Jan 1.