Introduction to hierarchical Bayes methods for data analysis

October 13, 2009

Hierarchical Bayes methods enable the combining of information from similar and independent experiments, yielding improved inference for both individual and shared model characteristics.

As a result of recent advances in computing and the consequent ability to evaluate complex models, Bayesian methods have increased in popularity in data analysis. This course introduces hierarchical Bayes methods, demonstrates their usefulness in challenging applied settings, and shows how they can be implemented using modern Markov chain Monte Carlo (MCMC) computational methods. We also provide an introduction to WinBUGS, the most general Bayesian software package available to date. Use of the methods will be demonstrated in advanced high-dimensional model settings (e.g., nonlinear longitudinal modeling or clinical trial design and analysis), where the MCMC Bayesian approach often provides the only feasible alternative that incorporates all relevant model features.

Webinar participants should have an M.S. (or advanced undergraduate) understanding of mathematical statistics at, say, the Hogg and Craig (1978) level. Basic familiarity with common statistical models (e.g., the linear regression model) and computing will be assumed, but we will not assume any significant previous exposure to Bayesian methods or Bayesian computing. The course is generally aimed at students and practicing statisticians who are intrigued by all the fuss about Bayes and Gibbs, but who may still mistrust the approach as theoretically mysterious and practically cumbersome.

Use of genomics and predictive biomarkers in the design and analysis of Phase III clinical trials

September 18, 2009

Current methods for the design and analysis of phase III clinical trials often results in the approval and use of drugs in broad populations of patients, many of whom do not benefit. This has serious limitations for patients and for health care economics. Current methods are also problematic for the development of molecularly targeted drugs which are expected to benefit only a subset of traditionally diagnosed patients. New paradigms for the design and analysis of clinical trials are needed for the new era of genomic technologies for characterizing diseases and for evaluation of molecularly targeted therapeutics.

We will focus on recent developments in the prospective use of predictive biomarkers in the design and analysis of phase III therapeutic clinical trials. The presentation will not be about exploratory analysis of data from clinical trials, but rather on the use of the use of genomic biomarkers in the design and analysis in a sufficiently structured and prospective manner that the conclusions about treatment effects and how they relate to biomarker specified subsets have the degree of reliability normally associated with phase III clinical trials. We will cover the targeted "enrichment" design in which a classifier test result is used as an eligibility criterion. The efficiency of that design and how it depends on the specificity of treatment effect and test performance characteristics will be discussed as well as limitations of that design. We will discuss "stratified designs" in which the test result is not used to restrict eligibility but as part of the primary analysis plan of the trial. Specific analysis plans and sample size considerations will be discussed. Both the enrichment design and stratification design require that the classifier be completely specified and analytically validated prior to the start of the pivotal trial. We will discuss various approaches to easing that requirement including the "adaptive biomarker threshold" design and the "adaptive signature" design. Recent extensions of those design concepts will also be described.

We will present a viewpoint that some of the conventional wisdom concerning the analysis of clinical trials is not appropriate for clinical trials in which a predictive biomarker is incorporated in the primary analysis of treatment effects. These conventions include the requirement of significant overall treatment effects or significant interactions in order to justify analysis of treatment effects in subsets. We will also present a new framework for the analysis of clinical trials that incorporates both hypothesis testing and predictive modeling. This framework provides for complementary roles of frequentist and Bayesian methods but requires that models be justified based on predictive accuracy.

Genomic data analysis with targeted maximum likelihood and super learning

August 24, 2009

Current statistical practice to assess an effect of an intervention or exposure on an outcome of interest often involves either maximum likelihood estimation for a priori specified regression model, or, manual and/or data adaptive interventions to fine tune a choice of model. In both cases, bias in the point estimates and the estimate of the signal to noise ratio are rampant, causing an epidemic of false claims based on data analyses. In this talk we present our efforts to construct machine learning algorithms for estimating a causal or adjusted effect that take away the need for specifying regression models, while still providing maximum likelihood based estimators and inference. Two fundamental concepts underlying this methodology are super learning, i.e., the very aggressive use of cross-validation to select optimal combinations of many model fits, and subsequent targeted maximum likelihood estimation to target the fit towards the causal effect of interest. Our maximally unbiased and efficient estimates are accompanied with statistical inference. In addition, multiple testing methods are employed in case one pursues effect estimation across a large set of variables.

We illustrate this method in observational studies for assessing the effect of mutations in the HIV virus that cause resistance to a particular drug regimen. We also illustrate the performance for assessing the effect on the outcome or response to treatment of single nucleotide polymorphisms and gene-expressions in genomic studies, including randomized trials. In particular, we demonstrate the performance of the super learning in prediction.

Large-scale significance testing of genomic data

June 23, 2009

The presenter will discuss recent advances in performing many hypothesis tests in the context of genomics data. This will include discussion on the false discovery rate, accounting for latent structure, and borrowing information across variables to increase power.

Another look at stratified analysis of binary data

May 7, 2009

In this webinar, we will look at the two classic weighting choices to combine binary data from multiple strata. The two choices use the inverse weighting and the Cochran-Mantel-Haenszel weighting. The former is popular among meta analysts while the latter is frequently used by statisticians. We will look at the implications of these two choices under different treatment effect scenarios. In addition to stratified analyses of efficacy data, we will also examine the situation where safety data are pooled from multiple studies to create an integrated safety summary. Experience has shown us that integration of safety data is vulnerable to the mischief of the Simpson's Paradox. We will show that Simpson's Paradox not only affects the inferential comparison between treatment groups with respect to adverse event proportions, it also affects the estimation of the proportions. We will discuss a proposal to adjust proportions when reporting proportions is necessary, as is the current practice in a product's package insert. We will conclude the Webinar with some practical recommendations.

Case study in parametric survival modeling

April 3, 2009

In this webinar, advantages of parametric survival modeling are discussed, contrasting with the Cox semiparametric proportional hazards model. Common parametric models such as exponential, Weibull, and log-normal will be overviewed. Then a comprehensive case study of the development of a log-normal multivariable survival model will be presented. Covariate effects are modeling flexibly without assuming linearity, model assumptions are checked, and the model is interpreted by a variety of graphical devices.

Classical sample-size analysis for hypothesis testing (Part II)

February 12, 2009

This session will quickly review the essentials of the first session and then continue with the malaria example to explore more vital questions that classical sample-size analyses fails to address. That is, if the planned study yields a significant p-value, what is the chance this is a Type I error? Likewise, if the study turns out non-significant, what is the chance this is a Type II error? By using judgments about the probability that the null hypothesis is false, we apply Bayes Theorem (taught with simple calculations in a table, no formulas) to assess these "crucial" Type I and II error rates, and we show (using a simple a Excel program) that they can differ greatly from their classical counterparts. Importantly, both crucial error rates are reduced by increasing the statistical power. Studies with small N that propose to test speculative hypotheses are prone to large crucial error rates. The final phase of the session deals with an actual early trial of a highly novel treatment for atherosclerosis in which a 0.02 p-value was deemed to be "the first convincing demonstration" of efficacy. What the investigators failed to understand, however, is that their crucial Type I error rate may have been well over 85%. We will end by going though a mock study planning exercise to design the follow-up study.

Text book

O'Brien R, Castelloe J. (2007). Sample-size analysis for traditional hypothesis testing: Concepts and issues. Pharmaceutical Statistics Using SAS: A Practical Guide. Dmitrienko A, Chuang-Stein C, D'Agostino R. (editors). SAS Press: Cary, NC.

Classical sample-size analysis for hypothesis testing (Part I)

January 29, 2009

This session will quickly review the essentials of the first session and then continue with the malaria example to explore more vital questions that classical sample-size analyses fails to address. That is, if the planned study yields a significant p-value, what is the chance this is a Type I error? Likewise, if the study turns out non-significant, what is the chance this is a Type II error? By using judgments about the probability that the null hypothesis is false, we apply Bayes Theorem (taught with simple calculations in a table, no formulas) to assess these "crucial" Type I and II error rates, and we show (using a simple a Excel program) that they can differ greatly from their classical counterparts. Importantly, both crucial error rates are reduced by increasing the statistical power. Studies with small N that propose to test speculative hypotheses are prone to large crucial error rates. The final phase of the session deals with an actual early trial of a highly novel treatment for atherosclerosis in which a 0.02 p-value was deemed to be "the first convincing demonstration" of efficacy. What the investigators failed to understand, however, is that their crucial Type I error rate may have been well over 85%. We will end by going though a mock study planning exercise to design the follow-up study.

Text book

O'Brien R, Castelloe J. (2007). Sample-size analysis for traditional hypothesis testing: Concepts and issues. Pharmaceutical Statistics Using SAS: A Practical Guide. Dmitrienko A, Chuang-Stein C, D'Agostino R. (editors). SAS Press: Cary, NC.