You are here

A Student’s Guide to Bayesian Statistics
Share

A Student’s Guide to Bayesian Statistics

  • Ben Lambert - Imperial College London (London, United Kingdom)
Additional resources:


August 2018 | 520 pages | SAGE Publications Ltd

Supported by a wealth of learning features, exercises, and visual elements as well as online video tutorials and interactive simulations, this book is the first student-focused introduction to Bayesian statistics.

Without sacrificing technical integrity for the sake of simplicity, the author draws upon accessible, student-friendly language to provide approachable instruction perfectly aimed at statistics and Bayesian newcomers. Through a logical structure that introduces and builds upon key concepts in a gradual way and slowly acclimatizes students to using R and Stan software, the book covers:

  • An introduction to probability and Bayesian inference
  • Understanding Bayes' rule 
  • Nuts and bolts of Bayesian analytic methods
  • Computational Bayes and real-world Bayesian analysis
  • Regression analysis and hierarchical methods

This unique guide will help students develop the statistical confidence and skills to put the Bayesian formula into practice, from the basic concepts of statistical inference to complex applications of analyses.


 
Chapter 1: How to best use this book
The purpose of this book

 
Who is this book for?

 
Pre-requisites

 
Book outline

 
Route planner - suggested journeys through Bayesland

 
Video

 
Problem sets

 
Code

 
R and Stan

 
Why don’t more people use Bayesian statistics?

 
What are the tangible (non-academic) benefits of Bayesian statistics?

 
 
Part I: An introduction to Bayesian inference
 
Chapter 2: The subjective worlds of Frequentist and Bayesian statistics
Bayes’ rule - allowing us to go from the effect back to its cause

 
The purpose of statistical inference

 
The world according to Frequentists

 
The world according to Bayesians

 
Do parameters actually exist and have a point value?

 
Frequentist and Bayesian inference

 
Bayesian inference via Bayes’ rule

 
Implicit versus Explicit subjectivity

 
 
Chapter 3: Probability - the nuts and bolts of Bayesian inference
Probability distributions: helping us explicitly state our ignorance

 
Independence

 
Central Limit Theorems

 
A derivation of Bayes’ rule

 
The Bayesian inference process from the Bayesian formula

 
 
Part II: Understanding the Bayesian formula
 
Chapter 4: Likelihoods
What is a likelihood?

 
Why use ‘likelihood’ rather than ‘probability’?

 
What are models and why do we need them?

 
How to choose an appropriate likelihood?

 
Exchangeability vs random sampling

 
Maximum likelihood - a short introduction

 
 
Chapter 5: Priors
What are priors, and what do they represent?

 
The explicit subjectivity of priors

 
Combining a prior and likelihood to form a posterior

 
Constructing priors

 
A strong model is less sensitive to prior choice

 
 
Chapter 6: The devil’s in the denominator
An introduction to the denominator

 
The difficulty with the denominator

 
How to dispense with the difficulty: Bayesian computation

 
 
Chapter 7: The posterior - the goal of Bayesian inference
Expressing parameter uncertainty in posteriors

 
Bayesian statistics: updating our pre-data uncertainty

 
The intuition behind Bayes’ rule for inference

 
Point parameter estimates

 
Intervals of uncertainty

 
From posterior to predictions by sampling

 
 
Part III: Analytic Bayesian methods
 
Chapter 8: An introduction to distributions for the mathematically-un-inclined
The interrelation among distributions

 
Sampling distributions for likelihoods

 
Prior distributions

 
How to choose a likelihood

 
Table of common likelihoods, their uses, and reasonable priors

 
Distributions of distributions, and mixtures - link to website, and relevance

 
 
Chapter 9: Conjugate priors and their place in Bayesian analysis
What is a conjugate prior and why are they useful?

 
Gamma-poisson example

 
Normal example: giraffe height

 
Table of conjugate priors

 
The lessons and limits of a conjugate analysis

 
 
Chapter 10: Evaluation of model fit and hypothesis testing
Posterior predictive checks

 
Why do we call it a p value?

 
Statistics measuring predictive accuracy: AIC, Deviance, WAIC and LOO-CV

 
Marginal likelihoods and Bayes factors

 
Choosing one model, or a number?

 
Sensitivity analysis

 
 
Chapter 11: Making Bayesian analysis objective?
The illusion of the ’uninformative’ uniform prior

 
Jeffreys’ priors

 
Reference priors

 
Empirical Bayes

 
A move towards weakly informative priors

 
 
Part IV: A practical guide to doing real life Bayesian analysis: Computational Bayes
 
Chapter 12: Leaving conjugates behind: Markov Chain Monte Carlo
The difficulty with real life Bayesian inference

 
Discrete approximation to continuous posteriors

 
The posterior through quadrature

 
Integrating using independent samples: an introduction to Monte Carlo

 
Why is independent sampling easier said than done?

 
Ideal sampling from a posterior using only the un-normalised posterior

 
Moving from independent to dependent sampling

 
What’s the catch with dependent samplers?

 
 
Chapter 13: Random Walk Metropolis
Sustainable fishing

 
Prospecting for gold

 
Defining the Metropolis algorithm

 
When does Metropolis work?

 
Efficiency of convergence: the importance of choosing the right proposal scale

 
Metropolis-Hastings

 
Judging convergence

 
Effective sample size revisited

 
 
Chapter 14: Gibbs sampling
Back to prospecting for gold

 
Defining the Gibbs algorithm

 
Gibbs’ earth: the intuition behind the Gibbs algorithm

 
The benefits and problems with Gibbs and Random Walk Metropolis

 
A change of parameters to speed up exploration

 
 
Chapter 15: Hamiltonian Monte Carlo
Hamiltonian Monte Carlo as a sledge

 
NLP space

 
Solving for the sledge motion over NLP space

 
How to shove the sledge

 
The acceptance probability of HMC

 
The complete Hamiltonian Monte Carlo algorithm

 
The performance of HMC versus Random Walk Metropolis and Gibbs

 
Optimal step length of HMC: introducing the “No U-Turn Sampler”

 
 
Chapter 16: Stan
Why Stan, and how to get it

 
Getting setup with Stan using RStan

 
Our first words in Stan

 
Essential Stan reading

 
What to do when things go wrong

 
How to get further help

 
 
Part V: Hierarchical models and regression
 
Chapter 17: Hierarchical models
The spectrum from fully-pooled to heterogeneous

 
Non-centered parameterisations in hierarchical models

 
Case study: Forecasting the EU referendum result

 
The importance of fake data simulation for complex models

 
 
Chapter 18: Linear regression models
Example: high school test scores in England

 
Pooled model

 
Interactions

 
Heterogeneous coefficient model

 
Hierarchical model

 
Incorporating LEA-level data

 
 
Chapter 19: Generalised linear models and other animals
Example: electoral participation in European countries

 
Discrete parameter models in Stan

 

Supplements

Click for online resources
https://study.sagepub.com/lambert

An excellent resource on Bayesian analysis accessible to students from a diverse range of statistical backgrounds and interests.  Easy to follow with well documented examples to illustrate key concepts.

Bronwyn Loong
College of Business and Economics, Australian National University

When I was a grad student, Bayesian statistics was restricted to those with the mathematical fortitude to plough through source literature. Thanks to Lambert, we now have something we can give to the modern generation of nascent data scientists as a first course. Love the supporting videos, too!

Wray Buntine
Information Technology, Monash University

Written in highly accessible language, this book is the gateway for students to gain a deep understanding of the logic of Bayesian analysis and to apply that logic with numerous carefully selected hands-on examples. Lambert moves seamlessly from a traditional Bayesian approach (using analytic methods) that serves to solidify fundamental concepts, to a modern Bayesian approach (using computational sampling methods) that endows students with the powerful and practical powers of application. I would recommend this book and its accompanying materials to any students or researchers who wish to learn and actually do Bayesian modeling. 

Fred Oswald
Psychology, Rice University

A balanced combination of theory, application and implementation of Bayesian statistics in a not very technical language. A tangible introduction to intangible concepts of Bayesian statistics for beginners.

Golnaz Shahtahmassebi
Senior Lecturer in Statistics, School of Science & Technology, Nottingham Trent University

The late, famous statistician Jimmie Savage would have taken great pleasure in this book based on his work in the 1960s on Bayesian statistics.   He would have marveled at the presentations in the book of many new and strong statistical and computer analyses.

Gudmund R. Iversen
Professor Emeritus of Statistics, Swarthmore College

While there is increasing interest in Bayesian statistics among scholars of different social science disciplines, I always looked for a text book which is accessible to a wide range of students who do not necessarily have extended knowledge of statistics. Now, I believe that this is the first textbook of Bayesian statistics, which can also be used for social science undergraduate students. Ben Lambert begins with a general introduction to statistical inference and successfully brings the readers to more specific and practical aspects of Bayesian inference. In addition to its well-considered structure, many graphical presentations and reasonable examples contribute for a broader audience to obtain well-founded understanding of Bayesian statistics. 

Susumu Shikano
Political Methodology, Centre for Data and Methods, University of Konstanz

This book offers a path to get into the field of Bayesian statistics with no previous knowledge. Building from elementary to advanced topics, including theoretic and computational aspects, and focusing on the application, it is an excellent read for newcomers to the Bayesian world.

Panagiotis Tsiamyrtzis
Statistics, Athens University of Economics and Business

This book was used as essential reading throughout my module (on an MSc level) not just for learning the “what”, “why” and “how” about the key principles and theory behind Bayesian statistics; but for understanding the practical component for implementing statistical analysis the Bayesian way using Stan interfaced with RStudio through RStan package, as well as for learning the Stan and RStan coding etiquettes for implementing Bayesian modelling and gaining its mastery in Stan and RStudio.

This is an excellent book, which is excellent for students who have been exposed to statistics (e.g., at least GLM, hierarchical regression etc.).

Dr Anwar Musah
Department of Geography, University College London
April 12, 2023

Hands down the best introduction to Bayesian approaches. Unlike other "introductions", Lambert doesn't assume an acquaintance with integral calculus and helps the student instead to build an intuition about Bayesian approaches (and their distinction from frequentist approaches). I'm sure this will take its place alongside Field's book on SPSS as a must-have for psychology undergraduates and post-graduates.

Professor Sunjeev Kamboj
Clinical, Educational and Health Psychology, University College London
January 20, 2020

Probably the best introductory textbook for bayesian statistics. - In particular, it is very applied, provides a modern and up-to-date introduction, as well as clear guides how to best use the book.

Dr Thees Spreckelsen
Department of Social Policy and Intervention, University of Oxford
April 8, 2019