The use of computers in mathematics and statistics has opened up a wide range of techniques for studying otherwise intractable problems. Sampling-based simulation techniques are now an invaluable tool for exploring statistical models. This book gives a comprehensive introduction to the exciting area of sampling-based methods.

An Introduction to Statistical Computing introduces the classical topics of random number generation and Monte Carlo methods. It also includes some advanced methods such as the reversible jump Markov chain Monte Carlo algorithm and modern methods such as approximate Bayesian computation and multilevel Monte Carlo techniques

An Introduction to Statistical Computing:

• Fully covers the traditional topics of statistical computing.
• Discusses both practical aspects and the theoretical background.
• Includes a chapter about continuous-time models.
• Illustrates all methods using examples and exercises.
• Provides answers to the exercises (using the statistical computing environment R); the corresponding source code is available online.
• Includes an introduction to programming in R.

This book is mostly self-contained; the only prerequisites are basic knowledge of probability up to the law of large numbers. Careful presentation and examples make this book accessible to a wide range of students and suitable for self-study or as the basis of a taught course.

List of Algorithms

Preface

Frequently Used Notation

1 Random Number Generation

1.1 Pseudo Random Number Generators

1.1.1 The Linear Congruential Generator

1.1.2 Quality of Pseudo Random Number Generators

1.1.3 Pseudo Random Number Generators in Practice

1.2 Discrete Distributions

1.3 The Inverse Transform Method

1.4 Rejection Sampling

1.4.1 Basic Rejection Sampling

1.4.2 Envelope Rejection Sampling

1.4.3 Conditional Distributions

1.4.4 Geometric Interpretation

1.5 Transformation of Random Variables

1.6 Special-Purpose Methods

1.7 Summary and Further Reading

Exercises

2 Simulating Statistical Models 45

2.1 Multivariate Normal Distributions

2.2 Hierarchical Models

2.3 Markov Chains

2.3.1 Discrete State Space

2.3.2 Continuous State Space

2.4 Poisson Processes

2.5 Summary and Further Reading

Exercises

3 Monte Carlo Methods

3.1 Studying Models via Simulation

3.2 Monte Carlo Estimates

3.2.1 Computing Monte Carlo Estimates

3.2.2 Monte Carlo Error

3.2.3 Choice of Sample Size

3.2.4 Re_ned Error Bounds

3.3 Variance Reduction Methods

3.3.1 Importance Sampling

3.3.2 Antithetic Variables

3.3.3 Control Variates

3.4 Applications to Statistical Inference

3.4.1 Point Estimators

3.4.2 Con_dence Intervals

3.4.3 Hypothesis Tests

3.5 Summary and Further Reading

Exercises

4 Markov Chain Monte Carlo Methods

4.1 The Metropolis-Hastings Method

4.1.1 Continuous State Space

4.1.2 Discrete State Space

4.1.3 Random Walk Metropolis Sampling

4.1.4 The Independence Sampler

4.1.5 Metropolis-Hastings with Di_erent Move Types

4.2 Convergence of MCMC Methods

4.2.1 Theoretical Results

4.2.2 Practical Considerations

4.3 Applications to Bayesian Inference

4.4 The Gibbs Sampler

4.4.1 Description of the Method

4.4.2 Application to Parameter Estimation

4.4.3 Applications to Image Processing

4.5 Reversible Jump MCMC

4.5.1 Description of the Method

4.5.2 Bayesian Inference for Mixture Distributions

4.6 Summary and Further Reading

Exercises

5 Beyond Monte Carlo

5.1 Approximate Bayesian Computation

5.1.1 Basic ABC

5.1.2 ABC With Regression

5.2 Resampling Methods

5.2.1 Bootstrap Estimates

5.2.2 Applications to Statistical Inference

5.3 Summary and Further Reading

Exercises

6 Continuous-Time Models

6.1 Time Discretisation

6.2 Brownian Motion

6.2.1 Properties

6.2.2 Direct Simulation

6.2.3 Interpolation and Brownian Bridges

6.3 Geometric Brownian Motion

6.4 Stochastic Di_erential Equations

6.4.1 Introduction

6.4.2 Stochastic Analysis

6.4.3 Discretisation Schemes

6.4.4 Discretisation Error

6.5 Monte Carlo Estimates

6.5.1 Basic Monte Carlo

6.5.2 Variance Reduction Methods

6.5.3 Multi-Level Monte Carlo Estimates

6.6 Application to Option Pricing

6.7 Summary and Further Reading

Exercises

Appendix A Probability Reminders

A.1 Events and Probability

A.2 Conditional Probability

A.3 Expectation

A.4 Limit Theorems

A.5 Further Reading

Appendix B Programming in R

B.1 General Advice

B.2 R as a calculator

B.2.1 Mathematical Operations

B.2.2 Variables

B.2.3 Data Types

B.3 Programming Principles

B.3.1 Don't Repeat Yourself!

B.3.2 Divide and Conquer!

B.3.3 Test your Code!

B.4 Random Number Generation

B.5 Summary and Further Reading

Exercises

Appendix C Answers to the Exercises

C.1 Answers for Chapter 1

C.2 Answers for Chapter 2

C.3 Answers for Chapter 3

C.4 Answers for Chapter 4

C.5 Answers for Chapter 5

C.6 Answers for Chapter 6

C.7 Answers for Appendix B

References

Index