Statistical Image Processing and Multidimensional Modeling

Paul Fieguth

Table of Contents (see also List of Examples, List of Code Samples, Link to Chapter 1 (PDF))

The text is divided into three parts:

The parts are designed to be complementary, respectively emphasizing mathematical theory, modeling, and algorithms. Click on the bold chapter headings below to expand / collapse details of the table of contents:

Part 0: Preamble and Introduction

Preamble and Introduction:

Table of Contents
List of Examples (link)
List of Code Samples (link)
Chapter 1: Introduction (Link to chapter PDF)

Part I: Inverse Problems and Estimation

Chapter 2: Inverse Problems

2.1 Data Fusion
2.2 Posedness
2.3 Conditioning
2.4 Regularization and Prior Models
2.4.1 Deterministic Regularization
2.4.2 Bayesian Regularization
2.5 Statistical Operations
2.5.1 Canonical Problems
2.5.2 Prior Sampling
2.5.3 Estimation
2.5.4 Posterior Sampling
2.5.5 Parameter Estimation
Application 2: Ocean Acoustic Tomography
Summary, Further Study, Sample Problems

Chapter 3: Static Estimation and Sampling

3.1 Non-Bayesian Estimation
3.2 Bayesian Estimation
3.2.1 Bayesian Static Problem
3.2.2 Bayesian Estimation and Prior Means
3.2.3 Approximate Bayesian Estimators
3.2.4 Bayesian / NonBayesian Duality
3.3 Static Sampling
3.4 Data Fusion
Application 3: Atmospheric Temperature Inversion
Summary, Further Study, Sample Problems

Chapter 4: Dynamic Estimation and Sampling

4.1 The Dynamic Problem
4.1.1 First-Order Gauss--Markov Processes
4.1.2 Static --- Dynamic Duality
4.2 Kalman Filter Derivation
4.3 Kalman Filter Variations
4.3.1 Kalman Filter Algorithms
4.3.2 Steady-State Kalman Filtering
4.3.3 Kalman Filter Smoother
4.3.4 Nonlinear Kalman Filtering
4.4 Dynamic Sampling
4.5 Dynamic Estimation for Discrete-State Systems
4.5.1 Markov Chains
4.5.2 The Viterbi Algorithm
4.5.3 Comparison to Kalman Filter
Application 4: Temporal Interpolation of Ocean Temperature
Summary, Further Study, Sample Problems

Part II: Modelling of Random Fields

Chapter 5: Multidimensional Modelling

5.1 Challenges
5.2 Coupling and Dimensionality Reduction
5.3 Sparse Storage and Computation
5.3.1 Sparse Matrices
5.3.2 Matrix Kernels
5.3.3 Computation
5.4 Modelling
5.5 Deterministic Models
5.5.1 Boundary Effects
5.5.2 Discontinuity Features
5.5.3 Prior-Mean Constraints
5.6 Statistical Models
5.6.1 Analytical Forms
5.6.2 Analytical Forms and Nonstationary Fields
5.6.3 Recursive / Dynamic Models
5.6.4 Banded Inverse-Covariances
5.7 Model Determination
5.8 Choice of Representation
Application 5: Synthetic Aperture Radar Interferometry
Further Study, Sample Problems

Chapter 6: Markov Random Fields

6.1 One-Dimensional Markovianity
6.1.1 Markov Chains
6.1.2 Gauss--Markov Processes
6.2 Multidimensional Markovianity
6.3 Gauss--Markov Random Fields
6.4 Causal Gauss--Markov Random Fields
6.5 Gibbs Random Fields
6.6 Model Determination
6.6.1 Autoregressive Model Learning
6.6.2 Noncausal Markov Model Learning
6.7 Choices of Representation
Application 6: Texture Classification
Summary, Further Study, Sample Problems

Chapter 7: Hidden Markov Models

7.1 Hidden Markov Models
7.1.1 Image Denoising
7.1.2 Image Segmentation
7.1.3 Texture Segmentation
7.1.4 Edge Detection
7.2 Classes of Joint Markov Models
7.3 Conditional Random Fields
7.4 Discrete-State Models
7.4.1 Local Gibbs Models
7.4.2 Nonlocal Statistical-Target Models
7.4.3 Local Joint Models
7.5 Model Determination
Application 7: Image Segmentation
Further Study, Sample Problems

Chapter 8: Changes of Basis

8.1 Change of Basis
8.2 Reduction of Basis
8.2.1 Principal Components
8.2.2 Multidimensional Basis Reduction
8.2.3 Local Processing
8.3 FFT Methods
8.3.1 FFT Diagonalization
8.3.2 FFT and Spatial Models
8.3.3 FFT Sampling and Estimation
8.4 Hierarchical Bases and Preconditioners
8.4.1 Interpolated Hierarchical Bases
8.4.2 Wavelet Hierarchical Bases
8.4.3 Wavelets and Statistics
8.5 Basis Changes and Markov Random Fields
8.6 Basis Changes and Discrete-State Fields
Application 8: Global Data Assimilation
Summary, Further Study, Sample Problems

Part III: Methods and Algorithms

Chapter 9: Linear Systems Estimation

9.1 Direct Solution
9.1.1 Gaussian Elimination
9.1.2 Cholesky Decomposition
9.1.3 Nested Dissection
9.2 Iterative Solution
9.2.1 Gauss-Jacobi / Gauss-Seidel
9.2.2 Successive Overrelaxation (SOR)
9.2.3 Conjugate Gradient and Krylov Methods
9.2.4 Iterative Preconditioning
9.2.5 Multigrid
Application 9: Surface Reconstruction
Further Study, Sample Problems

Chapter 10: Kalman Filtering and Domain Decomposition

10.1 Marching Methods
10.2 Efficient, Large-State Kalman Filters
10.2.1 Large-State Kalman Smoother
10.2.2 Steady-State KF
10.2.3 Strip KF
10.2.4 Reduced-Update KF
10.2.5 Sparse KF
10.2.6 Reduced-Order KF
10.3 Multiscale
Application 10: Video Denoising
Summary, Further Study, Sample Problems

Chapter 11: Sampling and Monte Carlo Methods

11.1 Dynamic Sampling
11.2 Static Sampling
11.2.1 FFT
11.2.2 Marching
11.2.3 Multiscale Sampling
11.3 MCMC
11.3.1 Stochastic Sampling
11.3.2 Continuous-State Sampling
11.3.3 Large-Scale Discrete-State Sampling
11.4 Nonparametric Sampling
Application 11: Multi-Instrument Fusion of Porous Media
Further Study, Sample Problems

Appendices and Postmatter

Appendix A: Algebra

A.1 Linear Algebra
A.2 Matrix Operations
A.3 Matrix Positivity
A.4 Matrix Positivity of Covariances
A.5 Matrix Types
A.6 Matrix / Vector Derivatives
A.7 Matrix Transformations
A.7.1 Eigendecompositions
A.7.2 Singular Value Decomposition
A.7.3 Cholesky, Gauss, LU, Gram--Schmidt, QR, Schur
A.8 Matrix Square Roots
A.9 Pseudoinverses

Appendix B: Statistics

B.1 Random Variables, Random Vectors, and Random Fields
B.1.1 Random Variables
B.1.2 Joint Statistics
B.1.3 Random Vectors
B.1.4 Random Fields
B.2 Transformation of Random Vectors
B.3 Multivariate Gaussian Distribution
B.4 Covariance Matrices

Appendix C: Image Processing

C.1 Convolution
C.2 Image Transforms
C.3 Image Operations

Bibliography and Index

Reference Summary

(Page last updated September 13, 2016, [an error occurred while processing this directive])