Introduction to State-Space Models

.title[
# Introduction to State-Space Models
]
.subtitle[
## Mark Scheuerell (use the arrows keys to navigate the slide deck)
]

---

# ESA status reviews

## .blue[Required every 5 years to ensure appropriate protections]

## .blue[Considerations include]

> ### .gray[population trends, distribution, demographics & genetics]

> ### .gray[habitat conditions]

> ### .gray[past conservation measures]

> ### .gray[status & trends of threats]

---

# Surely listed species are closely monitored?

---

---

# The data are gappy

---

# The data are noisy

## .center.green[
Non-exhaustive surveys
]
## .center.blue[
Undetected individuals
]
## .center.orange[
Misidentified individuals
]

---

# The data are disparate

## .center.green[
Direct vs remotely sensed
]
## .center.blue[
Design-based vs opportunistic
]
## .center.orange[
Scientific surveys vs recreational sampling
]

---

> # .blue[
How can we use this information to inform decisions?
]
> # .green[
For trends over time, one option is a state-space model
]

---

# Part 1: State model

## Describes the .blue[true state of nature] over time

---

# States of nature might be

## .center.green[
Animal location
]
## .center.blue[
Species density
]
## .center.orange[
Age structure
]
## .center.purple[
Reproductive status
]

---

# Changes in the state of nature

## .blue[Generally driven by a combination of]

## .blue[&#x2022; intrinsic (eg, fecundity)]

## .blue[&#x2022; extrinsic factors (eg, temperature)]

---

# Process errors

## .blue[Some of the extrinsic drivers may be unknown]

## .blue[We often call these unknown extrinsic factors] _process errors_

---

class: frimg, center, bottom, inverse-black
background-image: url(figs/venice_carnival.jpg)
background-size: 100%

## .white[Revealing the true state requires observations]

---

# Observation (sampling) errors

## .blue[Collecting data depends on many factors]

### &#x2022; Environmental conditions (eg, cloud cover)

### &#x2022; Behavior (eg, threat avoidance)

### &#x2022; Demographics (age, sex, maturity)

### &#x2022; Sampling design/coverage

### &#x2022; Observer skill

---

# Observing nature can be easy

---

class: frimg, bottom, right
background-image: url(figs/sockeye.jpg)
background-size: cover

# .white[How many sockeye are there?]

---

# Observing nature can also be hard

---

class: frimg, bottom, right
background-image: url(figs/sockeye.jpg)
background-size: cover

# .white[How many mayflies are there?]

---

# Part 2: Observation model

# .center[.purple[Data] = .blue[Truth] &#177; .red[Errors]]

---

# Part 2: Observation model

---

# Why bother?

---

# Advantages

## 1. Can combine many different .purple[data types]

## .center.purple[
Changes in observers or sensors
]

## .center.purple[
Varying survey locations & effort
]

## .center.purple[
Direct & remote sampling  
]

---

# Advantages

## 2. .gray[Missing data] are easily accommodated

---

class: frimg
background-image: url(figs/Jonsen2016.png)
background-position: 50% 70%
background-size: 70%

# Advantages

## 3. Improved accuracy & precision

---

class: frimg
background-image: url(figs/robin.jpg)
background-position: 50% 80%
background-size: 30%

# Advantages

## 4. .gray[Data-poor] benefit from .green[data-rich]

---

# State-space model | General form

## .blue[State model (autoregressive)]

`\(\LARGE x_t = f(x_{t-1}, \text{process error})\)`

## .purple[Observation model]

`\(\LARGE y_t = g(x_t, \text{observation error})\)`

---

# State-space model | Random walk

## .blue[State model]

`\(\LARGE x_t = x_{t-1} + w_t ~~~~~~~~~~~ w_t \sim \text{N}(0,q)\)`

## .purple[Observation model]

`\(\LARGE y_t = x_t + v_t ~~~~~~~~~~~ v_t \sim \text{N}(0,r)\)`

---

# Examples of random walks

---

# State-space model | Biased random walk

## .blue[State model]

`\(\LARGE x_t = x_{t-1} + u + w_t ~~~~~~~~~~~ w_t \sim \text{N}(0,q)\)`

## .purple[Observation model]

`\(\LARGE y_t = x_t + v_t ~~~~~~~~~~~ v_t \sim \text{N}(0,r)\)`

---

# Examples of biased random walks

---

# State-space model | Mean-reverting

## .blue[State model]

`\(\LARGE x_t = b x_{t-1} + w_t ~~~~~~~~~~ |b| < 1 ~~~~~~~~~~~ w_t \sim \text{N}(0,q)\)`

## .purple[Observation model]

`\(\LARGE y_t = x_t + v_t ~~~~~~~~~~~ v_t \sim \text{N}(0,r)\)`

---

# Examples of mean-reverting models

---

# Consider these data

---

# Two model options

## .blue[1) Linear regression]

`\(\LARGE y_t = \alpha + \beta t + e_t\)`

## .green[2) Biased random walk]

`\(\LARGE x_t = x_{t-1} + u + e_t\)`

---

# 1) Linear regression as a state-space model

## .blue[State model]

`\(\LARGE x_t = \alpha + \beta t\)`

## .purple[Observation model]

`\(\LARGE y_t = x_t + v_t ~~~~~~~~~~~ v_t \sim \text{N}(0,r)\)`

---

# Estimated state

---

# Observation errors

---

# 2) Biased random walk as a state-space model

## .blue[State model]

`\(\LARGE x_t = x_{t-1} + u + w_t ~~~~~~~~~~~ w_t \sim \text{N}(0,q)\)`

## .purple[Observation model]

`\(\LARGE y_t = x_t\)`

---

# Estimated state

---

# Process errors

---

# What if we combined the two models?

---

# Biased random walk with observation error

## .blue[State model]

`\(\LARGE x_t = x_{t-1} + u + w_t ~~~~~~~~~~~ w_t \sim \text{N}(0,q)\)`

## .purple[Observation model]

`\(\LARGE y_t = x_t + v_t ~~~~~~~~~~~ v_t \sim \text{N}(0,r)\)`

---

# Estimated state

---

# Process errors

---

# Observation errors

---

> # .blue[How is it possible to separate the process and observation errors?]

> # .green[They have different temporal patterns]

---

---

# Model fitting: The Kalman filter

### .blue[In the 1950s, Rudolf Kalman began developing a new algorithm for processing signals obtained from noisy sensors]

### .blue[In the 1960s, NASA used it for guidance & navigation computers* in Apollo missions]

### .blue[Now, the so-called "Kalman filter" is now widely used in statistics, aeronautics & robotics]

.citation.gray[*Mark's laptop has 8 million times the RAM]

---

# Kalman filter: 3 types of estimates

---

# More information

## [.purple[Time series course at UW (FISH 550)]](https://atsa-es.github.io/atsa/)

## [.purple[eBook]](https://atsa-es.github.io/atsa-labs/)

## [.purple[YouTube channel]](https://www.youtube.com/@SAFSTimeSeries)

---

class: frimg
background-image: url(figs/SSM_review.png)
background-position: 50% 50%
background-size: 80%

# More information