A simple linear model

Bayesian workshop - STEP 2023

Scott James Perry

University of Alberta

By the end of this lesson, you will be able to…

  1. List all the components of a simple linear model
  2. Specify priors for model with one continuous predictor
  3. Explain the concept behind how we fit Bayesian models
  4. Create simple visualizations of posterior distributions

Building a linear model


Using a Normal distribution to model reaction times:

  • mean rt - \(\mu\)
  • standard deviation - \(\sigma\)


Modelling changes in the mean only


Most models in linguistics model changes in the mean only


The distribution moves up or down based on another predictor

Lexical frequency and reaction times

  1. Model rt coming from Normal distribution with mean (\(\mu\)) and standard deviation (\(\sigma\))
  2. Our mean can change based on values of Frequency (Conditional mean)
  • What’s the problem having the mean rt equal Frequency here?


\(rt \sim Normal(\mu,\sigma)\)
\(\mu = Frequency\)

Adding \(\beta_1\) allows different relationships between Frequency and rt


  • If \(\beta_1 = 0\), rt does not change with Frequency
  • If \(\beta_1 > 0\), rt \(\uparrow\) when Frequency \(\uparrow\)
  • If \(\beta_1 < 0\), rt \(\downarrow\) when Frequency \(\uparrow\)
  • \(|\beta_1|\) determines how much rt changes


\(rt \sim Normal(\mu,\sigma)\)
\(\mu = \beta_1Frequency\)



In this model, what happens when Frequency = 0?


  • If \(\beta_1 = 0\), rt does not change with Frequency
  • If \(\beta_1 > 0\), rt \(\uparrow\) when Frequency \(\uparrow\)
  • If \(\beta_1 < 0\), rt \(\downarrow\) when Frequency \(\uparrow\)
  • \(|\beta_1|\) determines how much rt changes


\(rt \sim Normal(\mu,\sigma)\)
\(\mu = \beta_1Frequency\)



Model with an intercept


We add another parameter: Intercept


This is the value of rt when Frequency = 0


\(rt \sim Normal(\mu,\sigma)\)
\(\mu = Intercept + \beta_1 Frequency\)

A full description of a Bayesian model


This is how Bayesian models are often written. We specify a model, then specify priors for all parameters

Model:
\(rt \sim Normal(\mu,\sigma)\)
\(\mu = Intercept + \beta_1 Frequency\)

Priors:

\(Intercept \sim Normal(?,?)\)
\(\beta_1 \sim Normal(?,?)\)
\(\sigma \sim Normal_+(?,?)\)

What priors do we need to set?


Intercept:
- Prior belief on what rt could be when Frequency = 0

\(\beta_1\):
- Prior belief on how rt changes when Frequency increases by 1

\(\sigma\):
- Prior belief on how variable reaction times are


Model:
\(rt \sim Normal(\mu,\sigma)\)
\(\mu = Intercept + \beta_1 Frequency\)

Priors:

\(Intercept \sim Normal(?,?)\)
\(\beta_1 \sim Normal(?,?)\)
\(\sigma \sim Normal_+(?,?)\)

Let’s practice setting priors for a model predicting rt by Frequency



Open the script entitled S2_E1_linear_model.R