Help The Education Support Forum through MathsGee serve learners across Africa with a DONATION

0 like 0 dislike
7 views
1. Let $X_t$ be the independent variable, and $Y_t$ be the dependent variable. We will use $X_t$ to predict $Y_t$, using several models of regression.
2. $M1: Y_t =aX_t +eta$
3. $M2: Y_t = aX_t +b+ eta$
4. $M3: Y_t= aX^2_t +bX_t+c+eta$

where we fit the constants $a, b$ and $c$ from data. We assume that $\text{Eta}\sim N(0,\sigma^2)$ and $\sigma^2$ is estimated from the training data. Let us choose a model from M1, M2 and M3 using the AIC.

• How many degrees of freedom does each of the three models (M1, M2, ) M3 have?
• Let log likelihood(data| ML parameters of model M1) = -130.4, log likelihood(data| ML parameters of model M2) = -108.1, log likelihood(data| ML parameters of model M3) = -107.99 Based on the ML framework, which model should we choose ?
• Based on the AIC, which model should we choose? Does the choice change if we observe the same likelihoods, but learn that M1, M2 and M3 all had 2 more degrees of freedom ?

| 7 views

0 like 0 dislike

a. How many degrees of freedom does each of the three models (M1, M2, ) M3 have?
Degrees of freedom for Model 1 = 2 (a and eta)
Degrees of freedom for Model 1 = 3 (a, b and eta)
Degrees of freedom for Model 1 = 4 (a,b, c and eta)

b. Let log likelihood(data| ML parameters of model M1) = -130.4,
log likelihood(data| ML parameters of model M2) = -108.1,
log likelihood(data| ML parameters of model M3) = -107.99
Based on the ML framework, which model should we choose ?

According to the ML framework, we should choose the model with the max likelihood, and also max log likelihood : M3

c. Based on the AIC, which model should we choose? Does the choice change if we observe the same likelihoods, but learn that M1, M2 and M3 all had 2 more degrees of freedom ?
According to the ML framework, we should choose the model with the max likelihood, and also max log likelihood : M3
AIC score = log likelihood(data|model) – degrees of freedom
AIC score for M1 = -130.4 – 2

AIC score for M2 = -108.1 - 3
AIC score for M3 = -107.99 - 4

Thus, under maximizing AIC criterion, M2 should be chosen. If an equal number of degrees of freedom are added to every model, the model choice does not change if the likelihd is remains unchanged, as an equal number is decreased from each AIC score.

by Wooden (693 points)

1 like 0 dislike