Get Help And Discuss STEM Concepts From Math To Data Science & Financial Literacy
STEM Gender Equality | Join us on ZOOM | Spreading Mathematical Happiness

Help The Education Support Forum through MathsGee serve learners across Africa with a DONATION

0 like 0 dislike
7 views
  1. Let $X_t$ be the independent variable, and $Y_t$ be the dependent variable. We will use $X_t$ to predict $Y_t$, using several models of regression.
  2. $M1: Y_t =aX_t +eta$
  3. $M2: Y_t = aX_t +b+ eta$
  4. $M3: Y_t= aX^2_t +bX_t+c+eta$

where we fit the constants $a, b$ and $c$ from data. We assume that $\text{Eta}\sim N(0,\sigma^2)$ and $\sigma^2$ is estimated from the training data. Let us choose a model from M1, M2 and M3 using the AIC. 

  • How many degrees of freedom does each of the three models (M1, M2, ) M3 have? 
  • Let log likelihood(data| ML parameters of model M1) = -130.4, log likelihood(data| ML parameters of model M2) = -108.1, log likelihood(data| ML parameters of model M3) = -107.99 Based on the ML framework, which model should we choose ?
  • Based on the AIC, which model should we choose? Does the choice change if we observe the same likelihoods, but learn that M1, M2 and M3 all had 2 more degrees of freedom ?

 

in Data Science by Diamond (49,543 points) | 7 views

1 Answer

0 like 0 dislike

a. How many degrees of freedom does each of the three models (M1, M2, ) M3 have?
Degrees of freedom for Model 1 = 2 (a and eta)
Degrees of freedom for Model 1 = 3 (a, b and eta)
Degrees of freedom for Model 1 = 4 (a,b, c and eta)


b. Let log likelihood(data| ML parameters of model M1) = -130.4,
log likelihood(data| ML parameters of model M2) = -108.1,
log likelihood(data| ML parameters of model M3) = -107.99
Based on the ML framework, which model should we choose ?


According to the ML framework, we should choose the model with the max likelihood, and also max log likelihood : M3

c. Based on the AIC, which model should we choose? Does the choice change if we observe the same likelihoods, but learn that M1, M2 and M3 all had 2 more degrees of freedom ?
According to the ML framework, we should choose the model with the max likelihood, and also max log likelihood : M3
AIC score = log likelihood(data|model) – degrees of freedom
AIC score for M1 = -130.4 – 2

AIC score for M2 = -108.1 - 3
AIC score for M3 = -107.99 - 4


Thus, under maximizing AIC criterion, M2 should be chosen. If an equal number of degrees of freedom are added to every model, the model choice does not change if the likelihd is remains unchanged, as an equal number is decreased from each AIC score.

by Wooden (693 points)

Welcome to MathsGee Q&A Bank, Africa’s largest personalized STEM and Financial Literacy education network that helps people find answers to problems, connect with others and take action to improve their outcomes.


MathsGee Q&A is the STEM and Financial Literacy knowledge-sharing community where students and experts put their heads together to crack their toughest questions.


Help The Education Support Forum through MathsGee serve learners across Africa with a DONATION

Enter your email address:

11,239 questions
9,393 answers
96 comments
10,231 users