Linear Regression without intercept: form of the i-th fitted value

[mck]

This is a question from the book "Introduction to Statistical Learning" and I don't know how to solve it.

---

5. Consider the fitted values that result from performing linear regression without an intercept. In this setting, the i-th fitted value takes the form

. . . . .\(\displaystyle \hat{y}_i\, =\, x_i\, \hat{\beta}\)

where

. . . . .\(\displaystyle \displaystyle \hat{\beta}\, =\, \left(\sum_{i=1}^n\, x_i\, y_i\right)\, /\, \left(\sum_{i' = 1}^n\, x_{i'}^2\right)\)

Show that we can write

. . . . .\(\displaystyle \displaystyle \hat{y}_i\, =\, \sum_{i' =1}^n\, a_{i'}\, y_{i'}\)

What is \(\displaystyle a_{i'}\)?

Note: We interpret this result by saying that the fitted values from linear regression are linear combinations of the response values.

---

I generally have problems with these types of questions (especially when summation signs are involved), is there a text book that explains these concepts with applications anyone can recommend?

If you could help me with this question I'd be very thankful!

 

[Dr.Peterson]

This is a question from the book "Introduction to Statistical Learning" and I don't know how to solve it.

---

5. Consider the fitted values that result from performing linear regression without an intercept. In this setting, the i-th fitted value takes the form

. . . . .\(\displaystyle \hat{y}_i\, =\, x_i\, \hat{\beta}\)

where

. . . . .\(\displaystyle \displaystyle \hat{\beta}\, =\, \left(\sum_{i=1}^n\, x_i\, y_i\right)\, /\, \left(\sum_{i' = 1}^n\, x_{i'}^2\right)\)

Show that we can write

. . . . .\(\displaystyle \displaystyle \hat{y}_i\, =\, \sum_{i' =1}^n\, a_{i'}\, y_{i'}\)

What is \(\displaystyle a_{i'}\)?

Note: We interpret this result by saying that the fitted values from linear regression are linear combinations of the response values.

---

I generally have problems with these types of questions (especially when summation signs are involved), is there a text book that explains these concepts with applications anyone can recommend?

If you could help me with this question I'd be very thankful!

One way to get more familiar with summations is to write them out for a small case, so you can use your knowledge of ordinary expressions. You are given this:

\(\displaystyle \displaystyle\hat{y}_i = \dfrac{\sum_{i=1}^{n}x_i y_i}{\sum_{i'=1}^{n}{x_{i'}}^2}\hat{x}_i\)

For n=3, that is

\(\displaystyle \displaystyle\hat{y}_i = \frac{x_1 y_1 + x_2 y_2 + x_3 y_3}{{x_1}^2 + {x_2}^2 + {x_2}^2}\hat{x}_i\)

But that denominator is independent of the y's; you can just divide each term by it separately:

\(\displaystyle \displaystyle\hat{y}_i = \left(\frac{x_1 y_1}{{x_1}^2 + {x_2}^2 + {x_3}^2} + \frac{x_2 y_2}{{x_1}^2 + {x_2}^2 + {x_3}^2} + \frac{x_3 y_3}{{x_1}^2 + {x_2}^2 + {x_3}^2}\right)\hat{x}_i\)

or

\(\displaystyle \displaystyle\hat{y}_i = \frac{x_1 y_1}{{x_1}^2 + {x_2}^2 + {x_2}^2}\hat{x}_i + \frac{x_2 y_2}{{x_1}^2 + {x_2}^2 + {x_3}^2}\hat{x}_i + \frac{x_3 y_3}{{x_1}^2 + {x_2}^2 + {x_3}^2}\hat{x}_i\)

Do the same thing with the goal expression, and see if you can match them up.

 

[mck]

One way to get more familiar with summations is to write them out for a small case, so you can use your knowledge of ordinary expressions. You are given this:

\(\displaystyle \displaystyle\hat{y}_i = \dfrac{\sum_{i=1}^{n}x_i y_i}{\sum_{i'=1}^{n}{x_{i'}}^2}\hat{x}_i\)

For n=3, that is

\(\displaystyle \displaystyle\hat{y}_i = \frac{x_1 y_1 + x_2 y_2 + x_3 y_3}{{x_1}^2 + {x_2}^2 + {x_2}^2}\hat{x}_i\)

But that denominator is independent of the y's; you can just divide each term by it separately:

\(\displaystyle \displaystyle\hat{y}_i = \left(\frac{x_1 y_1}{{x_1}^2 + {x_2}^2 + {x_3}^2} + \frac{x_2 y_2}{{x_1}^2 + {x_2}^2 + {x_3}^2} + \frac{x_3 y_3}{{x_1}^2 + {x_2}^2 + {x_3}^2}\right)\hat{x}_i\)

or

\(\displaystyle \displaystyle\hat{y}_i = \frac{x_1 y_1}{{x_1}^2 + {x_2}^2 + {x_2}^2}\hat{x}_i + \frac{x_2 y_2}{{x_1}^2 + {x_2}^2 + {x_3}^2}\hat{x}_i + \frac{x_3 y_3}{{x_1}^2 + {x_2}^2 + {x_3}^2}\hat{x}_i\)

Do the same thing with the goal expression, and see if you can match them up.

First of all, thanks a lot that has already helped

Ok, so when I do the same with the goal expression, I get

y_i=a₁y₁+a₂y₂₊a₃y₃

a₁=(x₁/x₁²)xi would that be a1? But how do I generalize that

 

[Dr.Peterson]

First of all, thanks a lot that has already helped

Ok, so when I do the same with the goal expression, I get

y_i=a₁y₁+a₂y₂₊a₃y₃

a₁=(x₁/x₁²)xi would that be a1? But how do I generalize that

You're leaving off the "hats", which changes everything; you meant this (doing the best I can without Latex for your sake):

y_i^=a₁y₁+a₂y₂₊a₃y₃

a₁=(x₁/x₁²)x_i^

This conclusion would be right if you hadn't changed the denominator, and used i some places and 1 in others.

Here's your summation written out:

\(\displaystyle \hat{y}_i=a_1 y_1 + a_2 y_2 + a_3 y_3\)

Here's mine:

\(\displaystyle \displaystyle \hat{y}_i=\frac{x_1 y_1 \hat{x}_i}{{x_1}^2 + {x_2}^2 + {x_2}^2} + \frac{x_2 y_2 \hat{x}_i}{{x_1}^2 + {x_2}^2 + {x_2}^2} + \frac{x_3 y_3 \hat{x}_i}{{x_1}^2 + {x_2}^2 + {x_2}^2}\)

You're just matching up the coefficients of y_i in each term (that is, of y₁, y₂, y₃ respectively), right? Do that for each term.