Multivariate Linear Regression Models

[zoshkil]

Please check the question in Subhotosh's post below.

Am so in need of help!!

 

[Deleted member 4993]

please check the question out on this link as the file was too big to upload and the equations couldn't be writte using above tools. Am so in need of help!!

Please share your work with us .

If you are stuck at the beginning tell us and we'll start with the definitions e.g. define linear Regression Model

You need to read the rules of this forum. Please read the post titled "Read before Posting" at the following URL:

http://www.freemathhelp.com/forum/th...217#post322217

 

[zoshkil]

Everytime I click on the rules it just comes up with ERROR? I understand the basics of it hence why I posted this question in the advanced section, it's just this particular question and it's confusing variables that I am not understanding! Hope someone out there knows how to do this!

 

[Deleted member 4993]

Everytime I click on the rules it just comes up with ERROR? I understand the basics of it hence why I posted this question in the advanced section, it's just this particular question and it's confusing variables that I am not understanding! Hope someone out there knows how to do this!

Although you say you understand the basics, but you don't show any work!!

In the fist equation - what does y, X, ß and u represent?

 

[zoshkil]

I don't know what they represent thats why I am uploading the question? Last time I did I received about three brilliant answers to my question which not only answered the questions, but helped me to further understand the question and learn from their answers? Why has my thread been capped? I'm really confused, if you can't answer it can you put it back up on the thread so others can peruse it and see if they can help me?

 

[zoshkil]

I just need some help hence why I can't show any working, I just am not sure where to begin....

 

[DrPhil]

Everytime I click on the rules it just comes up with ERROR? I understand the basics of it hence why I posted this question in the advanced section, it's just this particular question and it's confusing variables that I am not understanding! Hope someone out there knows how to do this!

Pay particular to which variables are vectors and which are matrices, and what the dimensions are. For instance, consider the first equation to be a system of N equations in k unknowns. Each measured value $\displaystyle y_j$ is a linear combination of $\displaystyle k$ independent variables, but the coefficients $\displaystyle \beta$ are to be determined:

$\displaystyle \displaystyle y_j = \sum_{i=1}^k X_{j,i}\ \beta _i \ + u_j$

Does it help to expand the vector equations like this? Can you now explain what y, X, ß, and u represent?

The subsequent equations result from matrix operations. Note that the author did not propagate $\displaystyle u$ through the analysis, but only ask you to consider it after the fitting has been done.

 

[DrPhil]

see "Probability / Statistics" forum - please don't duplicate

http://www.freemathhelp.com/forum/t...-Linear-Regression-Models?p=342413#post342413

 

[DrPhil]

I just need some help hence why I can't show any working, I just am not sure where to begin....

Perhaps taking a step backwards . . .

Suppose you have a MODEL for a process, but you can't predict the coefficients of the model. For instance, you believe $\displaystyle y$ should be proportional to A, but with a correction proportional to A^2, and another term depending on Z. Your model is

$\displaystyle y = \beta_1 \times A + \beta_2 \times A^2 + \beta_3 \times Z$

So you set up an experiment to measure y for as wide a range of the parameters (A,Z) as you can manage. Note that A and Z are either controlled, or are found as part of the measurement. In either case they are known "perfectly" without any uncertainty. All of the uncertainty of each measurement is associated with $\displaystyle y$, and is usually expressed as a standard deviation, $\displaystyle u$.

1st measurement: $\displaystyle y_1 = \beta_1 A_1 + \beta_2 A_1^2 + \beta_3 Z_1 + u_1$
2nd....................$\displaystyle y_2 = \beta_1 A_2 + \beta_2 A_2^2 + \beta_3 Z_2 + u_2$
. . .
jth......................$\displaystyle y_j = \beta_1 A_j + \beta_2 A_j^2 + \beta_3 Z_j + u_j$

The next task is to find a set of coefficients $\displaystyle (\beta_1\ \beta_2\ \beta_3)$ that make a "best fit" of the model to the data. One of the most frequently used procedures is "ordinary least squares," in which the sum of the squares of the differences between the measured and predicted values of $\displaystyle y$ is minimized. ["Ordinary" means the uncertainties $\displaystyle u$ are not propagated - my own preference is to do a "weighted" least squares with each datum weighted as $\displaystyle 1/u_j^2$.]

Let.........$\displaystyle \displaystyle S = \sum_{j=1}^N (y_j - \hat y)^2 = \sum_{j=1}^N(y_j - \beta_1 A_j - \beta_2 A_j^2 - \beta_3 Z_j )^2$

then set..$\displaystyle \dfrac{\partial S}{\partial \beta_i} = 0\ \ \ \text{ for }i = 1 \text{ to }k$

which is now a set of $\displaystyle k$ equations in $\displaystyle k$ unknowns. The questions you are being asked have to do with the properties of this system of equations.

Does it help to start with a somewhat more explicit model?