Sum of squared erros of a function of several variables?

[Cait]

Hello! I need to answer this question asap, but I don't know how exactly to do so. Could you help me, please?

In theory, the execution time T of a program should be a linear function of the size N of one it's inputs. Ideally, we should have T=aN+b, for the real constants a and b. But when the program was tested, the actual values were:

T	N
5	1
10	3
15	4

a) Write the function f(a,b) of the sum of the squared erros.
b) Find the values of a*,b* that minimize f.


I didn't understand how should I do the first one. I found out that the actual function of the program is g(N)=5N²/6 - 5N/6 + 5. Ok, this is an one variable function. Then, the squared error should be (g(N) - (aN+b))².

But this is a three variable function f(a, b, N). This means that it's the function of the error on the point N, right? But I need to find a two variable function... And what does "the sum of the squared errors" mean? The integral? Shouldn't the sum of the erros of this two functions be infinity? It really looks ugly and this just put more doubts in my head. D:

And then, for the b question, should I take this (now imaginary) f and find the partial derivatives and do the Lagrange multipliers? Or there is a better way? (Sorry, I just learned that way till now)


Any guidance is really appreciated!
Thank you in advance. c:

 

[Ishuda]

Hello! I need to answer this question asap, but I don't know how exactly to do so. Could you help me, please?

In theory, the execution time T of a program should be a linear function of the size N of one it's inputs. Ideally, we should have T=aN+b, for the real constants a and b. But when the program was tested, the actual values were:

T	N
5	1
10	3
15	4

a) Write the function f(a,b) of the sum of the squared erros.
b) Find the values of a*,b* that minimize f.


I didn't understand how should I do the first one. I found out that the actual function of the program is g(N)=5N²/6 - 5N/6 + 5. Ok, this is an one variable function. Then, the squared error should be (g(N) - (aN+b))².

But this is a three variable function f(a, b, N). This means that it's the function of the error on the point N, right? But I need to find a two variable function... And what does "the sum of the squared errors" mean? The integral? Shouldn't the sum of the erros of this two functions be infinity? It really looks ugly and this just put more doubts in my head. D:

And then, for the b question, should I take this (now imaginary) f and find the partial derivatives and do the Lagrange multipliers? Or there is a better way? (Sorry, I just learned that way till now)


Any guidance is really appreciated!
Thank you in advance. c:

Let's suppose we have a set of measurements (your T) for a set of independent variables (your N) given by
{n_j, t_j} (your table of T and N). Assume further that t is a function of n of the form
t(n) = a n + b.
Your measurements contain some errors so your date really looks like
t_i = a n_j + b + e_j
where e_j is the error. This error can consist of several things, for example a measurement error or an error in assuming that t is linear in n. What we would like to do is to choose a and b to minimize the squared error [if we use just the simple error we could have some large positive and large negative error which would balance out which would be bad, if we use the absolute value error, we would have a hard time working with the equations, etc.] So

E = \(\displaystyle \Sigma\, e_i^2\, =\, \Sigma\, (t_i\, -\, a\, n_i\, -\, b)^2 =\, \Sigma\, (\, a\, n_i\, +\, b\, -\, t_i)^2\)

E is the sum of squared errors and it is a function of a and b and we want to minimize E with respect to a and b. How do we do that? Well, take the (partial) derivatives and set them to zero.

\(\displaystyle \frac{\partial\, E}{\partial\, a}\, =\, 2\, \Sigma\, n_i (a\, n_i\, +\, b\, -\, t_i)\, =\, 0\)
\(\displaystyle \frac{\partial\, E}{\partial\, b}\, =\, 2\, \Sigma\, \,\,\,\,\, (a\, n_i\, +\, b\, -\, t_i)\, =\, 0\)
which leads to the usual equations
\(\displaystyle a\, \Sigma\, n_i^2\, +\, b\, \Sigma\,n_i\, =\, \Sigma\, n_i\, t_i\)
\(\displaystyle a\, \Sigma\, n_i\, +\, b\, \Sigma\,1\, =\, \Sigma\, t_i\)

Notice that these are linear formulas in a and b and that is why it is called linear regression.

Is that enough of a hint?

 

[Cait]

Let's suppose we have a set of measurements (your T) for a set of independent variables (your N) given by
{n_j, t_j} (your table of T and N). Assume further that t is a function of n of the form
t(n) = a n + b.
Your measurements contain some errors so your date really looks like
t_i = a n_j + b + e_j
where e_j is the error. This error can consist of several things, for example a measurement error or an error in assuming that t is linear in n. What we would like to do is to choose a and b to minimize the squared error [if we use just the simple error we could have some large positive and large negative error which would balance out which would be bad, if we use the absolute value error, we would have a hard time working with the equations, etc.] So

E = \(\displaystyle \Sigma\, e_i^2\, =\, \Sigma\, (t_i\, -\, a\, n_i\, -\, b)^2 =\, \Sigma\, (\, a\, n_i\, +\, b\, -\, t_i)^2\)

E is the sum of squared errors and it is a function of a and b and we want to minimize E with respect to a and b. How do we do that? Well, take the (partial) derivatives and set them to zero.

\(\displaystyle \frac{\partial\, E}{\partial\, a}\, =\, 2\, \Sigma\, n_i (a\, n_i\, +\, b\, -\, t_i)\, =\, 0\)
\(\displaystyle \frac{\partial\, E}{\partial\, b}\, =\, 2\, \Sigma\, \,\,\,\,\, (a\, n_i\, +\, b\, -\, t_i)\, =\, 0\)
which leads to the usual equations
\(\displaystyle a\, \Sigma\, n_i^2\, +\, b\, \Sigma\,n_i\, =\, \Sigma\, n_i\, t_i\)
\(\displaystyle a\, \Sigma\, n_i\, +\, b\, \Sigma\,1\, =\, \Sigma\, t_i\)

Notice that these are linear formulas in a and b and that is why it is called linear regression.

Is that enough of a hint?

Thank you very much for replying Ishuda! Best hint ever! This helped me a lot, really. I found a= 45/14 and b=10/7. I hope it's right! xD

 

[Ishuda]

Thank you very much for replying Ishuda! Best hint ever! This helped me a lot, really. I found a= 45/14 and b=10/7. I hope it's right! xD

Sounds good to me. Oh, one way to check is to put the values back in the equation. That is, compute e_i and n_i*e_i, add them up (individually) and see if they are zero.

 

[Cait]

Sounds good to me. Oh, one way to check is to put the values back in the equation. That is, compute e_i and n_i*e_i, add them up (individually) and see if they are zero.

Yes, I just checked and it's right! Thanks again!