The Problem
I'm trying to solve (in a practical sense) for the best parameters in a set of linear equations in an OLS style that look almost like the usual OLS but not quite. It's like this, for i=1 to N:
yi = ki ( x1i + a x2i + b x3i )
There N equations.
I know the "target" value of each Yi, so I know the residuals for any given sets of a,b,K.
a and b want to be constant across all equations. But ki comes in "families". So for example I might have 22 equations in total, where 6 of them share the same k, six share another k, and the remaining 8 share a third k.
So I could write it like this: yij = ki ( x1ij + a x2ij + b x3ij)
where j goes from 1 to the number of elements in the family and i goes from 1 to the number of families.
I need to solve for the "best" a, b, and all the Ks.
In reality there are many more equations and families, but always the number of families is a fraction of the number of equations (about one tenth) and there are always more than one elements in a family.
What I Think
I can see this could be done with OLS if it weren't for the Ks (I could stand to have a parameter with x1 as well as x2 and x3 - I guess it makes no diff but might be a more elegant solution). I guess I could do a gradient descent type solution but I get the feeling this is doable closed form, and I would rather a closed-form solution than a gradient descent. I just about understand the linear algebra of OLS but I'm not good enough to be creative with it.
What I have tried
I have tried to figure out the right linear algebra solution, staying close to OLS, but I'm just not good enough at linear algebra and I'm not sure it's the right approach because I don't know how to express the families of k.
Initially I tried to put the Ki inside the bracket and do OLS but then I'm solving for ki, kia and kib, so I can figure out a and b but they're no longer constant across all equations, and I can't force Ki to be equal for all equations in the family.
Please help!
I'm trying to solve (in a practical sense) for the best parameters in a set of linear equations in an OLS style that look almost like the usual OLS but not quite. It's like this, for i=1 to N:
yi = ki ( x1i + a x2i + b x3i )
There N equations.
I know the "target" value of each Yi, so I know the residuals for any given sets of a,b,K.
a and b want to be constant across all equations. But ki comes in "families". So for example I might have 22 equations in total, where 6 of them share the same k, six share another k, and the remaining 8 share a third k.
So I could write it like this: yij = ki ( x1ij + a x2ij + b x3ij)
where j goes from 1 to the number of elements in the family and i goes from 1 to the number of families.
I need to solve for the "best" a, b, and all the Ks.
In reality there are many more equations and families, but always the number of families is a fraction of the number of equations (about one tenth) and there are always more than one elements in a family.
What I Think
I can see this could be done with OLS if it weren't for the Ks (I could stand to have a parameter with x1 as well as x2 and x3 - I guess it makes no diff but might be a more elegant solution). I guess I could do a gradient descent type solution but I get the feeling this is doable closed form, and I would rather a closed-form solution than a gradient descent. I just about understand the linear algebra of OLS but I'm not good enough to be creative with it.
What I have tried
I have tried to figure out the right linear algebra solution, staying close to OLS, but I'm just not good enough at linear algebra and I'm not sure it's the right approach because I don't know how to express the families of k.
Initially I tried to put the Ki inside the bracket and do OLS but then I'm solving for ki, kia and kib, so I can figure out a and b but they're no longer constant across all equations, and I can't force Ki to be equal for all equations in the family.
Please help!