Multi-Variable chain rule

calvingloster

New member
Joined
Aug 15, 2018
Messages
2
Hi All


New to the forum. I am in the process of learning about backpropagation for use in AI. My maths is very rusty and i am stuck at intuitively understanding the chain rule. I have designed a spreadsheet as to confirm my understanding of how the chain rule works with multi-variable functions.


There is a function Z which is dependent on X and Y where X and Y are dependent on N. I want to know the following: If i make a small change to N how much will this affect the output of function Z. I get to an answer but this does not correspond to change in the output of Z when i actually make a small change to N. See below image of spreadsheet.
chain rule.jpg
 
I can't see the image. Furthermore, if x and y are functions of n, and z is a function of x and y, then z is just a function of n. What is multivariable about that? Finally, the chain rule applies to composition of functions. Is that what you have? In any case if you have a function why not calclulate it directly?

\(\displaystyle z(n) = x(n) + y(n) = 2n^3 - 3n^2 \implies\)

\(\displaystyle z'(n) = x'(n) + y'(n) = 6n(n - 1) \implies\)

\(\displaystyle z(n + \Delta n) \approx z(n) + ( z'(n) * \Delta n ) \text { for small } \Delta n.\)

\(\displaystyle z(4.01) = (2 * 4.01^3) - (3 * 4.01^2) = 2 * 64.481201 - (3 * 16.0801) =\)

\(\displaystyle 128.962402 - 48.2403 = 80.722102.\) Simple with a computer.

Now if you are working by hand it may be easier (and less error prone) to do this.

\(\displaystyle z(4) + (z'(4) * 0.01) = 2 * 64 - 3 * 16 + \{0.01 * 6 * 4 * (4 - 1)\} =\)

\(\displaystyle 128 - 48 + 0.01(72) = 80.72 \approx z(4.01). \) Error is minus 0.26%.

The chain rule comes in for more complex types of functions.
 
Last edited:
Hi All


New to the forum. I am in the process of learning about backpropagation for use in AI. My maths is very rusty and i am stuck at intuitively understanding the chain rule. I have designed a spreadsheet as to confirm my understanding of how the chain rule works with multi-variable functions.


There is a function Z which is dependent on X and Y where X and Y are dependent on N. I want to know the following: If i make a small change to N how much will this affect the output of function Z. I get to an answer but this does not correspond to change in the output of Z when i actually make a small change to N. See below image of spreadsheet.
View attachment 9944

I can't quite read your image. One way to ensure that an image has the appropriate resolution is to display your original on your computer and use the Windows snipping tool, or equivalent, to make a copy of the part you want us to see.

Once we have a better picture of what you are asking, we can help.
 
If we have f(x,y,z) and z is a function of x and y then we can write f as a function of x and y only, say f(x, y, z)= g(x,y), and, by the chain rule, have
\(\displaystyle \frac{\partial g}{\partial x}= \frac{\partial f}{\partial x}+ \frac{\partial f}{\partial z}\frac{\partial z}{\partial x}\)
and
\(\displaystyle \frac{\partial g}{\partial y}= \frac{\partial f}{\partial y}+ \frac{\partial f}{\partial z}\frac{\partial z}{\partial y}\)

If, further, x and y are functions of yet another variable, n, then we can write f(x, y, z)= g(x, y)= h(n) and have
\(\displaystyle \frac{dh}{dn}= \frac{\partial g}{\partial x}\frac{dx}{dn}+ \frac{\partial g}{\partial y}\frac{dy}{dn}\).

Inserting the expressons for \(\displaystyle \frac{\partial g}{\partial x}\) and \(\displaystyle \frac{\partial g}{\partial y}\) in terms of f, above, we have
\(\displaystyle \frac{dh}{dn}= \left(\frac{\partial f}{\partial x}+ \frac{\partial f}{\partial z}\frac{\partial z}{\partial x}\right)\frac{dx}{dn}+ \left(\frac{\partial f}{\partial y}+ \frac{\partial f}{\partial z}\frac{\partial z}{\partial y}\right)\frac{dy}{dn}\).

(Some text just use "f(x,y)" and "f(n)" where I have introduced "g(x,y)" and "h(n)" but I think that becomes confusing, with things like \(\displaystyle \frac{\partial f}{\partial x}\) on both sides of the equation with different meanings.)
 
Top