Taking the derivative of a sigmoid function with respect to x

I

illo

New member
Joined
Mar 31, 2021
Messages
7
I have worked out a solution but am getting conflicting answers with a friend. I was just hoping someone could verify the result I am getting.
Thank you for your time.

Question:

Find the derivative of function [MATH]f(x)=\sigma(wx+b)[/MATH] with respect to [MATH]x[/MATH], where [MATH]\sigma(z)=\frac{1}{1+e^{-z}}[/MATH]
My Solution:

[MATH]f(x)=\frac{1}{1+e^{-(wx+b)}}[/MATH]
[MATH]\Rightarrow \frac{\partial{f(x)}}{\partial{x}}= \frac{\partial}{\partial{x}}(1+e^{-(wx+b)})^{-1}[/MATH]
[MATH]\Rightarrow \frac{\partial{f(x)}}{\partial{x}}=-(1+e^{-(wx+b)})^{-2}\cdot \frac{\partial}{\partial{x}}(1+e^{-(wx+b)})[/MATH]
[MATH]\Rightarrow \frac{\partial{f(x)}}{\partial{x}}=-\frac{(\frac{\partial}{\partial{x}}(1)+\frac{\partial}{\partial{x}}(e^{-(wx+b)})}{(1+e^{-(wx+b)})^2}[/MATH]
let [MATH]u=g=−b−wx[/MATH] and [MATH]f(u)=e^u.[/MATH]
[MATH]\Rightarrow \frac{\partial{f(x)}}{\partial{x}}=-\frac{\frac{\partial}{\partial{u}}(e^{u})\cdot\frac{\partial}{\partial{x}}(-b-wx)}{(1+e^{-(wx+b)})^2}[/MATH]
[MATH]\Rightarrow \frac{\partial{f(x)}}{\partial{x}}=-\frac{-we^{-(wx+b)}}{(1+e^{-(wx+b)})^2}[/MATH]

[MATH]\boxed{\Rightarrow \frac{\partial{f(x)}}{\partial{x}}=\frac{we^{-(wx+b)}}{(1+e^{-(wx+b)})^2}}[/MATH]
 
illo said:
I have worked out a solution but am getting conflicting answers with a friend. I was just hoping someone could verify the result I am getting.
Thank you for your time.

Question:

Find the derivative of function [MATH]f(x)=\sigma(wx+b)[/MATH] with respect to [MATH]x[/MATH], where [MATH]\sigma(z)=\frac{1}{1+e^{-z}}[/MATH]
My Solution:

[MATH]f(x)=\frac{1}{1+e^{-(wx+b)}}[/MATH]
[MATH]\Rightarrow \frac{\partial{f(x)}}{\partial{x}}= \frac{\partial}{\partial{x}}(1+e^{-(wx+b)})^{-1}[/MATH]
[MATH]\Rightarrow \frac{\partial{f(x)}}{\partial{x}}=-(1+e^{-(wx+b)})^{-2}\cdot \frac{\partial}{\partial{x}}(1+e^{-(wx+b)})[/MATH]
[MATH]\Rightarrow \frac{\partial{f(x)}}{\partial{x}}=-\frac{(\frac{\partial}{\partial{x}}(1)+\frac{\partial}{\partial{x}}(e^{-(wx+b)})}{(1+e^{-(wx+b)})^2}[/MATH]
let [MATH]u=g=−b−wx[/MATH] and [MATH]f(u)=e^u.[/MATH]
[MATH]\Rightarrow \frac{\partial{f(x)}}{\partial{x}}=-\frac{\frac{\partial}{\partial{u}}(e^{u})\cdot\frac{\partial}{\partial{x}}(-b-wx)}{(1+e^{-(wx+b)})^2}[/MATH]
[MATH]\Rightarrow \frac{\partial{f(x)}}{\partial{x}}=-\frac{-we^{-(wx+b)}}{(1+e^{-(wx+b)})^2}[/MATH]

[MATH]\boxed{\Rightarrow \frac{\partial{f(x)}}{\partial{x}}=\frac{we^{-(wx+b)}}{(1+e^{-(wx+b)})^2}}[/MATH]
Click to expand...
I get the same result, more easily, by using the chain rule after finding \(\sigma'(z)\).

The partials are not needed, and you should not use f to mean two different things, but otherwise your work looks correct.
 
Dr.Peterson said:
I get the same result, more easily, by using the chain rule after finding \(\sigma'(z)\).

The partials are not needed, and you should not use f to mean two different things, but otherwise your work looks correct.
Click to expand...
Much appreciated and thank you for the advice.
 
Why partials? Your last line (the one in the box) is off by a negative sign.
 
Jomo said:
Why partials? Your last line (the one in the box) is off by a negative sign.
Click to expand...
I am aware of the partials I would fix it if I could edit the post. How is it off by a negative sign?
 
illo said:
I am aware of the partials I would fix it if I could edit the post. How is it off by a negative sign?
Click to expand...
It isn't. Jomo needs better glasses, to see both negatives on the line above it. (I momentarily made the same mistake when I first read it.)
 
Awesome solution. I think that you are studying complex analysis.
 
Updated solution with corrections suggested by Dr.Peterson

[MATH] f(x)=\frac{1}{1+e^{-(wx+b)}}\\ \Rightarrow f'(x)= \frac{d}{d{x}}(1+e^{-(wx+b)})^{-1}\\ \Rightarrow f'(x)=-(1+e^{-(wx+b)})^{-2}\cdot \frac{d}{d{x}}(1+e^{-(wx+b)})\\ \Rightarrow f'(x)=-\frac{(\frac{d}{d{x}}(1)+\frac{d}{d{x}}(e^{-(wx+b)})}{(1+e^{-(wx+b)})^2} \\ \space \text{let} \space u=g=−b−wx \space \text{and} \space h(u)=e^u \\ \Rightarrow f'(x)=-\frac{\frac{d}{d{u}}(e^{u})\cdot\frac{d}{d{x}}(-b-wx)}{(1+e^{-(wx+b)})^2}\\ \Rightarrow f'(x)=-\frac{-we^{-(wx+b)}}{(1+e^{-(wx+b)})^2}\\ \Rightarrow\boxed{ f'(x)=\frac{we^{-(wx+b)}}{(1+e^{-(wx+b)})^2}} [/MATH]
 
illo said:
Question:

Find the derivative of function [MATH]f(x)=\sigma(wx+b)[/MATH] with respect to [MATH]x[/MATH], where [MATH]\sigma(z)=\frac{1}{1+e^{-z}}[/MATH]
Click to expand...
Here is my solution:

[MATH]\sigma(z)=\frac{1}{1+e^{-z}} = (1+e^{-z})^{-1}[/MATH][MATH]\sigma'(z)=-(1+e^{-z})^{-2}(-e^{-z}) = \frac{e^{-z}}{(1+e^{-z})^{-2}}[/MATH]
[MATH]f(x)=\sigma(wx+b)[/MATH][MATH]f'(x)=\sigma'(wx+b)\frac{d}{dx}(wx+b) = \frac{e^{-z}}{(1+e^{-z})^{-2}}(w) = \frac{we^{-z}}{(1+e^{-z})^{-2}}= \frac{we^{-(wx+b)}}{(1+e^{-(wx+b)})^{-2}}[/MATH]
This requires a lot less juggling.
 
Dr.Peterson said:
Here is my solution:

[MATH]\sigma(z)=\frac{1}{1+e^{-z}} = (1+e^{-z})^{-1}[/MATH][MATH]\sigma'(z)=-(1+e^{-z})^{-2}(-e^{-z}) = \frac{e^{-z}}{(1+e^{-z})^{-2}}[/MATH]
[MATH]f(x)=\sigma(wx+b)[/MATH][MATH]f'(x)=\sigma'(wx+b)\frac{d}{dx}(wx+b) = \frac{e^{-z}}{(1+e^{-z})^{-2}}(w) = \frac{we^{-z}}{(1+e^{-z})^{-2}}= \frac{we^{-(wx+b)}}{(1+e^{-(wx+b)})^{-2}}[/MATH]
This requires a lot less juggling.
Click to expand...
Oh wow, very clean solution thank you for sharing. There is just one small "slip of the pen" with the denominator raised to the [MATH]-2[/MATH]
 
You must log in or register to reply here.
Top