Understanding forward and backprop in neural networks

Some of the essences of neural networks.
convnet is a famous example of feed-forward network.
$$f(x)=\frac{1}{1-e^{-x}}
$$

$$\begin{align}
f’(x) &= -\frac{e^{-x}}{(1-e^{-x})^2}\
&= \frac{1}{1-e^{-x}} \cdot (1 - \frac{1}{1-e^{-x}})\
&= f(x)(1-f(x))
\end{align}$$

The same relation also holds for $\displaystyle f(x)=\frac{1}{1+e^{-x}}$

For deep learning model containing tens of layers, activation functions should be selected very carefully to prevent the problems of gradient bombing or disappering.