Understanding forward and backprop in neural networks

Some of the essences of neural networks.
convnet is a famous example of feed-forward network.

$$f(x)=\frac{1}{1-e^{-x}}$$ $$\begin{align*} f'(x) &= -\frac{e^{-x}}{(1-e^{-x})^2}\\ &= \frac{1}{1-e^{-x}} \cdot (1 - \frac{1}{1-e^{-x}})\\ &= f(x)(1-f(x)) \end{align*}$$

The same relation also holds for $\displaystyle f(x)=\frac{1}{1+e^{-x}}$

For deep learning model containing tens of layers, activation functions should be selected very carefully to prevent the problems of gradient bombing or disappering.