*chevron_left*Activation functions

# Comprehensive Guide on Sigmoid Function

*chevron_right*

*chevron_right*

*schedule*Jul 17, 2022

*toc*Table of Contents

*expand_more*

# What is the sigmoid function?

The sigmoid function, often denoted as $\sigma(x)$, is defined as follows:

The sigmoid function is named as such because it is S-shaped, as can be seen from its graph:

Note the following:

the domain of the sigmoid function is unbounded from $-\infty$ to $\infty$.

the range is bounded between $0$ and $1$; the output can become extremely close to $0$ or $1$, but it will never reach the boundaries.

# Derivative of Sigmoid Function

The beauty of the sigmoid function is that its derivative is extremely clean:

This is a very useful property because we often encounter situations in machine learning where we need to compute the derivative of the sigmoid function. For instance, the update rule for gradient descent in logistic regression requires this derivative.

The proof does not require any advanced calculus:

## Graph of Sigmoid function and its derivative

Here's a graph of the sigmoid function and its derivative:

# Implementation using Python

Here's the sigmoid function as well as its derivative implemented in Python using NumPy:

```
```