What is cost function?
A cost function is a function that needs to be minimized or maximized in an optimization algorithm in machine learning. Gradient descent is a basic algorithm used to minimize or maximize the cost function.
Types of Cost Functions
MSE(Mean Squared Error) : $\frac{1}{N} \sum_{i=1}^{N}(y_{i} - \hat{y_{i}})^{2}$
MAE(Mean Absolute Error) : $\frac{1}{N} \sum_{i=1}^{N}|y_{i} - \hat{y_{i}}|$
Binary Cross-entropy(log loss) : $-\frac{1}{N} \sum_{i=1}^{N}(y_{i}\log(\hat{y_{i}}) + (1-y_{i})\log(1-\hat{y_{i}}))$
Multinomial-log loss : $-\frac{1}{N}\sum_{i=1}^{N}\sum_{i=1}^{K}(y_{i,k} \times log(\hat{y_{i,k}}))$
where $y_{i}=$ Ground Truth, $\hat{y_{i}}=$ Expected Value